I needed to filter data when accessing an API following the JSON:API specifications. For those not familiar with JSON:API, it provides a set of conventions on how to structure JSON responses. It also provides some guidance on how to request data. Similar to semantic versioning, the set of conventions cuts down on the debate on how to structure APIs.
Filtering by Passing Query Parameters
JSON:API describes how to filter parameters, through the use of a filter parameter.
Example (fetching all posts made by author Joe):
http://example.com/api/posts?filter=[author][joe]
What I don’t see in the examples is how to pass in a list of values. For example, filtering by multiple authors. For now, I’ll assume that it is acceptable to pass in a list of values:
I have a dynamic list of attributes that I needed to set values to an object. Ruby provides a send method to bypass the usual dot notation for invoking methods.
It is never too late for TIL (Today I Learned) type of blog posts. I hesitated making these types of posts because I can save the StackOverflow URL anyway. Unfortunately, now I have 12 years worth of StackOverflow URLs saved in my computer and I have just started to curate.
I’ve been waking up early for the past ten years. This habit has carried over, and has served me well, now that I have children. I sleep early and wake up a few hours before everyone else in my household. I use this time to work on what I deem to be the hard stuff.
Everyone struggles with controlling attention, so try not to get distracted on the first hour of waking up.
It helps to have something difficult lined up, such as going to the gym (or going for a walk indoors when its raining). Also, it helps to have something easy enough lined up, such as a failing test from yesterday’s work session.
Certain tasks, such as planning and reflection, require some headspace to be effective. I do these things early in the morning, too.
In 2020, I implemented a backend process responsible for paying out sellers in an e-commerce platform. This consists of two distinct phases: (1) calculating the weekly payout amount and (2) transferring funds using Stripe.
Calculating the payout amount works by accounting for the previous week’s sales and deducting fees and refunds. After computing the payout amount, a request is made to transfer funds, which happens some time later. A summary report containing all relevant information is also generated and sent out to sellers.
The initial version of the payout process did not consider errors occurring while calculating the payout amount or transferring funds. This made debugging challenging, sometimes months after a transfer has been completed.
Improving payouts
I wanted to enhance the payout process by adding well-known checkpoints: (1) when the process has successfully computed the payout amount, (2) when the process has determined that the platform has enough funds to transfer, and (3) when the funds have been transferred to sellers successfully. Along the way, errors could occur and we also need to be aware of these errors (and provide the necessary manual intervention).
One approach would be to use a state machine, but I needed something that could capture the a payout process’ journey through the checkpoints I’ve defined above. I also did not want to litter my ActiveRecord models with callbacks, because this becomes difficult to debug for various reasons.
I found this library called RailsEventStore that provides a way to define application events, publish these events, and subscribe to these events. This is made possible by having a single repository of events as a single table. Furthermore, RailsEventStore does not require any fancy storage backend. I was able to make this work using an existing PostgreSQL database (event storage) and Redis (publish-subscribe).
Domain events
A domain event is a record of a fact occurring in some part of a software system. An event could be something like “order has been confirmed” or “customer has signed-up for an account”. Other parts of a large software system could listen to these events and perform additional work (e.g. send emails, compute a rollup table, etc.). What events provide is a way to decouple these side-effects from the main task of some feature.
Example domain events
I’ve defined several events specific to payouts (e.g., PayoutComputed, FundsTransferCompleted, PayoutSendingSuccessful, etc.). I also defined events to capture error conditions (e.g. FundsTransferFailed, etc.)
When an error occurs, I’ve setup a subscription to the FundsTransferFailed event, which kickstarts an ActiveJob to send the necessary alerts.
The listing below shows how an event is published (ignore the Honeycomb span blocks):
A simple audit trail
In order to trace what happened for a particular payout run, RailsEventStore provides a way to enumerate a stream of events (I organized mine by payout run using an ID). This gives me a time-ordered list of events and the parameters passed for each event.
You may have had a few ideas on creating “reusable” components to “save labor” and “deliver quickly.” Inspired, you decided to set aside some time to build your idea, and you were able to build a library that resembles the first iteration of your vision. You then proceeded to “encourage” your team members to adopt this new “framework,” which they have started to adopt in their new projects. Unfortunately, other concerns beckoned, and you had to pause development and work on something else.
Meanwhile, your teammates have started to prepare feedback on this new library, but no guidance was forthcoming. Realizing they were on their own with regards to using this new library “to deliver quickly,” they proceeded to put workarounds to get the new library to work the way it needs to in the real world.
The new library aimed to “save labor” and “deliver quickly” ended up costing time in terms of training and workarounds (and you will hear comments like “it sounded like a good idea at the time”).
Remember: just because you can, doesn’t mean you should.
We woke up at 5:00am to get ready for our return to Manila. We took an Uber to the bus terminal (nobody wants to walk that distance again). Tokyo is designed for travelers and removes much of the friction of going about.
ANA Flight from HND to MNL
Turbulent flight on the way back to Manila.
NAIA Terminal 3
I almost forgot how crowded it was at the arrival section: men wearing work-variety barong standing around, people selling real estate and cellphone load, and a duty free stall that did not make sense to me.
To kill time before the Ghibli schedule, we went to Tokyu Hands to buy small stuff. One group went to Harajuku. We re-convened at the JR station (this station is huge) before heading to Ghibli.
StudioGhibli
The trip to Ghibli was worth it. The first exhibit had a zoetrope, which for some reason had brought such joy when I saw it (a robot with pigeons flying).
The museum had so much attention to detail, even the diaper changer had to be on theme.
One section of the museum was devoted to the craftsman’s workshop, where several drafts are shown. One room was specifically for food and how the animators studied how to draw food by simulation (they had lifelike replicas).
We were able to make it to the 9:00 bus for the airport.
Sapporo to Tokyo
Glad we came to the airport early because the check in line is long. It took us more than one hour to check in and get past security.
The food selection past the security line is not so good. Only coffee and pastries are being sold. There will be a lot of dinner options once we settle in Shinjuku.
Turbulent ride on our flight to Tokyo.
Shinjuku
There’s a stark contrast with the number of people on the streets of Shinjuku vs Sapporo. We took the long route and walked with our luggage to the Airbnb place (2 kilometres away). They’ve ignored my suggestion of taking a taxi/Uber. I was a bit annoyed and was in a bad mood for a good part of the evening.
We had dinner at Kabukicho (took and Uber to get there). My wife and I retraced the places we’ve visited in 2015.
Driving in the Philippines teaches you a lot of bad habits such as swerving. We returned the car without incident (hooray!). We were too lazy to go to the gas station so we ended up paying a premium for gas.
Thinking of getting a fast-charge power bank (Anker) this afternoon (update: it cost me 5k yen).
We bought tickets to an onsen, which includes a round-trip bus pass. The onsen experience was worth the long ride. We had authentic Indian lunch after our bath. The resort had an outdoor pool where you can sit naked in hot water while snowing. Very refreshing.
Went to Sapporo Beer Museum and had dinner at the beer garden (steak). Sapporo also produces a good stout beer. Today is my brother-in-law’s birthday.
We took a taxi back to the train station. I have a slight buzz after drinking a litre of beer. We were lucky to catch the 6:30pm shuttle back to the hotel.
Nestor, WTF?!? 