Context
In 2020, I implemented a backend process responsible for paying out sellers in an e-commerce platform. This consists of two distinct phases: (1) calculating the weekly payout amount and (2) transferring funds using Stripe.
Calculating the payout amount works by accounting for the previous week’s sales and deducting fees and refunds. After computing the payout amount, a request is made to transfer funds, which happens some time later. A summary report containing all relevant information is also generated and sent out to sellers.
The initial version of the payout process did not consider errors occurring while calculating the payout amount or transferring funds. This made debugging challenging, sometimes months after a transfer has been completed.
Improving payouts
I wanted to enhance the payout process by adding well-known checkpoints: (1) when the process has successfully computed the payout amount, (2) when the process has determined that the platform has enough funds to transfer, and (3) when the funds have been transferred to sellers successfully. Along the way, errors could occur and we also need to be aware of these errors (and provide the necessary manual intervention).
One approach would be to use a state machine, but I needed something that could capture the a payout process’ journey through the checkpoints I’ve defined above. I also did not want to litter my ActiveRecord models with callbacks, because this becomes difficult to debug for various reasons.
I found this library called RailsEventStore that provides a way to define application events, publish these events, and subscribe to these events. This is made possible by having a single repository of events as a single table. Furthermore, RailsEventStore does not require any fancy storage backend. I was able to make this work using an existing PostgreSQL database (event storage) and Redis (publish-subscribe).
Domain events
A domain event is a record of a fact occurring in some part of a software system. An event could be something like “order has been confirmed” or “customer has signed-up for an account”. Other parts of a large software system could listen to these events and perform additional work (e.g. send emails, compute a rollup table, etc.). What events provide is a way to decouple these side-effects from the main task of some feature.
Example domain events
I’ve defined several events specific to payouts (e.g., PayoutComputed, FundsTransferCompleted, PayoutSendingSuccessful, etc.). I also defined events to capture error conditions (e.g. FundsTransferFailed, etc.)
When an error occurs, I’ve setup a subscription to the FundsTransferFailed event, which kickstarts an ActiveJob to send the necessary alerts.
The listing below shows how an event is published (ignore the Honeycomb span blocks):
A simple audit trail
In order to trace what happened for a particular payout run, RailsEventStore provides a way to enumerate a stream of events (I organized mine by payout run using an ID). This gives me a time-ordered list of events and the parameters passed for each event.
See also
Nestor, WTF?!? 