Part 1 (re: Architectural Decisions) I was recently asked about the architectural decisions we’ve made in our most recent Ruby on Rails project (an e-commerce platform for sports apparel decoration).
We decided to stay close to Rails’ conventions as much as possible. This allowed the team to build out the first features quickly. This is usually not a problem for short-term projects, but it becomes a maintenance burden after a year, because after a year the team is also concerned about supporting what they’ve built despite changes to team structure or project focus.
Project-specific conventions
In some areas where Ruby on Rails has no opinion, we’ve established our conventions:
Using service classes (and using LightService)
Using namespaces to partition functionality (e.g., Dealers, Admin, Checkouts)
Avoiding the use of JavaScript-heavy frameworks (e.g., VueJS) in favor of Stimulus and HTML-over-the-wire (this was done pre-Hotwire)
Prefer application events over ActiveRecord callbacks when triggering jobs
Adopting Spree as the e-commerce engine instead of building from scratch
Conventions distilled into pattern
We also wanted to make it easy for a new developer to understand how a feature was built and where to make subsequent changes. Having these patterns in place and adopted across the entire codebase also helps the maintainers to troubleshoot issues two years after a feature has been deployed to production.
I want to upload a CSV file from a page, and I also want to resist the urge to install a gem, which seems to be the default Rails developer behavior. No need to build a model; I just need a reference for the CSV for further processing somewhere. Here’s what I did:
First, I built a simple page with a form and a submit button. Ruby on Rails provides some helper methods to generate forms, namely form_with, form_for, and form_tag. I don’t need a model, so I used `form_tag` first. Unfortunately, I ran into this problem where the CSV file handle is not passed to the request. Instead, I used form_with and I was able to parse the CSV.
It is never too late for TIL (Today I Learned) type of blog posts. I hesitated making these types of posts because I can save the StackOverflow URL anyway. Unfortunately, now I have 12 years worth of StackOverflow URLs saved in my computer and I have just started to curate.
In 2020, I implemented a backend process responsible for paying out sellers in an e-commerce platform. This consists of two distinct phases: (1) calculating the weekly payout amount and (2) transferring funds using Stripe.
Calculating the payout amount works by accounting for the previous week’s sales and deducting fees and refunds. After computing the payout amount, a request is made to transfer funds, which happens some time later. A summary report containing all relevant information is also generated and sent out to sellers.
The initial version of the payout process did not consider errors occurring while calculating the payout amount or transferring funds. This made debugging challenging, sometimes months after a transfer has been completed.
Improving payouts
I wanted to enhance the payout process by adding well-known checkpoints: (1) when the process has successfully computed the payout amount, (2) when the process has determined that the platform has enough funds to transfer, and (3) when the funds have been transferred to sellers successfully. Along the way, errors could occur and we also need to be aware of these errors (and provide the necessary manual intervention).
One approach would be to use a state machine, but I needed something that could capture the a payout process’ journey through the checkpoints I’ve defined above. I also did not want to litter my ActiveRecord models with callbacks, because this becomes difficult to debug for various reasons.
I found this library called RailsEventStore that provides a way to define application events, publish these events, and subscribe to these events. This is made possible by having a single repository of events as a single table. Furthermore, RailsEventStore does not require any fancy storage backend. I was able to make this work using an existing PostgreSQL database (event storage) and Redis (publish-subscribe).
Domain events
A domain event is a record of a fact occurring in some part of a software system. An event could be something like “order has been confirmed” or “customer has signed-up for an account”. Other parts of a large software system could listen to these events and perform additional work (e.g. send emails, compute a rollup table, etc.). What events provide is a way to decouple these side-effects from the main task of some feature.
Example domain events
I’ve defined several events specific to payouts (e.g., PayoutComputed, FundsTransferCompleted, PayoutSendingSuccessful, etc.). I also defined events to capture error conditions (e.g. FundsTransferFailed, etc.)
When an error occurs, I’ve setup a subscription to the FundsTransferFailed event, which kickstarts an ActiveJob to send the necessary alerts.
The listing below shows how an event is published (ignore the Honeycomb span blocks):
A simple audit trail
In order to trace what happened for a particular payout run, RailsEventStore provides a way to enumerate a stream of events (I organized mine by payout run using an ID). This gives me a time-ordered list of events and the parameters passed for each event.
I’ve watched the Kuby talk at Railsconf, and I have some opinions on Active Deployment. The talk attempts to provide a standard way of deploying Ruby on Rails using a cloud platform (Kubernetes). The talk also mentioned Capistrano and its still-relevant role in the community.
This left me with a question: if you plan to deploy a Rails application to production in 2022, are you now forced to use Kubernetes?
Over the years, I’ve been deploying Ruby on Rails applications on different types of infrastructure: virtual servers, managed platforms-as-a-service, and lately, Kubernetes clusters. Each type has its own strengths and weaknesses.
I think one way to help decide is how much resources in terms of skill, time, and money you can afford to spend on not just bringing up infrastructure but also sustaining its operation over time.
The talk mentioned several disadvantages of using Capistrano to deploy; one still needs to set up the software dependencies before successfully deploying a Rails application. I think this upfront cost is a valid trade-off in exchange for possibly cheaper operating costs (running a single server to run the whole stack versus provisioning multiple servers and running Kubernetes on top).
I’ve had two reasons to use Kubernetes in deploying a Rails application: (1) multitenancy, where multiple instances of the same application will be deployed, and (2) adopting a container-based delivery pipeline, which eliminates a whole class of problems (e.g., dependencies) in exchange for a different set of problems.
I wish that we keep our opinions on Active Deployment open. Not everyone needs a Kubernetes cluster to deploy, and there’s still value in keeping Capistrano around. There’s always a hosted platform, such as Heroku (and similar offerings), to ease the deployment burden for a price.
These are some of my notes on David Copeland’s interview on September 2022.
Pragmatic Programmers (PragProg) hosted an interview with David Copeland, author of Sustainable Web Development with Ruby on Rails. I bought the early edition of this book two years ago and found the lessons there to be valuable. The interview differentiates sustainability from scalability (a common theme used to evaluate a web development framework).
Sustainability refers to the ease of which changes to the application can be made over time. The book assumes that the application needs to survive over several iterations, which may span more than one year. In order to keep development sustainable, the application needs to be structured to allow for changes to be done easily, despite changes to the people working on them and the environment that it runs on.
Part of what makes an application easy to change is having tests to ensure that any code that was built before stays working. This assumes that the application has already settled on its primary purpose.
Adopting ease of change to promote sustainability also puts less focus on predicting future needs. The author advocates to only build software for today’s needs. However, certain decisions require some forward thinking (e.g., database structure and API endpoints) because these types of changes are much harder to roll back once released.
David also mentioned the role of models in safeguarding the database’s integrity. Having a collection of models act as a gatekeeper to the database prevents untoward data incidents (e.g., missing customer data). Losing or damaging data is far worse than losing the application, as the database tends to survive the application that uses it.
Nestor G Pestelos Jr (ngpestelos) 