Dimiter Petrov

"We don't deploy on Fridays"

2024-11-06T00:00:00Z

Hearing "we don't deploy on Fridays" makes me sad, because I always encounter it as a precautionary measure.

Fear and precaution

The subtext is "it's not safe to deploy" and "if there is a problem, it takes too long to fix".

There is plenty out there about this topic. Search for "deployment on Friday". I almost forgot I've also written about this before. The arguments are more or less the same. If you are afraid to deploy, it may be because:

you have insufficient monitoring and alerting; or
the deployment process is error-prone; or
deployments, and thus rollbacks, are too slow; or
if something breaks, you don't know how to fix it; or
risky changes are deployed all at once (vs using feature flags, gradual roll-outs, etc.).

Those are all valid reasons. But they're also valid reasons on a Monday morning. So, do address them.

Valid reasons to not deploy on Fridays

Still, I'd love to hear no-deployment Fridays associated with joy rather than fear.

On a literal level not everyone works on Friday. A Monday-to-Friday workweek is not universal.

That got me thinking: what if the work culture made it so that there is nothing to deploy on Fridays?

We don't deploy on Fridays, because we work a 4-day week and take Friday off (but we do deploy on Thursdays!).

We don't deploy on Fridays, because we contribute to open source.

We don't deploy on Fridays, because we conduct interviews.

We don't deploy on Fridays, because we write blog posts.

We don't deploy on Fridays, because we check and update documentation.

We don't deploy on Fridays, because we work on making our test suite faster.

We don't deploy on Fridays, because the developers all do customer support to get a better understanding about how the product is used.

We don't deploy on Fridays, because Fridays are a dedicated learning day.

How I test Stimulus controllers

2024-07-02T00:00:00Z

I caught myself relying too much on manual tests for a feature I was building on top of a Stimulus controller. Manual tests were enough for the initial proof of concept phase. However as I was adding more features to the prototype, I was also causing regressions.

As I threw away the prototype and started over with a test-driven approach, I thought about how I test in general and how that applies to Stimulus.js

No jsdom-based tests with sample HTML

I'll start by stating how I do not test Stimulus controllers.

It would be possible to attach the controller to some fixture HTML and test things on top of jsdom. That approach would be reasonable for a library. But I'm not packaging a stimulus controller. I'm using it directly in my application.

In applications, the mistakes I commonly make with Stimulus controllers are all in the integration with the page. Maybe I forgot to set a target, or I have a typo in a value, or it's not listening to the right event.

Because these mistakes can happen at every use of a controller, I could get it right in the test code, but wrong in the production.

1-2 integration tests with Capybara (Rails system test)

What I do instead is test the functionality from a user's perspective in a Rails system test. The controller is an implementation detail that is tested implicitly.

Unit tests

Think about a Rails controller. With a handful of code paths, testing the controller is not much of a problem. But the more code paths and edge cases there are, the harder it becomes to prepare the initial test state for each scenario. Not to mention that all these tests exercise the request/response stack and are slow in aggregate.

Usually, with a Rails controller, I extract a domain object and put the business logic there. The controller is left with HTTP concerns: parsing parameters and headers for the request, setting headers and rendering the response.

A few tests would cover the integration between the domain object and the controller. The majority of tests test the domain object in isolation. Given these inputs, it has those outputs and/or side-effects.

I see a Stimulus controller the same way. It excels at being the glue between the HTML document and the business logic. It allows easily subscribing to events, reading properties and setting properties in the DOM. The core business logic doesn't need to directly depend on the DOM. It can be extracted and tested in isolation.

An example with a music player

Consider a music player on a web page. Browsers have a default user interface for audio and video elements, but they vary between browsers. We want the player to follow the brand guidelines. Also, the player can play playlists (say that five times faster). There is no web standard for that, so we have to implement it ourselves.

I like two things about this example. First, it's extracted (and slightly simplified) from a real application. Second, developers tend to not test things when it looks too hard and testing audio playback seemed too hard to do automatically to me at first.

How do I break down this feature and test it?

The HTML

The basis is a <video> element. That can play both audio-only and video files. The native player controls are hidden, but there are custom controls for play/pause as well as going to the previous and next track in a playlist.

Here are the relevant bits of the code:

<div data-controller="player">
  <video
    data-player-target="player"
    data-action="
      playing->player#playing
      pause->player#paused
      ended->player#ended
      playable:play@window->player#play
    "
    data-testid="player">
  </video>

  <div>
    <button data-action="player#previous">
      Previous
    </button>

    <button data-player-target="playButton" data-action="player#resume">
      Play
    </button>

    <button data-player-target="pauseButton" data-action="player#pause" class="hidden">
      Pause
    </button>

    <button data-action="player#next">
      Next
    </button>
  </div>
</div>

Notice the video element is a Stimulus target. It also has event listeners attached. These allow us to keep the player controls in sync with the player state.

Separation of concerns with cross-controller communication

Playback starts when pressing the play button of a song or a playlist on the page. There's a separate controller that dispatches a custom event. The following Stimulus action is what listens to play request events and reacts by actually starting the payback:

data-action="playable:play@window->player#play"

This follows the pattern for Stimulus.js cross-controller communication with events.

In order to keep the example focused, I won't show this controller's code. The cross-controller communication is covered by the system test that I show at the end.

The player controller

The player controller looks roughly like this:

import { Controller } from "@hotwired/stimulus"
import PlayableList from "../player/playable_list"

export default class extends Controller {
  static targets = ["playButton", "pauseButton", "player"]
  static classes = ["hidden"]

  connect() {
    this.playableList = new PlayableList()
  }

  /*
   * API
   */

  play(event) {
    this.playableList.playNow(event.detail)
    this.switchTo(this.playableList.currentTrack)
  }

  resume() {
    // ...
  }

  pause() {
    // ...
  }

  next() {
    let old = this.playableList.currentTrack
    this.playableList.skipForward()
    if (old !== this.playableList.currentTrack) {
      this.switchTo(this.playableList.currentTrack)
    }
  }

  previous() {
    // ...
  }

  switchTo(track) {
    this.playSource(track.source)
  }

  playSource(source) {
    this.playerTarget.src = source
    this.playerTarget.play().catch((error) => this.handleError(error))
  }

  handleError(error) {
    this.playerTarget.pause()
  }

  /*
   * Events
   *
   * Named after HTMLMediaElement events
   * https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement#events
   */

  playing(event) {
    this.hidePlayButton()
    this.showPauseButton()
  }

  paused(event) {
    this.showPlayButton()
    this.hidePauseButton()
  }

  ended(event) {
    this.showPlayButton()
    this.hidePauseButton()
    this.next()
  }

  // https://developer.mozilla.org/en-US/docs/Web/API/MediaError
  error(event) {
    handleError(event.error)
  }

  /*
   * User interface
   */

  hidePlayButton() {
    // ...
  }

  showPlayButton() {
    // ...
  }

  hidePauseButton() {
    // ...
  }

  showPauseButton() {
    // ...
  }
}

The core business logic and its tests

The player controller can hold player state, but it doesn't have to know the implementation details. Those details are in what I call here PlayableList, which controls moving back and forth in the list of tracks.

export default class PlayableList {
  constructor({ history = [], current = null, queue = [] } = {}) {
    this.history = history
    this.current = current
    this.queue = queue
  }

  skipBackward() {
    // ...
  }

  skipForward() {
    // ...
  }

  playNow(playable) {
    // ...
  }

  get currentTrack() {
    // ...
  }
}

It's precisely thanks to that abstraction that we can now easily unit test. Look at the following test for example.

import PlayableList from "player/playable_list"
import Track from "player/track"

describe("PlayableList", () => {
  test("skipping to next then previous returns to the initial item", () => {
    let track1 = new Track({title: "track1"})
    let track2 = new Track({title: "track2"})
    let list = new PlayableList({
      current: track1,
      queue: [track2]
    })

    list.skipForward()
    list.skipBackward()

    expect(list).toEqual(new PlayableList({
      history: [track1],
      current: track1,
      queue: [track2]
    }))
  })
})

The assertion is comparing two data structures (Jest's toEqual matcher does a "deep" equality check).

The JavaScript unit tests all look like that. They are fast, they are short, they are clear.

I used Jest for the JavaScript tests, but the test framework doesn't matter. I only needed to configure two things to make these unit tests run with Jest: the location of the test files (testMatch) and the location of the production code (moduleDirectories). This is my jest.config.js:

module.exports = {
  testMatch: ['**/test/**/*.test.js'],
  cacheDirectory: './tmp/cache/jest',
  transformIgnorePatterns: ['node_modules'],
  moduleDirectories: ['node_modules', 'app/javascript']
};

There's a common argument that modifying code just for the sake of testing is an anti-pattern. I have two counter arguments specific to this example:

When parts of the business logic are self-contained, it becomes easier to focus on the individual modules. I've hit enough edge cases that I was glad to be able to quickly modify that part of the code in isolation and test it in isolation.
An older version of this app had JavaScript controllers with a similar structure, but not powered by Stimulus.js, and the playlist management logic was intertwined in a controller. If the PlayableList abstraction had existed, it would have made the transition to Stimulus faster.

The system test

Now let's look at the system test. I've hard-coded some strings to make the excerpt easier to read outside of the application.

require "application_system_test_case"

class PlayerTest < ApplicationSystemTestCase
  driven_by :selenium, using: :headless_chrome

  test "playing a playlist" do
    visit '/playlists/1'
    find_button("Play Playlist 1").click
    find_button("Pause")
    player = find("[data-testid=player]")
    assert_playing(player)
    first_source = player["src"]
    find_button("Next").click
    assert_playing(player)
    assert first_source != player["src"]
  end

  def assert_playing(element)
    Timeout.timeout(Capybara.default_max_wait_time) do
      playing = false
      loop do
        playing = page.evaluate_script(<<~JS, element)
          (function(element) {
            return (
              element.duration > 0 &&
                 !element.paused &&
                 !element.ended &&
                 element.readyState > 2
            )
          })(arguments[0])
        JS
        break if playing
      end
      assert playing
    end
  end
end

It checks the common path of visiting a page and pressing play. It even tests pressing "Next" for good measure. If the workflow were even longer, I'd still put it all in one system test. I like it when a system test/feature test tells a complete story. I also like to avoid repeating expensive setup and teardown between tests when possible.

The usual advice about browser-based tests applies:

minimize race conditions by first waiting for things to be on the page, or be in a stable state
lean on Capybara's retry-aware finders
implement custom auto retry logic if really necessary

In the past, I've resorted to using Cypress for tests of complex JavaScript code. For this player system test, I tried Playwright directly in a Capybara test with the Playwright Ruby driver. In the end, I realized the Capybara evaluate_script helper is enough to query the player state. That's lucky, because sticking to the Rails defaults reduces the long-term maintenance burden.

Defaults matter

I think part of the reason most Rails projects lack JavaScript tests is lack of examples.

The Rails documentation includes a testing guide. Rails generators create test files and placeholders for Ruby tests. But there are no examples with JavaScript tests.

I hope my example above inspires you to write more JavaScript tests for your application.

Dealing with flaky tests

2024-05-27T00:00:00Z

Flaky tests waste your team’s time. Have a process for quarantining them and debugging them.

Service objects are poorly-named models

2023-12-11T00:00:00Z

Look at any Rails codebase that's older than a few months. I bet you'll find a sub-directory of app called services.

It comes with the best of intentions for separation of concerns. The developers want to keep controllers straightforward and focused on understanding a request and providing a response. They also want to keep models small, because those files have gotten too big and tedious to navigate and modify.

What are service objects anyway?

I think sometimes it's supporting code, "infrastructure" if you will. For example an HTTP client for an API. I am not going to say much about this type of code. Put that in a lib directory if you must, and go on with your day.

Sometimes a "service object" performs a series of steps, often with some conditional logic. Send a welcome email only if the user record was successfully created.

Other times it's seemingly a single operation, but with a long implementation hidden away in private methods. You didn't want those polluting your ActiveRecord model class, so you put them here.

At least that's how I've used and seen these service objects used. I've been writing Rails applications since 2011 and I still don't understand why we call this concept a service object. Maybe I should read the Domain Driven Design book.

Aside: suffixes for the sake of suffixes

Rails has a convention of suffixes. It's a UsersController, and a UserMailer, and an ArchivalJob. So developers try to follow that convention and name something UserSignupService. Again, because I don't understand what a service is, I keep reading the meaningless suffix everywhere. Why not call it UserSignup or Registration?

A representation of a business domain concept is called a model

Let's come back to my main point. I think we should stop thinking in terms of services and start thinking in terms of domain objects, otherwise known as models.

I admit "model" is a generic name in itself. However it is a core concept in Rails' Model View Controller architecture and it is widely understood.

A technique I use is to step back from the code and think about the noun or noun groups used to describe concepts in the domain of the application. That also works for processes or operations.

For example, when talking about sending notifications, the noun is "notification". A noun group that comes to mind is "notification delivery". Rather than a NotificationService or a NotificationManager, I define a Notification with a deliver method.

class Notification
  def initialize(title:, body:)
    # ...
  end

  def deliver
    # ...
  end
end

Depending on the domain, you can be more specific with the nouns. In case the delivery is a complex concept, I would extract a dedicated NotificationDelivery class. If there is something special about the formatting of the notification, that logic could go in a NotificationPayload.

I avoid the names with the shape of SomethingDoer or DoesSomething.

Naming is hard, but taking the time to pick a good name is worth it. You'll be able to explain the concepts more easily. The application routes will fall into place. The controllers will make sense. Watch In Relentless Pursuit of REST for some inspiration.

Aside: directory structure

I've tried adding nuanced directory structures in projects. For example having categories such as errors, forms, policies, presenters, serializers, validators, etc. I think some amount of organisation is good, but I've definitely overdone it and I tend to put a lot more in the models directory nowadays.

With fuzzy finders, go to definition, and other tooling, the location of a file in a project doesn't really matter. In fact, I don't use the project file tree view when programming.

app/services doesn't bother me from a discovery perspective. However, I think that the models directory encourages you to name the domain object in a better way. Another benefit is that you avoid having to choose where to save a new file.

If you do feel strongly about directory structure, I find namespaces to be a great way to keep related concepts together.

Namespaces help with organisation

Namespaces will not only lead you to better names, but also you'll start putting things that belong together close by in the file system. Here is a good example I've seen in production code.

The price of a line item depends on the product and product options. The code for calculating the price has become too large and complex, and the maintainers have extracted it from a method on the LineItem model to its own class.

class LineItem < ApplicationRecord
  def price
    LineItems::Price.new(self).calculate
  end
end

Now if you want to understand the pricing code, it's in one clearly delineated place. The implementation details are no longer mixed with other (private) methods of the LineItem class.

The code is also close by in the project structure.

line_item.rb
line_items/
    price.rb

What's powerful in this example is that you retain the interface of the model (line_item.price) while separating self-contained parts of the implementation. That makes the abstractions easier to discover. You're not wondering whether you can call a method on the model directly or you have to hunt down some class in app/services. It goes "along the grain" of Rails.

Continuing the example about notifications from earlier, the Notification model could be the public interface, but different aspects about notifications could be in models in the Notifications namespace. For example:

class Notification
  def initialize(title:, body:)
    @payload = Notifications::Payload.new(title: title, body: body)
  end

  def deliver
    Notifications::Delivery.new(@payload.format).perform
  end
end

To be clear, I would start with fewer objects, and only introduce the extra ones as the application complexity increases over time.

Try it out

The next time you're adding a new class, whether it's in your current codebase or a new project, consider the tips about naming and organisation above. See how it affects the code's design.

Have we moved past contempt culture?

2023-12-07T00:00:00Z

In 2015 I read a great blog post on contempt culture. I had already noticed the described behaviour by that point, but I didn't have a word for it. Interestingly, the author of the post mentions starting programming in 2001. I started around that time, too. Probably 2002. It took 13 years before I stumbled upon an article pointing out bad behaviour I had both experienced and perpetuated. That article made a strong impression on me. In an effort of self-reflection, I'm looking back at how I've changed since reading it.

Being on the receiving side of contempt

First, I would like to mention two examples: one more general, and one more personal.

The general one is about the recurring meme of "Ruby on Rails doesn't scale". I've been using Ruby since 2011 and I have seen so many iterations of this debate over the years that it's become boring to me. So much energy is spent arguing about the supposed demise of a technology. I keep myself out of this as much as possible.

Once a friend said they don't understand why I'm still working with Ruby. I answered something vague about work opportunities and a sense of belonging to a community. We then moved on from the topic. Oof, friendship saved.

The personal example is about being labeled. Around 2016 I got interested in the Elm programming language. I gave a presentation about it at work and later even built a project in Elm. Co-workers who didn't like the language were engaging me on the topic for a while. It became clear that we wouldn't use Elm for future work. There are many factors that can lead to the success or failure of a project. Can the team work well with the technology? Does the team communicate well? Does the team learn and continuously adapt its processes? For me it was much more important to be able to work together as a team, than to push for a technology that I personally liked. The programming language is just one tool.

Even after we moved on, the discussions continued. At a party, someone from another company was making a joke about Elm. The person next them nodded in my direction and said "shush, he's really into Elm". I didn't say anything, but I took a few things from the experience:

I resented being labeled as "the Elm person". Perhaps I came out as overly-enthusiastic about the language. I decided that from that point on, if I liked something, I would keep my thoughts to myself.
I don't like my identity being tied to a category. I don't see myself as a Ruby developer, or an Elm developer. I even avoid the general label of developer.
People have a preferred technology and love making fun of other technologies, especially when they have little experience with them.

The thing that twisted the knife was a conversation at an Elm conference later on. I was telling another attendee about my experience using Elm at work. They told me that I had not advocated for it enough. Why do we go to extremes? Why is it either the best language that we should spread everywhere, or the worst thing that we should mock out of existence?

Examining my own behaviour

The personal experience I described above was neither the first, nor the last. I've been jaded and disillusioned, to the point where I don't even call out contempt. However I try to convey a nuanced position when asked directly about a topic. Here are a few examples.

I once had to add a feature to a program written in PHP. I didn't know PHP, only the reputation of PHP. I chose to ignore that. I approached the project with an open mind. I studied the program and added tests to understand its behaviour better. I refactored some code once it had test coverage. Finally, I added the new feature and deployed. I never complained and even made a mental note of the language constructs in PHP that I wish Ruby had.

At the end of a job interview, I was once asked what I thought of CSS. I said it's tricky to get right despite having used it for a long time, and that I have a lot of respect for people who write it well. The interviewer proceeded to say how bad it was and how they avoided it at all cost. I guess they were looking to bond over bad-mouthing something. I was a little puzzled, but also proud that I had not engaged in that.

During the hype period of cryptocurrency (and later web3), colleagues tried to get me to work on projects in that domain, whether for side projects or new business ideas. I always declined politely. While I personally object to working on such projects, I don't impose my point of view to anyone. Someone pressed the issue once. I told them that the domain doesn't interest me and I prefer to focus on things that I find more exciting. That's the truth, and I think it also counteracts contempt culture. Keep to the things that interest you and stop talking about the things that don't.

Another example are the countless lunchtime conversations about programming languages at work over the years. I usually stay silent during those, because I'm tired by the repeated arguments. This is one point on which I could improve. I could steer the conversation towards the interesting things that the language everyone hates enables. I could ask how we can get those benefits in the technology that this group prefers.

When it comes to actively fighting contempt, I'm proudest of my handling of difficult codebases. On many projects I keep reminding my colleagues to look past the current limitations of the code, not to blame previous developers, and focus on how we can make progress as a team.

Helvetic Ruby 2023: a personal retrospective

2023-11-26T00:00:00Z

I ran the first edition of Helvetic Ruby with two friends. This post reflects my own experience and my personal thoughts on it.

Preparation

We started working on the conference early. Our first meeting was in early January 2023 and the conference was on 24th November 2023.

While there were periods of more intense preparation, we usually did a little bit every week. That kept stress levels low for me. I was not even stressed on the day of the conference.

In a way, it felt easy, though I am probably looking back at it through rose-tinted glasses.

The main reason is that we had a common vision of what we wanted to achieve. We sometimes disagreed on details, but always found a good solution after some discussion. I never felt ignored and I never wanted to say "I told you so". When we did something, it's because we agreed on it as a team. If there was a problem with the outcome, we learned from it together. I guess this is the elusive alignment that all leaders talk about.

Because we trusted each other, we could also take complementary responsibilities. We worked on the initial stages of each aspect together, e.g. the mix of topics we wanted for talks, or the sponsorship packages we offered. Once we confirmed the ideas, we executed them independently. A lot of the communication was about sharing encountered problems and sometimes asking for advice on them.

Schedule

We had planned breaks between most talks. The breaks were of variable length and the shortest ones were not long enough. 10-15 minutes barely sufficed for people to go the toilet and get a coffee. In comparison, the 30 minute breaks worked really well. I wonder how it would have been to have more groups of 2 back-to-back talks and a longer break afterwards.

These short breaks also made it harder to get people to sit down for the next talk. Perhaps, we could get a little kitsch next time and use a cow bell to signal the end of the breaks.

Despite that, we managed to stick to the schedule. I consider that a big success. My experience as a conference attendee is that usually everything gets delayed more and more throughout the day.

Venue

The venue felt cramped during the breaks. At the end of the day I heard a suggestion about removing the last 1-2 rows of chairs and encouraging people to sit at the balcony. (The room had a main floor and a balcony at the back).

I was absorbed by other things and did not think about this at all. I wish we had thought about adjusting the layout on the spot.

Emceeing

I was the MC in the morning, Hana in the afternoon.

I try to keep good posture when I'm giving a talk, but emceeing didn't feel like giving a talk and posture was completely out of my head while I was on stage. I only noticed my bad posture on pictures after the event.

I was comfortable on stage and people laughed at my jokes. I did feel inadequate when the room was too noisy and I couldn't get people's attention immediately. I guess this requires both practice and patience.

Talks

I think most people attended all talks. I suspect three reasons for that:

There was a single track.
People did manage to talk enough during the breaks.
The program was compelling.

I found that the order of the talks and the variety of topics worked well. It paid to be intentional and know what we wanted to achieve before even starting the call for proposals.

Multiple people mentioned to me that they liked the mix of topics and I am glad they validated our choices.

Next edition

It had been 10 years since I was last involved in organising a conference and this time I had much more to do. It was a great learning experience and I am glad to have started a new conference for my community. I'm excited to work on the next edition of Helvetic Ruby.

3 ways to model an associated collection in Rails

2023-11-23T00:00:00Z

A look at the Postgres array type and good old join tables, each with their advantages and disadvantages.

Turn off notifications

2023-07-26T00:00:00Z

Do you find catching up tedious when you return to work after a holiday? I do. It took me a while to do something about it. I'd even start dreading my return to work on the last day of holiday. On my first day back, I'd open my email inbox and see tens, if not hundreds, of new messages. Then dozens of notifications in Slack, and GitHub, and Trello, and so on.

In recent years, I've drastically reduced that anxiety by reducing the number of notifications. It's not perfect, and I'm still working on it, but it has served me well. Here's what I do. I hope it may help you, too!

Make it obvious you are unavailable

Set an email auto-responder. Update your status and/or name in the chat app to "out of office until...". Mark yourself as out-of-office in the calendar and automatically decline all invitations. Announce your absence to colleagues, customers, and whoever else might be trying to contact you.

This deals with humans. Now onto services and transactional emails.

Remove duplicate notifications

Turn off emails from third-party services. Most services already have in-application notification systems. If I first see the email and follow the link to the service, that usually clears the in-app notification. If I first see the notification in the app, the email is redundant and will take me time to process later for no good reason.

I used to think that disabling email notifications would help only during time-off, but I've come to appreciate it even during work. It's resulted in less mindless clicking through emails.

If you've been using email as your only inbox, disabling those notifications will break that process. That brings me to the next point.

Multiple inboxes, pull-based vs push-based notifications

I turned off emails from GitHub a decade ago. When I am ready to review code, I open the website and look at the notifications page for review requests. Going one step further, I might ignore notifications and go directly to the pull requests page of a repository and decide what to review there.

On projects using Trello, I've found the activity log of the board very valuable. It's always there and in chronological order, so I can catch up any time. That also makes notifications less important.

Overall, notifications introduce a sense of urgency. I get interrupted and feel the need to act on the new information. Switching to "pull-based" system removes the time component and gives me more control.

Identify what's worthy of your attention

You don't need to be part of every discussion and make every decision. I've found that "out of sight, out of mind" is a good counter to the fear of missing out.

Sometimes I reply to a Slack thread asking for my input, but many other people do as well. The replies and notifications continue well past the point of any of the new messages being relevant to me. The context menu has an action called "Turn off notifications for replies". After that, I am free.

Similarly, I may have contributed to a discussion on GitHub, but have nothing more to add, because others have it covered. I go to the notification settings of the issue or pull request and click "Unsubscribe". If my participation is still needed, someone will mention me in the discussion and I'll get a new notification.

You can mute or leave Slack channels you don't use. You can stop watching GitHub repositories you don't work on anymore. It's okay to let go.

Check notification settings for each new service you use

Over time you inevitably sign up for new services and the number of emails creeps up again. This is because every company insists on sending emails for every event. They also send marketing emails about product updates.

I've started checking the email settings of every app right after creating an account and disabling communication by email.

Mark all as read

I've rarely done that, because I'm afraid to miss something important. Then again, important things tend to re-surface.

Be considerate to others

Now that you're on top of your inbox(es), help others out. My strategy for that is delay and summarise.

Delay. Check if the person is working before sending them a message. Try to find a solution on your own. If you cannot, take notes and wait for the person to come back to discuss with them.

Summarise. When a colleague comes back to work, write down a few bullet points with the most important things that happened in their absence. Include links to more context if they want to dive into some of the topics. I like to mix all kinds of updates. Here is what that summary might look like, with placeholders:

[Person] joined the team

We found a solution to [problem that occurred before you left on holiday], here's a [link] to the details.

We are currently working on [new feature]

[Person] announced [news important to everyone in the company]

Further considerations

The accumulation of emails and other notifications during time-off was an obvious pain point. While avoiding that pain was the motivation of the actions I described, the outcome for me has been better focus on a daily basis.

There is more room for improvement. On days when I get too distracted at work, I completely close Slack and the Mail program. That obviously doesn't reduce the number of messages, but I can read them in batches, the same way I approach code review.

A further step in that direction would be to disable desktop notifications and only look at my inboxes a few times per day.

GitHub pull request merge strategies

2023-07-17T00:00:00Z

An overview of what the options provided by the pull request merge button on GitHub do and how to make the best out of each strategy

Rails developers write some Rust: a review of Axum 0.6

2023-01-16T00:00:00Z

Is Rust Web yet? We used our Rails knowledge to try writing a Web application in Axum in two days

Trying to prevent errors is good, being able to recover from errors is better

2022-12-05T00:00:00Z

I'm the person on your team who does not fear deploying half an hour before the end of the workday. Not because I'm reckless, but because I'm confident in the process of recovering from errors. I work in short iterations, developing and deploying small changes one at a time. I have automated tests, code review, production monitoring and error alerts. If the deployment breaks something, it's easy to revert the last commit and deploy again.

In my experience, a lot of web-based business software can benefit from more confidence in the release cycle. The more you give into fear of failure, the more you postpone releases. Eventually you are too afraid to break things so you change nothing in production. You test extensively on pre-production environments, which never quite match the real production environment. That takes a long time so your development cadence slows down. Releasing becomes a big event. The team is on edge, because the accumulated changeset is large and many things can go wrong.

The remedy to this downward spiral is accepting that things will eventually go wrong, no matter how hard you try to prevent that. Learning from incidents will teach you how to react to future incidents and you will become better at recovery.

Investing in prevention still plays as a big role. Type systems, automated tests, manual tests, feature flags... all of these tools are worth it. Just don't forget to plan for failure: improve observability, run incident recovery drills, write documentation. Having these conversations in your team and getting everyone involved will help with confidence, release speed and stress levels.

Thoughts on onboarding and offboarding

2022-11-16T00:00:00Z

When I'm joining a new organisation, I want to be useful as early as possible. I record what I observe until I get used to the new environment. Chances are my observations can turn into documentation or change of process that will make onboarding easier for the next newcomer. I want to prove myself, to show what I can do and be given responsibility.

When I'm onboarding somebody else, I want them to feel welcome and heard and seen, to show them we can work together well. I hope to gain their trust, so that they feel safe giving feedback.

When I'm offboarding someone, I need to pretend to not trust them, even though I do. I follow a checklist to revoke their access to services, information and infrastructure. I make sure that neither they, by negligence, nor a third party, by malice, can harm the organisation with forgotten accounts or elevated privileges.

When I'm offboarding myself, I try to share my knowledge well in advance, so that nobody needs me when I'm gone. I delete my own accounts and check that nothing breaks and nothing is lost. I give the team time to transition.

SVG icon sizing and alignment

2022-07-25T00:00:00Z

Making SVG icons look right alongside text in web applications can be tricky. I'm writing down the techniques that have worked well for me, because I tend to forget them by the time I need them again.

For me it's about:

preserving the icon's aspect ratio and internal spacing,
scaling the icon proportionally to the text,
vertically aligning the icon to the text.

Let's how these come together.

Icon proportions in isolation from accompanying text

The viewBox property defines the viewport into the SVG image. With it you can shift and crop as you like.

Consider a disc positioned at coordinates (10, 10) with a radius of 10.

<svg>
  <circle cx="10" cy="10" r="10">
</svg>

By varying the viewBox we can change the perceived zoom level and offset.

viewBox="10 5 15 15"

viewBox="-5 -5 30 30"

viewBox="0 0 20 20"

Icon designers visually balance this framing to achieve harmony within the icon set. For example the following icons come from the same set (Heroicons) but have different spacing.

Without the viewBox information, the SVG images may appear at the wrong dimension, e.g. with the wrong "zoom level" and/or the wrong aspect ratio. If your icons don't have this property consider adding it yourself. I recommend a square aspect ratio and starting with no space between the shape and its bounding box. Then you can tweak the spacing and alignment within the SVG.

Proportional text and icons

When it comes to icon size, I first set a fallback using the width and height attributes directly on the SVG. That way if the CSS doesn't load for some reason we don't end up with the browser default, which is to render SVG images really large. I set a reasonable size, keeping the square aspect ratio, for example width="20" height="20".

Then I override the size fallback with CSS rules (or utility classes) to make the SVG image take as much space as its parent element:

svg {
  height: 100%;
  width: 100%;
}

Now I can resize the icon as needed in different contexts without modifying the SVG source or targeting the icon specifically with CSS rules.

For example, if I'm making a form button containing the search icon:

I would wrap the icon in a div and set the div element's height (inline style for illustration here, I would do it differently in real application code).

<button type="submit">
  <div style="height: 1rem; width: 1rem;">
    <!-- search icon rendered here -->
  </div>
  Search
</button>

The key is to use units proportional to the text size, like em or rem, for the height of the wrapper element. That way the icon scales with the text.

Aligning icons with multi-line text

For the button above, I centered the icon and text vertically in the button using flexbox.

button[type="submit"] {
  display: inline-flex;
  align-items: center;
}

This works well with a single line of text, but may not be what you want when the text spans multiple lines.

In the following example, the icon is centered vertically.

Message title

A long and detailed message description.

Flex children aligned with align-items: center

What if we wanted to align it with the first line? In my opinion both align-items: start and align-items: baseline look off.

Message title

A long and detailed message description.

Aligned with align-items: start

Message title

A long and detailed message description.

Aligned with align-items: baseline

The trick is to have a sibling text node to which you can centrally align the icon (like in the single line case), but then use baseline alignment on the parent, so that the text from the two columns aligns. Let's visualize that before getting to the code:

reference

Message title

A long and detailed message description.

Two columns aligned with align-items: baseline and an icon aligned to the text in the first column with align-items: center

The icon is centrally aligned to the text which says "reference". "reference" is baseline-aligned to "Message title".

Of course you don't want actual text next to the icon. The trick is to use a zero-width space, which you can insert by writing its HTML entity: . Normal white space does not work because HTML parsers strip that.

Check out the simplified code. (Again the styles are inline to make it easier to understand what is going on)

<div style="display: flex; align-items: baseline">
  <div style="display: flex; align-items: center">
    &#x200B; <!-- zero-width space, for alignment -->
    <div style="height: 1em; width: 1em;">
      <!-- SVG icon goes here, omitted for brevity -->
    </div>
  </div>
  <div>
    <!-- text from the right column goes here -->
  </div>
</div>

Here's the result:

Message title

A long and detailed message description.

It's easier to compare with larger icons:

Message title

A long and detailed message description.

Aligned with align-items: baseline

Message title

A long and detailed message description.

Aligned with align-items: center at the inner level and align-items: baseline at the outer level.

In my projects, I usually encapsulate the necessary markup in a function or component. For example, I have this view helper in a Rails project (the utility classes are from Tailwind CSS):

def inline_icon(icon_name, class: 'w-4')
  capture do
    content_tag(:div, class: 'flex items-center') do
      concat '&#x200B;'.html_safe # zero-width space character to set the correct height (in combination with a 100% height on the svg icon itself)
      concat (content_tag(:div, class: binding.local_variable_get(:class)) do
        render "icons/#{icon_name}"
      end)
    end
  end
end

Working with constant interruptions

2022-05-12T00:00:00Z

In 2018 I chose to become an engineering manager. It was clear to me that good teams, like good code bases, require constant work. Only, I wasn't just interested in shaping my team. I wanted to influence how the company worked in general, all with the top-level goal of creating a good environment for myself.

I started getting involved in hiring, sales pitches, writing offers, strategic meetings and everything else that makes a software agency tick. Simultaneously I was on multiple client projects, leading one or two myself.

Unsurprisingly, focusing on anything for more than 2 hours became impossible. I somehow adapted and this is how I've been working ever since.

Why I'm writing this

I can't pretend that there is some wisdom to glean. Rather, I want to record where I stand after a few years of working in such a scattered way. I hope to reflect on that in the future and see what has changed.

Facts and feelings

These may not be all be true right now, but have occurred at various times.

It feels like I have very little time during a given day, yet I sometimes don't remember what I've done by the end of it.
I get interrupted a lot and I'm often multitasking.
I spend 4-5 hours per week in 1-on-1s and 3-6 hours on other regularly scheduled meetings.
I review between 50 and 100 pull requests a month according to GitHub.
People say "I know you're busy so I'll try to not take too much of your time". I'd much rather dedicate time to them than the dozens other things on my list that I have to do alone.
The term "glue work" describes a lot of what I do.

How I still get things done

I knew in theory about the benefits of breaking down tasks. Now I also do it because of necessity. I try to make everything achievable in small chunks. For example:

I encourage developers to make smaller pull requests that can be reviewed in only a few minutes.
Most of my writing happens in 3 phases, spread from hours to months apart: i) notes captured as quickly as possible, ii) fleshing out the text, iii) a final edit and review.
I deliver improvements to projects in tiny steps. That aligns well with my development philosophy and is easy to do with feature flags.

I still reserve time for learning, thinking and experimentation, though mostly outside normal work hours. Because of that I am careful to invest in learning what is interesting and useful to me. If it also benefits my employer, I record the time spent as work time and use my training/education budget.

Whether with normal work or experiments, my plans are often ahead of the execution and I don't have time for everything. That forces me to make a compelling case for my ideas and get other people to help.

What I want to change

I am doing some of these already, but I'd like to go further.

I'd like to get better at delegation.
I hope that by spending more time on documentation, I will spend less time on repeating myself.
I want to take more time to mentor others, even when it doesn't "scale", because I really like teaching.
Generally, I want to make myself replaceable, so that I can focus on what matters to me.

Clean your backlog

2021-11-10T00:00:00Z

The problem with big backlogs in software projects and how to keep the backlog small

Tips for applying to a job at a small company

2021-10-12T00:00:00Z

Advice and encouragement for job applicants reflecting the Simplificator hiring process back in 2021

How to connect to an Apple Thunderbolt display from Linux

2021-09-20T00:00:00Z

Although the Apple Thunderbolt Display is discontinued, it's still a good monitor and can serve as a docking station. It has a serviceable webcam, loudspeakers, an Ethernet port and USB 2 ports.

The Linux kernel supports Thunderbolt. After connecting the device, check if it's listed by lspci. If it's not, you may have to enable "Thunderbolt assist mode" in the BIOS (at least for Thinkpads).

The Thunderbolt device manager for user space is called bolt. boltctl allows you to list Thunderbolt devices and authorize them. While the screen itself may work without special action, the webcam and Ethernet require authorization.

One time authorization is done with boltctl authorize. boltctl enroll remembers the device and automatically authorizes it at future connections. Check the boltctl manual (man boltctl) and the Linux kernel Thunderbolt documentation.

A subtle change in PDF output

2021-07-05T00:00:00Z

A debugging story of rendering differences in PDFs after changing application hosting infrastructure

The trade-offs of two feature flag implementations

2020-12-30T00:00:00Z

There is a lot of value in testing work-in-progress features in production. You discover edge cases by using real data and you can measure the performance impact in a production setting.

Of course, testing in production doesn't necessarily mean releasing an unfinished feature to all users. This is where feature flags come in, allowing you to enable or disable functionality for specific users or groups (say admins or testers).

I often implement feature flags from scratch, as the feature flag libraries out there do a lot I don't need. I usually don't need a user interface, or to store configuration in Redis, or to let non-developers toggle flags.

In one case, I thought we might need more complex scenarios in the future, so I started by persisting the feature flags in a database table. 5 years later, I realised we had never taken advantage of the extra flexibility and migrated the flags to static configuration. It proved harder than anticipated.

Feature flags in the database

When the configuration is in the database, you can potentially change it anytime. That also makes testing easier.

Consider this piece of production code:

<% if Feature.enabled?(:facebook_like) %>
  <div class="fb-like">
    <%# the like button is here %>
  </div>
<% end %>

Tests can verify different scenarios by updating the feature flags table in the database.

scenario 'User sees a like button when enabled' do
  user = users(:alice)

  visit profile_path(user)
  refute_css('.fb-like')

  Feature.create!(name: 'facebook_like')
  visit profile_path(user)
  assert_css('.fb-like')
end

Feature.create!(name: 'facebook_like') is what enables the flag. The test depends on an implementation detail (the flag is stored in the database), but that could be solved by a helper function. Overall it's very easy to enable or disable flags in tests.

There are a few downsides of this approach outside of the test environment.

Flags are disabled by default, which is a good safeguard. However that also means that you must remember to update the database after deploying code that uses a flag. That is a manual step for each deployment environment.

Additionally, reading flags incurs database reads, which makes response times slower.

Feature flags in a file

Storing flags in the application configuration makes the deployment story easier. The flag goes through code review and is then automatically updated during deployment.

As the configuration is under version control, you can also easily debug. When an error occurs in a release, you can check which flags are enabled for that release. If the flags are in the database, you may not have this information anymore unless you implemented an audit trail or include feature flag configuration in the error notification context.

Reading flags is also faster. You can store flags directly in a data structure in a class, or you can store them in a file and read that file once when the application starts. Then the flags are always in memory.

That is great for production, but makes testing harder. Let's look at a slightly modified version of the test from earlier:

scenario 'User sees a like button when enabled' do
  user = users(:alice)

  visit profile_path(user)
  refute_css('.fb-like')

  with_feature_enabled('facebook_like') do
    visit profile_path(user)
    assert_css('.fb-like')
  end
end

with_feature_enabled needs to modify the value of the flag for the scope of the block. As I said, the feature flag value is normally read once at application start and cannot be modified anymore. In a unit test I would override configuration by passing it as a constructor parameter. In an integration test as the one above, that's not possible.

That leads us to metaprogramming shenanigans.

Stubs

Feature.enabled? can be called from any part of the code. The dirty secret to making the test above pass is to conditionally stub enabled? for the duration of a block.

Why conditionally? Because we have to take into account the name of the flag. Let's say we had this structure:

if Feature.enabled?(:feature1)
  # something

  if Feature.enabled?(:feature2)
    # ... something extra
  end
end

We might want to test with feature1 enabled, but feature2 disabled, so we cannot return the same value for any call to Feature.enabled?

Minitest#stub can help us. It takes three arguments: the method name, a "value or callable" and a block. From the documentation:

Add a temporary stubbed method replacing name for the duration of the block. If val_or_callable responds to #call, then it returns the result of calling it, otherwise returns the value as-is. If stubbed method yields a block, block_args will be passed along. Cleans up the stub at the end of the block. The method name must exist before stubbing.

The unconditional stub would pass a value as the second argument:

Feature.stub :enabled?, true do
 # Feature.enabled?(:feature1) returns true here
 # So does Feature.enabled?(:feature2)
 # In fact, all feature flags are enabled now
end

We can make the returned value depend on the parameter with a Proc (i.e. a "callable"):

callable = Proc.new do |arg|
  arg.to_s == "feature1"
end

Feature.stub :enabled?, callable do
 # calling Feature.enabled?(:feature1) returns true
 # calling Feature.enabled?(:feature2) returns false
end

This code enables a certain flag, but disables all others. What we want is to control one flag, but "pass through" to the default configuration for any other flags.

Taking inspiration from the Minitest stub implementation itself, we can keep a copy of the original Feature.enabled? method:

metaclass = Feature.instance_eval { class << self; self; end }
metaclass.alias_method :original_enabled?, :enabled?

We saved the method as Feature.original_enabled?. That allows us to use it in the "callable" argument once the Feature.enabled? method is overwritten by the stub:

callable = Proc.new do |arg|
  if arg.to_s == "feature1"
    true
  else
    Feature.original_enabled?(arg)
  end
end

Feature.stub :enabled?, callable do
  # Feature.enabled?(:feature1) returns true
  # Other flags are untouched
end

Let's clean up original_enabled? at the end:

metaclass.alias_method :enabled?, :original_enabled?
metaclass.undef_method :original_enabled?

Combining all of the above in a test helper gives us this monstrosity:

def with_feature_enabled(name, &block)
  metaclass = Feature.instance_eval { class << self; self; end }
  metaclass.alias_method :original_enabled?, :enabled?

  stub = Proc.new do |arg|
    if arg.to_s == name.to_s
      true
    else
      Feature.original_enabled?(arg)
    end
  end

  Feature.stub(:enabled?, stub, &block)
ensure
  metaclass.alias_method :enabled?, :original_enabled?
  metaclass.undef_method :original_enabled?
end

We could maybe have a simpler helper if we the Feature class supported overriding flags. For example:

def with_feature_enabled(name)
  Feature.overrides = { name => true}
  yield
  Feature.overrides = {}
end

However that would make the implementation of the Feature class more complex. I consider that worse in this case, as this functionality would not be used outside of the test helper.

Conclusion

Unsurprisingly, the design that allows for dynamic toggling of flags is easier to use in tests.

I generally find ways to write tests without stubs, but I don't see a good alternative in the application from which I took the example. Using a global configuration (be it read from the database or a file) for the feature flags is the most ergonomic for production use in this case.

If there was a business need to override feature flags without deploying, I'd revert to database storage or control via URL query parameters. Both of those would allow removing stubs from the tests.

As it is, it's great that Ruby allows us to get our hands dirty when needed.

Broken video in Safari after DOM change

2020-06-22T00:00:00Z

Let's say you're using early 2000s web tech to speed up page transitions, but with slightly newer APIs. When a link is clicked, the new page is loaded asynchronously (for example with fetch). Then a subsection of the current page is replaced by (a subsection of) the new page, using replaceWith (or replaceChild or a similar method depending on the implementation. There are many out there).

It looks like Safari has some trouble when that new page contains a video element.

I encountered this bug with a video element which defines a poster image, enables native controls and asks the browser to preload the metadata.

<video preload="metadata" poster="thumbnail.jpeg" controls>
  <source src="video.mp4">
</video>

Safari displays the poster image after navigating to the new page, but it does not show the video controls. A quick look at the developer tools reveals that it also does not make any network requests to preload the metadata. On the other hand it's still possible to programmatically load and play the video from the developer tools console.

Firefox and Chrome don't have this problem. Also, everything works normally after reloading the page in Safari, but that defeats the purpose of the speed optimization.

It looks like this is related to how the video element is inserted into the page. The bug occurs with replaceWith, but also with replaceChild and re-implementations with insertAjacentHTML, insertBefore or appendChild.

I couldn't find any information about this online except a passing comment from 2012.

What I did find out after experimenting some more is that the unsafe innerHTML (or its companion outerHTML) fixes the bug.

I would not replace the entire page with innerHTML. However the part with the video contains no user input. My workaround is to continue using replaceWith, but then refresh only the video elements:

document.querySelectorAll('video').forEach((video) => {
  const content = video.outerHTML;
  video.outerHTML = content;
})

This removes event handlers from the "reset" video element, so I also needed to re-attach those.

Github OAuth authorization with serverless functions

2019-12-13T00:00:00Z

At work I take care of many aspects of an application's life cycle: provisioning servers and configuring monitoring, setting up and verifying backups, automating certificate renewal, developing and maintaining the application, etc. A lot of it is a consequence of client requirements about the location of their data. Still, I'm always on the lookout for ways to eliminate most of this work for projects with no hosting restrictions.

One case that's been bothering me is our company's website, which is powered by a custom-built content management system. While hosting it on Heroku alleviates some of the burden, we still need to keep up with language and library versions. Thus, I've been thinking about alternatives.

The reason for the content management system is to allow more than developers to update the website. Any replacement would have to maintain this property.

While exploring alternatives I experimented with the combination of the Eleventy static site generator, Netlify CMS for editing and ZEIT Now for hosting.

Eleventy seems to offer a lot of flexibility without needing much configuration. Netlify CMS provides a user-friendly interface for editing content, but still integrates with git. Finally, ZEIT Now automatically issues certificates and deploys every commit.

Netlify CMS integrates nicely with the Netlify service and I would use that in production. However, I learned a more by not hosting on Netlify in my experiments.

Netlify CMS authentication backends

Netlify CMS requires read access to the repository to present the current content to editors and write access to save changes. This is solved with OAuth. A user logs in with Github and the access token is transmitted to Netlify CMS which can then commit on the user's behalf.

Github implements OAuth2 with an authorization code grant. The Netlify CMS is entirely a client side solution and cannot obtain the access token directly with this type of grant. Normally it relies on Netlify to facilitate authentication, but also supports a custom backend.

I read through the existing implementations, but did not like them. None of them have tests and many are susceptible to cross-site forgery attacks, so I decided to write my own backend (famous last words!).

"Serverless functions"

I realized I could host the authentication code directly on ZEIT Now using their "serverless functions". The concept is that if your project contains an api/ directory with one request handler per file, Now deploys the functions alongside your static content.

Up until that point I had not encountered a good use case for "serverless". An OAuth 2.0 client workflow seemed like one. It suddenly clicked.

Requesting an authorization code from Github

The login flow starts with a redirect to the Github authorization screen. This screen informs about the requested permissions defined by the scope parameter.

const { URLSearchParams } = require('url');
const { sign } = require('jsonwebtoken');
const uuidv4 = require('uuid/v4');

module.exports = (req, res) => {
  const params = {
    client_id: process.env.GITHUB_CLIENT_ID,
    scope: 'repo',
    state: sign(
      { nonce: uuidv4() },
      process.env.JWT_SECRET,
      { expiresIn: 30 }
    )
  };

  res.writeHead(307, {
    'Location': `https://github.com/login/oauth/authorize?${
      new URLSearchParams(params)
    }`
  })
  res.end();
}

The state parameter is the one that helps us to protect against cross-site forgery attacks. In the request phase you set its value to something unique that cannot be guessed by an attacker. In the callback phase (see below), you check the state returned by the provider matches the one you sent.

Here, the unique part is the UUID. It is transported as a JSON web token signed with a symmetric secret key. Additionally the token expires after 30 seconds. That may be a bit short for the very first login as part of it requires human interaction: authorizing Netlify CMS to write to the repo. After that first time the process is fully automated and I expect Github to respond in less than 30 seconds.

Exchanging the authorization code a for an access token

If the user grants the rights to the app, they get redirected back to a predefined endpoint.

There are two query parameters to check before going forward:

code: this is the authorization code. If there is none, someone may be trying to bypass the Github identity check.
state: this is the JSON web token sent in the first phase. Again, if that is invalid someone is trying to forge the request.

Assuming the parameters are correct, the authorization code can be exchanged for an access token.

The described callback handler could look like this:

const url = require('url');
const { URLSearchParams } = url;
const { verify } = require('jsonwebtoken');
const requestToken = require('../lib/request-token');
const { renderSuccess, renderError } = require('../lib/netlify-cms-login');

module.exports = async (req, res) => {
  const queryParams = new URLSearchParams(url.parse(req.url).query);
  const code = queryParams.get('code');
  const state = queryParams.get('state');
  const origin = process.env.ORIGIN;

  if (!code) {
    res.writeHead(400, { 'Content-Type': 'text/html; charset=utf-8' });
    res.end(renderError(origin, 'Code parameter missing'));
    return;
  }

  verify(state, process.env.JWT_SECRET, (error, _) => {
    if (error) {
      res.writeHead(400, { 'Content-Type': 'text/html; charset=utf-8' });
      res.end(renderError(origin, 'Invalid state parameter'));
      return;
    }
  });

  const tokenResponse = await requestToken(code);
  const { access_token, error, error_description } = tokenResponse;

  if (error) {
    res.writeHead(400, { 'Content-Type': 'text/html; charset=utf-8' });
    res.end(renderError(origin, error));
    return;
  }

  res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
  res.end(renderSuccess(origin, access_token));
};

The renderSuccess and renderError functions return responses that can be handled by Netlify CMS.

The Netlify CMS login handshake

I haven't found documentation about this part. It seems like everybody reads the Netlify CMS source code to understand how it works.

When you press the login button, Netlify CMS open the aforementioned login entry point in a new window. Then it stars listening for messages from that window. When that pop-up window has obtained an access token, it is expected to transmit it back to its parent window.

In the case of the Github workflow, the communication goes something like this:

The pop-up window sends the message authorizing:github to the window that spawned it.
The original window (where Netlify CMS is running) sends back something to the effect of "Oh yeah? Do you have an access token?" (actually it echoes back the same message as far as I can tell)
The pop-up relays Github's response in of these two forms:

// error case
authorization:github:error:{"message": "The error message"}

// success case
authorization:github:success:{"token":"access token value"}

The communication occurs via window.postMessage. It's crucial to verify who is the sender of such a message before acting on its contents.

With that in mind here is an example of what renderSuccess from the previous section might return:

<!doctype html><body><script>
  (function() {
    function recieveMessage(e) {
      if (e.origin !== "https://app.domain:1234") { return }
      window.opener.postMessage(
        'authorization:github:success:{"token":"sometoken"}',
        e.origin
      )
    }

    window.addEventListener("message", recieveMessage, false)
    window.opener.postMessage(
      "authorizing:github",
      "https://app.domain:1234"
    )
  })()
</script></body></html>

Testing

As seen above, ZEIT works with regular Node.js request handlers. One way to test a handler is to start an HTTP server, install that handler and send requests to it.

Here is for example the test for the /api/auth endpoint.

import test from 'ava';
import http from 'http';
import listen from 'test-listen';
import fetch from 'node-fetch';
import { verify } from 'jsonwebtoken';
import { URLSearchParams } from 'url';
import auth from '../api/auth';

test('redirects to Github OAuth workflow start', async t => {
  let server = http.createServer(auth);
  let url = await listen(server);
  let env = process.env;
  process.env.GITHUB_CLIENT_ID = 'dummy-id'
  process.env.JWT_SECRET = 'jwt-secret'

  let response = await fetch(url, { redirect: 'manual' });
  let location = response.headers.get('Location');

  t.is(response.status, 307);
  t.true(location.startsWith('https://github.com/login/oauth/authorize?client_id=dummy-id&scope=repo'))
  verify((new URLSearchParams(location)).get('state'), 'jwt-secret', (error, _) => {
    if (error) {
      t.fail(`Expected a valid JSON web token to be sent as state, got ${error}`);
    }
  });

  process.env = env;
  server.close();
});

The request handler reads the secrets from the process environment. That is why I set some environment variables at the start and restore the environment after the test.

The test-listen library makes the HTTP server listen on an arbitrary unused port and returns the complete URL. That way all tests can run in parallel without port collisions.

Example source code

Read the complete source code for this experiment for more details about integrating with the Netlify CMS.

A tale of flaky Cypress tests

2019-10-24T00:00:00Z

I wrote some flaky frontend tests using Cypress and then learned how to make them deterministic. This is the tale of that adventure.

An application I am working has a video player that should continue playing when changing pages. Additionally the player should be visible on the dedicated page of the video it is playing and hidden on other pages.

The <video> element cannot move within the DOM, because it would be destroyed and recreated, which interrupts the playback. The trick is to only change part of the page. That particular application achieves that with pjax (which is also still used by Github at the time of writing -- search for "pjax" in their HTML responses). The player lies outside the part that gets swapped during navigation. It is placed in a predefined area using CSS positioning.

I wanted to replace a lot of the player code, but there was no test coverage of the aforementioned requirements. I decided to add some using Cypress. I ended up with a long-ish integration test that does the following:

Visit a page where the player should be displayed and press play
Assert the player is covering its dedicated placeholder
Navigate to a another page and assert the player is hidden
Navigate to a page with a placeholder/preview of a different video and assert the player remains hidden
Go back to the page of the currently playing media and assert the player is placed correctly again

The first assertions was already causing me problems as a newcomer to Cypress.

Scroll, scroll, scroll

Cypress scrolls an element into view before clicking. The scrolling algorithm puts the element at the top of the page.

If the top of the page is covered by a floating header that always remains there, you will need to disable the checks before the click:

cy.get(selector).click({force: true})

Working directly with a DOM element

The test clicks the play button and some music plays. I don't want the hear the music when running the test suite, so I tried to mute or pause it.

Usually, you would call a function with invoke. This does not work though:

cy.get('[data-cy=player]')
  .should('be.visible')
  .invoke('pause')

It fails with the following error:

cy.invoke() waited for the specified property 'pause' to exist, but it never did.

That is because get() returns a list of elements even if there's only one.

My first instinct to unwrap it also did not work:

// Results in error
cy.get('@player').invoke('get', 0).invoke('pause')

This did:

cy.get('@player').then(element => element.get(0).pause())

Flaky behaviour

I then went on to test that the player covers its placeholder:

cy.get('[data-cy=player]').then((player) => {
  cy.get('[data-cy=player-placeholder]').then((placeholder) => {
    cy.wrap(player.offset()).should('deep.eq', placeholder.offset())
    cy.wrap(player.width()).should('eq', placeholder.width())
    cy.wrap(player.height()).should('eq', placeholder.height())
  })
})

That worked sometimes in the Cypress UI and never in headless mode. Because the player is only displayed once it starts playing, sometimes Cypress would check its position before it has moved.

CypressError: Timed out retrying: expected { top: -10000, left: 0 } to deeply equal { top: 138, left: 0 }

Cypress deals with race conditions by automatically retrying commands and assertions.

It does not retry every command, though, and I spent a long time searching for a way to retry getting the player position and dimensions. Some of it was reading the Cypress source code. One attempt even resulted in an infinite loop that kept retrieving the same DOM element!

I eventually found a comment describing a hack using the public API.

Make any function call retry-able

The trick involves a custom command.

function try(subject, fn) {
  return cy.wrap({ try: () => fn(subject) }, { log: false })
    .invoke('try')
}

(log is set to false to exclude these implementation details from the command trace in the Cypress log)

Finally, I had a reliable test.

cy.get('[data-cy=player-placeholder]').then(placeholder => {
  cy.get('@player').try(boundingBox).should(box => {
    assert.deepEqual(box, boundingBox(placeholder),
      'the player is covering its placeholder during playback')
  })
})

// ...

function boundingBox(element) {
  return element[0].getBoundingClientRect()
}

When the assertion in the callback to should() fails, Cypress retries the last command in the chain.

More flakiness

Funnily enough I had another race condition in the test that I did not understand until I started writing this post. The music did not pause during some test runs. The real test code that should pause it is as follows.

cy.get('[data-cy=player]')
  .should('be.visible')
  .then(container => container.find('audio').get(0).pause())

The culprit is the visibility assertion. The element is hidden by default using a large negative margin for historical reasons involving old Internet Explorer versions and a Flash fallback for the player. IE would completely disregard the Flash object if it had visiblity: hidden. The workaround that tricked IE into believing the off-screen player was visible also has an effect on Cypress. There is nothing in Cypress' definition of visibility saying an element which is ten thousand pixels above the viewport is hidden.

My assumption was that a "visible" player is playing, because in the implementation we change its coordinates after starting playback. However for Cypress the element is always visible, so it immediately tries to pause which causes the race condition.

I can think of two ways of fixing this. One is to mute the sound at the start of the test and let the audio/video play.

cy.get('[data-cy=player] audio')
  .then(player => player[0].muted = true)

The other is to explicitly wait for the playback to start before pausing:

cy.get('[data-cy=player] audio')
  .should($element => {
    const player = $element.get(0)
    assert.isTrue(isPlaying(player), "the media is playing")
    player.pause()
  })

// ...

function isPlaying(media) {
  return (media.duration > 0 &&
    !media.paused &&
    !media.ended &&
    media.readyState > 2);
}

should() is retried if the assertion in the callback fails. Pausing the player after the assertion ensures it will be playing by that time. (The isPlaying() helper is a bit awkward, because there is no playing property on HTMLMediaElement.)

How can we make sure this works? By introducing a delay in the production code.

new Promise(resolve => setTimeout(resolve, 2000))
  .then(() => play())

Setting up Cypress with Rails

2019-10-11T00:00:00Z

Tips for integrating the Cypress testing framework into a Ruby on Rails application

Decoding disparate JSON fields into a custom Elm type

2019-03-20T00:00:00Z

Update (2019-11-06): I think what I describe is an example of what people call "parse, don't validate".

I wrote a simple video player in Elm as an experiment. My earliest realization was that authoritative source for player state the HTML media element (<audio> or <video>) and not the Elm application. I decided to keep the Elm model in sync by sending the JSON-encoded HTMLMediaElement over a port whenever any change in the media element occurs.

var events = [
  "abort",
  "canplay",
  "canplaythrough",
  "change",
  "error",
  /* ... and many more...  */
];

events.forEach(function(eventName) {
  media.addEventListener(eventName, function(event) {
    app.ports.inbound.send(event.target);
  })
});

It's tempting to define a model on the receiving side that represents the JSON data structure as is. The fields would be as defined by the specification. It would be easy to look up documentation and trace the flow of values. For example, one could look at the Elm debugger and see that there is a media.readyState value of 4, then look up what a "ready state" is and what HAVE_ENOUGH_DATA (defined as 4) means.

However, I don't like that:

mutually exclusive states are represented as boolean attributes. For example, ended and paused will never both be true in practice, but the data structure allows for that combination.
somewhat related, error is null when there are no errors and contains a code and message if there is an error.
it's hard to tell whether the media is playing. There is a playing event, but no playing attribute, like there is for example a seeking or paused attribute.

I prefer the following data structure:

{-| Current status of media playback
-}
type Playback
    = Playing
    | Paused
    | Ended
    | PlaybackError Error

type Error
    = UnsupportedSource
    | BrowserPolicyViolation
    | NetworkConnectivity

It's missing some states I didn't need for my experiment, like "loading metadata" or "buffering", but it's easy to add them.

What's neat about this type is that the variants are mutually exclusive. It can avoid errors like trying to programmatically pause a video that has already ended. It can also make state machine transitions explicit, like "load a new video when this one ends playing".

handleMediaUpdate : Model -> Media.State -> ( Model, Cmd Msg )
handleMediaUpdate model media =
    case media.playback of
        Media.Ended ->
            playNext model

        -- ...

How do we transform the JSON representation of the media element into this handy data structure?

The JSON has this shape:

{
  "autoplay": false,
  "controls": false,
  "currentTime": 3.999659,
  "ended": false,
  "error": null,
  "paused": true,
  "seeking": false,
  "src": "http://localhost:8000/test.wav",
  "volume": 1
  // and many other fields...
}

To build a Playback value out of this, we need to decode multiple fields, then combine them.

{-| State decoder for HTMLMediaElement objects
-}
decoder : Json.Decode.Decoder Playback
decoder =
    Json.Decode.map3 toPlayback
        (Json.Decode.field "paused" Json.Decode.bool)
        (Json.Decode.field "ended" Json.Decode.bool)
        (Json.Decode.field "error" errorDecoder)

The trick is in the toPlayback function:

{-| A helper function to decode related fields into a custom type. Be careful
with the order of arguments, there is potential for confusing the two booleans
-}
toPlayback : Bool -> Bool -> Maybe Error -> Playback
toPlayback paused ended maybeError =
    case ( paused, ended, maybeError ) of
        ( _, _, Just error ) ->
            PlaybackError error

        ( _, True, _ ) ->
            Ended

        ( True, _, _ ) ->
            Paused

        _ ->
            -- FIXME: it's only really playing if readyState is 3 or 4
            Playing

(The catch-all branch is an oversimplification. As far as I know, we also need to take into account the readyState to derive whether the media is playing. I took a shortcut in the initial implementation.)

With this, the complexity is handled at the system boundary. It is the only spot that deals with booleans. It's also the only place in the Elm code which creates a Playback value. I would know where to look if there was a bug in the playback state.

Long-lived background Ruby processes

2018-10-28T00:00:00Z

I needed a way to run a process alongside a Rails application for an always-running live data ingestion task. Sidekiq is only meant for short jobs, but Sidekiq itself is a long-lived process. That gave me the idea to look at the source code of Sidekiq, Resque and Puma for inspiration.

Update: In the meantime, Sidekiq itself has removed this capability and relies on a supervising process like systemd.

In order to have a long-running process, we need:

A way to daemonize the process
A logger, so that we know what's going on
A PID file, so that we can stop the process
A signal handler to handle shutdowns gracefully (look at Sidekiq for a more flexible take on this)
A Rake task to easily launch the process with a Rails context

Start

The easiest way to start the daemon is with a Rake task:

task ingest: :environment do
  Ingestion.new(
    logfile: Rails.root.join('log', 'ingestion.log'),
    pidfile: Rails.root.join('tmp', 'pids', 'ingestion.pid')
  ).ingest
end

The dependency to environment is essential. It's what makes sure Rails is loaded.

The tasks delegates to the Ingestion class, which orchestrates the work:

def ingest
  IngestionDaemon.new(logger: logger, pidfile: pidfile).work do
    # ingest data
  end
end

Shutdown

Since we want to keep track of the background process, we write its ID to a file.

File.open(pidfile, 'w') { |f| f << Process.pid }
at_exit { FileUtils.rm_f pidfile }

The pidfile is cleaned up just before the program exits

We can terminate the daemon with the kill command:

kill $(cat tmp/pids/ingestion.pid)

Signal handling

We can politely ask the program to stop by sending it a signal.

Before putting the program in the background, we define which signals to handle and how using Signal.trap:

trap('INT') { interrupt }
trap('TERM') { interrupt }

The interrupt method above raises Interrupt which is also the exception raised by Ruby when you press Control-C to stop a program. Here we explicitly handle this case, but also the termination signal. That one is sent by default by the kill command (see the kill(1) and signal(7) manual pages).

Then it's a matter to rescuing the Interrupt exception at the appropriate place in the program and cleaning up.

Read up on the self-pipe trick for more robust signal handling. I did not need it for this program.

Detach the process from controlling terminal

The Ruby API for this is a bit confusing:

Process.daemon(true, false)

The first argument controls whether or not to change the current working directory to root (/). true tells Ruby to keep the current directory. The second argument controls whether to keep the input and output streams or redirect them to /dev/null. false tells Ruby explicitly to redirect to /dev/null. If you don't "close" the streams, the process will continue writing to the terminal even after it's detached.

Wrap-up

Here is the entire class:

class IngestionDaemon
  def initialize(logger:, pidfile:)
    @logger = logger
    @pidfile = pidfile
  end

  def work(&block)
    register_signal_handlers
    daemonize
    write_pid
    ingest(&block)
  rescue Interrupt
    shutdown
  end

  private

  attr_reader :logger, :pidfile

  def register_signal_handlers
    trap('INT') { interrupt }
    trap('TERM') { interrupt }
  end

  def daemonize
    Process.daemon(true, false)
  end

  def write_pid
    File.open(pidfile, 'w') { |f| f << Process.pid }
    at_exit { delete_pidfile }
  end

  def delete_pidfile
    FileUtils.rm_f pidfile
  end

  def ingest
    logger.info 'Starting ingestion daemon'
    yield
    logger.info 'Ingestion daemon ran out of work'
  end

  def interrupt
    raise Interrupt
  end

  def shutdown
    logger.info 'Shutting down ingestion daemon'
    exit(0)
  end
end

Keeping Elm JSON decoders in sync with your Rails API

2017-09-11T00:00:00Z

Implementing and consuming APIs is very error-prone. I've now worked on multiple single page applications written in Elm and I've grown to love the constraint of having to explicitly define JSON decoders. They are very helpful in exposing bugs and errors in documentation.

In this post, I describe a strategy I've adopted to catch compatibility mistakes as early as possible. It involves JSON Schema in automated tests for both the backend and frontend. JSON Schema is not a silver bullet, but it has been useful in my personal experience and is worth the trouble to gain some extra confidence in a program.

The main ideas behind the approach are:

the API producer and consumers should always comply with the same schema version
maintaining and sharing the schema and validating against the schema should be very convenient, even automated if possible, otherwise you will stop doing it.
tests should fail when either side breaks the contract

This post uses Elm 0.18 and Ruby on Rails, but it may be relevant to future Elm versions and certainly applies to other backend programming languages and ecosystems.

Describing JSON schemas in Elm

json-elm-schema is an Elm library for defining JSON schemas. For example:

module Schema.Profile exposing (schema)

import Json.Encode as Encode
import JsonSchema exposing (..)


schema : Schema
schema =
    object
        [ title "User profile"
        , properties
            [ required "name" <| string []
            , optional "age" <|
                integer
                    [ description "Age in years"
                    , minimum 0
                    ]
            ]
        , examples Encode.object
            [ [ ( "name", Encode.string "Jane Doe" )
              , ( "age", Encode.int 42 )
              ]
            , [ ( "name", Encode.string "John Smith" )
              , ( "age", Encode.int 25 )
              ]
            ]
        ]

This package helps:

prevent schema authoring mistakes
validate JSON structures against a schema
generate JSON structures that correspond to a schema
convert (in both directions) between a schema written in Elm like above and a normal JSON Schema

It is going to be the foundation of the entire approach.

Easy fuzz tests for JSON decoders

Say we have a function allowing us to decode the API response described above into an Elm record.

module Profile exposing (Profile, decoder)

import Json.Decode as Decode exposing (Decoder)


type alias Profile =
    { name : String
    , age : Int
    }


decoder : Decoder Profile
decoder =
    Decode.map2
        Profile
        (Decode.field "name" Decode.string)
        (Decode.field "age" Decode.int)

One way to test the decoder is to give it some example input and assert that we get correctly decoded data back.

decoderTest =
    test "profile decoder" <|
        \_ ->
            Json.Decode.decodeString Profile.decoder
            """
            { "name": "Jane Doe", "age": 42 }
            """
                |> Expect.equal
                    { name = "Jane Doe", age = 42 }

Instead of writing examples by hand, we can generate a large number of random JSON documents that conform to the schema we wrote before and make sure that we can decode all of them. The JsonSchema.Fuzz module provides just what we need.

module ProfileTest exposing (suite)

import Test exposing (..)
import JsonSchema.Fuzz exposing (schemaValue)
import Schema.Profile
import Profile
import Json.Decode exposing (decodeValue)
import Helpers exposing (expectOk)


suite : Test
suite =
    describe "Profile"
        [ fuzz
            (schemaValue Schema.Profile.schema)
            "decoder complies with the profile JSON schema"
          <|
            \value ->
                decodeValue Profile.decoder value
                    |> expectOk
        ]

(Read more about fuzzers and fuzz tests in the elm-test documentation.)

expectOk is a custom expectation which takes a Result (which decodeValue returns) and only succeeds if that result is not an error.

module Helpers exposing (..)

import Expect exposing (Expectation)


expectOk : Result String value -> Expect.Expectation
expectOk val =
    case val of
        Ok _ ->
            Expect.pass

        Err err ->
            Expect.fail err

I use this helper because JSON values are randomly generated and we do not know anything about their contents except whether they are valid according to the schema. Some decoders may still benefit from example-based tests where we do make assertions about the concrete output values.

I have not yet needed JSON encoders on the Elm side, but json-elm-schema also has a validator module which can help test them.

Sharing the JSON schemas with Ruby

Next, we want to make the schemas available to the Ruby code. Elm cannot do file system operations, but the elm-json-schema-compiler npm module allows you to compile the schemas defined in Elm code to regular JSON schema files. Unfortunately, I could not use it directly since I wanted some extra flexibility. Namely, I want to generate all the schemas in one go and keep the Elm json-elm-schema package as a test dependency only.

I modified the command line tool as shown below. There are two parts: an Elm program that encodes the schemas and sends them through a port and a Node.js program that compiles and runs the Elm program, saving the JSON strings to files as they come in through the port.

port module Main exposing (..)

import JsonSchema exposing (Schema)
import JsonSchema.Encoder exposing (encode)
import Schema.Profile


type alias NamedSchema =
    ( String, Schema )


schemas : List NamedSchema
schemas =
    [ ( "profile", Schema.Profile.schema ) ]


emitSchemas : List NamedSchema -> Cmd ()
emitSchemas namedSchemas =
    namedSchemas
        |> List.map (encodeNamedSchema >> emit)
        |> Cmd.batch


encodeNamedSchema : NamedSchema -> ( String, String )
encodeNamedSchema ( title, schema ) =
    ( title, encode schema )


main : Platform.Program Never () ()
main =
    Platform.program
        { init = ( (), emitSchemas schemas )
        , update = \_ _ -> ( (), Cmd.none )
        , subscriptions = \_ -> Sub.none
        }



{-| Emits pairs of schema name and schema JSON string -}
port emit : ( String, String ) -> Cmd a

The important part for everyday use is the schemas function where we declare the file name for each schema (the node script takes care of adding the .json suffixes). In this example we only have one schema:

[ ( "profile", Schema.Profile.schema ) ]

The program which does all the input/output work takes two arguments: the path to the Elm file presented above and an output directory for the generated schemas.

#!/usr/bin/env node

if (process.argv.length < 4) {
  fail(`Generate JSON schema files using json-elm-schema
Usage:
  bin/generate-schemas <elm-schema-file> <output-directory>
`)
}

const fs = require('fs')

const path = require('path')
const temp = require('temp').track()
const compiler = require('node-elm-compiler')

const sourcePath = path.resolve(process.argv[2])
const prefixPath = path.resolve(process.argv[3])

const targetPath = temp.path({ suffix: '.js' })

compiler.compileSync(sourcePath, {
  yes: true,
  output: targetPath,
  cwd: 'tests',
})

const Elm = require(targetPath)
const app = Elm.Main.worker()

app.ports.emit.subscribe(function (payload) {
  const title = payload[0]
  const json = payload[1]
  const fileName = title + '.json'
  const schemaPath = path.join(prefixPath, fileName)

  fs.writeFile(schemaPath, json, (err) => {
    if (err) throw err

    console.log('Wrote schema for "' +
      title + '" to ' + schemaPath)
  })
})

function fail (msg) {
  process.stderr.write(msg)
  process.exit(1)
}

In hindsight, there is probably a way to solve using the original npm package and a bash script, but on the upside I learned how to run "headless" Elm programs and carry out side-effects for them.

Schema validation in Ruby tests

Thanks to the json_schema gem we can verify that the API server complies with the schema. Generate the schemas into a pre-defined directory such as test/schemas.

$ bin/generate-schemas tests/Schema/Main.elm test/schemas

Don't forget to include this as a build step in your continuous integration pipeline to ensure you are always using the latest schemas.

Define a test helper like so:

require 'json_schema'

module SchemaHelper
  def assert_response_schema(schema_name)
    full_path =
      Rails.root.join('test', 'schemas', schema_name)

    schema_data = JSON.parse(File.read(full_path))
    schema = JsonSchema.parse!(schema_data)
    schema.expand_references!

    payload = JSON.parse(response.body)

    schema.validate!(payload)
  end
end

This loads the specified schema and validates the server response against it. It is meant for use in a controller test.

require 'test_helper'
require 'schema_helper'

class ProfilesControllerTest < ActionDispatch::IntegrationTest
  include SchemaHelper

  test 'the output corresponds to the schema' do
    get '/profile'

    assert_response :success
    assert_response_schema 'profile.json'
  end
end

You could also modify the helper assertion to explicitly take the JSON structure as an argument, allowing you to unit test your JSON serializers.

A Self-Testing Test Library

2016-08-18T00:00:00Z

After watching Ryan Davis present a way to write a test framework from scratch, I knew I wanted to try that on my own. Before I could get to it, I stumbled upon a video of Kent Beck writing a small self-tested test library in CoffeeScript. In it, Kent Beck demonstrates the effect immediate feedback has on a test-driven workflow, but it was the self-testing aspect that fascinated me.

I set out to write a Ruby self-tested library from scratch, describing the process as I go. You can follow along with the project's git repository.

Assertions

Assertions are the building blocks of this test library. We start with the assert function, which takes a boolean parameter. It does nothing if the parameter evaluates to true and complains if it evaluates to false. In Ruby, raising an exception is one way to signal failure. It is convenient for assert since uncaught exceptions interrupt the program. Incidentally, I named the library Raisin since we will be raisin' exceptions.

As we said, assert does nothing if the provided value is true. Let's write our very first failing test:

assert(true)

puts "Success!"

puts "Success!" should stay at the very bottom of the file as we add new code. Run the script:

ruby raisin.rb

It does not print "Success!" yet, because assert is undefined. As an aside, I defined ,t as a shortcut in vim (in normal mode press comma, then press "t") to run the tests, i.e. the current file as the code is self-tested.

:noremap <leader>t :!ruby %<CR>

Defining the function with an empty implementation makes the test pass:

def assert(condition)
end

Now let's cover the other intended behaviour: parameters that do not evaluate to true cause an exception.

class AssertionError < StandardError
end

class NothingRaised < StandardError
end

raised = false
begin
  assert(false)
rescue AssertionError
  raised = true
end
raise NothingRaised unless raised

We define AssertionError for use within our assertions. That way we have meaningful error messages: we know specifically that an assertion failed and not something else in the code. In a similar manner, we define NothingRaised, but just for the purposes of this test. If the assertion fails, we print "Success!", otherwise we raise NothingRaised so we know something is wrong.

We run the script to confirm the test failure:

raisin.rb:18:in `<main>': NothingRaised (NothingRaised)

Then we implement assert to make the test pass.

def assert(condition)
  raise AssertionError unless raised
end

(Commit eb31d6e)

assert_equal is an assertion that I use most of the time. We can implement this on top of assert. Add an empty implementation and a failing test:

def assert_equal(expected, actual)
end

assert_equal(true, true)

raised = false
begin
  assert_equal("foo", "bar")
rescue AssertionError
  raised = true
end
raise NothingRaised unless raised

We are repeating the structure of that second test, but we can refactor that after we make it pass.

def assert_equal(expected, actual)
  assert(expected == actual)
end

Let's now refactor the verification that AssertionError was raised.

def test_failure(&block)
  raised = false
  begin
    yield
  rescue AssertionError
    raised = true
  end
  raise NothingRaised unless raised
end

test_failure do
  assert(false)
end

test_failure do
  assert_equal("foo", "bar")
end

test_failure takes a block so that we can execute it at the right spot in the test. If we took the assertion as an argument (test_failure(assert(false))), then the assertion would be evaluated before the code in test_failure and we wouldn't be able to catch the AssertionError. Since test_failure is a helper function in our own tests, we are not going to write a test for it, but still want to test it manually. Comment out the implementation of assert and confirm that the tests fail:

def assert(condition)
  #raise AssertionError unless condition
end

test_failure do
  assert(false)
end

raisin.rb:14:in `test_failure': NothingRaised (NothingRaised)
        from raisin.rb:23:in `<main>'

All good.

(Commit d6b9a3e)

In the case of a failing assert_equal we should get more information than just AssertionError. Exceptions optionally carry messages. We can use those to convey details about assertion failures.

def assert(condition, failure_reason = nil)
  unless condition
    raise AssertionError, failure_reason
  end
end

Now we can update assert_equal to report what went wrong:

def assert_equal(expected, actual)
  message = "Expected #{expected.inspect}, got #{actual.inspect}"
  assert(expected == actual, message)
end

Let's try it:

assert_equal("foo", "bar")

raisin.rb:19:in `assert': Expected "foo", got "bar" (AssertionError)
        from raisin.rb:32:in `assert_equal'
        from raisin.rb:41:in `<main>'

Without the calls to inspect the printed message would be Expected foo, got bar which is less readable than when the strings are quoted.

Do we test the message? I feel like that would be testing Ruby itself, which is not very useful. What we can do is test the message format later if we decide to add pretty printing.

(Commit 0b9fa6b)

Test suites

Let's declare a test suite that uses our library and run it.

def greet(name = nil)
  ['Hello', name].compact.join(", ") + "!"
end

[
  ->() { assert_equal("Hello, Bob!", greet("Bob")) },
  ->() { assert_equal("Hello!", greet) }
].map(&:call)

Here, each lambda is a test case. The problem with lambdas is that they leak state between tests. Consider these tests:

x = 1
[
  ->() { assert_equal 1, x },
  ->() { x = x + 1; assert_equal 2, x },
  ->() { assert_equal 1, x }
].map(&:call)

The third test fails:

raisin.rb:19:in `assert': Expected 1, got 2 (AssertionError)
        from raisin.rb:32:in `assert_equal'
        from raisin.rb:63:in `block in <main>'
        from raisin.rb:64:in `map'
        from raisin.rb:64:in `<main>'

Methods keep state isolated. We will represent test cases as methods.

def test_with_name
  assert_equal("Hello, Bob!", greet("Bob"))
end

def test_without_name
  assert_equal("Hello!", greet)
end

test_with_name
test_without_name

Now our tests also have names to convey their purpose. The downside is that we need to call each function. We can overcome that with a little meta-programming.

class GreetingTestSuite
  def test_with_name
    assert_equal("Hello, Bob!", greet("Bob"))
  end

  def test_without_name
    assert_equal("Hello!", greet)
  end
end

suite = GreetingTestSuite.new
suite.public_methods(false).each do |test|
  suite.send(test)
end

We take advantage of the public_methods method. We pass it false to only get methods that we have defined and not superclass methods. Compare:

suite.public_methods

[:test_with_name, :test_without_name, :instance_of?, :public_send, :instance_variable_get, :instance_variable_set, :instance_variable_defined?, :remove_instance_variable, :private_methods, :kind_of?, :instance_variables, :tap, :method, :public_method, :singleton_method, :is_a?, :extend, :define_singleton_method, :to_enum, :enum_for, :<=>, :===, :=~, :!~, :eql?, :respond_to?, :freeze, :inspect, :display, :object_id, :send, :to_s, :nil?, :hash, :class, :singleton_class, :clone, :dup, :itself, :taint, :tainted?, :untaint, :untrust, :trust, :untrusted?, :methods, :protected_methods, :frozen?, :public_methods, :singleton_methods, :!, :==, :!=, :__send__, :equal?, :instance_eval, :instance_exec, :__id__]

and

suite.public_methods(false)

[:test_with_name, :test_without_name]

Once we have the correct method names we send those messages to the suite object. That code is reusable as is so we can put it in a method that takes a class instance as parameter.

def run_suite(suite)
  suite.public_methods(false).each do |test|
    suite.send(test)
  end
end

(Commit edf2e31)

Grouping the tests in a class also allows us to perform setup and tear-down if the test suite has any.

def run_suite(suite)
  suite.public_methods(false).each do |test|
    suite.send(:setup) if suite.respond_to?(:setup)
    suite.send(test)
    suite.send(:teardown) if suite.respond_to?(:teardowon)
  end
end

While I prefer a functional approach in general, I identify two code smells in the above code. First, the method is called run_suite and takes a suite parameter. It would make more sense to call suite.run. Second, we are checking if an object responds to a message before sending it that message. A more confident approach is to always run setup and teardown, but provide default implementations. We already have a special case of a test suite in the form of the example GreetingTestSuite. What we are missing is a generalized version:

class TestSuite
  def run
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      setup
      send(test)
      teardown
    end
  end

  def setup
  end

  def teardown
  end
end

The adopted convention is that the names of the methods that are test cases begin with test_. That way we make sure to not erroneously call the setup and teardown methods. It also allows test suites to contain helper methods.

We need to to update the test code to use our new class:

class GreetingTestSuite < TestSuite
  # ...
end

GreetingTestSuite.new.run

(Commit 67b9a5d)

With multiple test suites we have to run each one manually, just like we did with test cases before that. We can automate that by keeping track of classes that inherit from TestSuite and running all of those test suites.

class TestSuite
  @suites = []

  def self.inherited(suite)
    @suites << suite
  end

  def self.run
    @suites.each do |suite|
      suite.new.run
    end
  end

  #...
end

We cannot yet get rid of GreetingTestSuite.new.run in favour of calling TestSuite.run at the very bottom of our file because of a subtlety with nested test suites that we will address later.

(Commit d6f15e6)

Reporting

In the current implementation failing tests manifest themselves by raising an error, but there is no way to distinguish between a test that has not been run and a test that passes. After each step in the previous section, I've been manually verifying that the tests are still running by making assertions fail. That reveals two things:

the test suite functionality is not tested rigorously;
reporting success is as important as reporting failure.

Let's address the first one by counting how many tests have been run. This will allow us to assert that all tests in a suite are being run.

class DummySuite < TestSuite
  def test_equality
    assert_equal(1, 1)
  end
end

assert_equal 1, DummySuite.new.run.runs

We got rid of the greetings code and associated tests in favor of a simpler example. The assertion fails with the following error:

raisin.rb:77:in `<main>': undefined method `runs' for [:test_equality]:Array (NoMethodError)

This is because TestSuite#run implicitly returns the array of test names. We are going to return a report instead.

class Report
end

class TestSuite
  def run
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      setup
      send(test)
      teardown
    end

    Report.new
  end
end

Running the tests again gives us:

raisin.rb:82:in `<main>': undefined method `runs' for #<Report:0x00555e6f553548> (NoMethodError)

With an empty implementation of runs in Report, we finally get an assertion error (Expected 1, got nil). We can make the test pass with the following:

class TestSuite
  def run
    runs = 0
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      setup
      send(test)
      runs = runs + 1
      teardown
    end

    Report.new(runs)
  end
end

class Report
  attr_reader :runs

  def initialize(runs)
    @runs = runs
  end
end

We can test that only methods starting with test_ are called by adding an extra method to DummySuite called foo for example. This does not fail the assertion, as expected.

The next step is to count the failed tests. I will spare you the TDD process. Here is one possible implementation:

class TestSuite
  def run
    runs = 0
    failures = 0
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      setup
      begin
        send(test)
      rescue AssertionError
        failures = failures + 1
      end
      runs = runs + 1
      teardown
    end

    Report.new(runs, failures)
  end
end

class Report
  attr_reader :runs, :failures

  def initialize(runs, failures)
    @runs = runs
    @failures = failures
  end
end

class DummySuite < TestSuite
  def test_equality
    assert_equal(1, 2) # should fail
  end

  def test_the_truth
    assert(true)
  end
end

result = DummySuite.new.run
assert_equal 2, result.runs
assert_equal 1, result.failures

(Commit 72f91c3)

We should also show progress as tests run. I particularly like the convention of printing a dot when a test passes and the letter "F" when it fails. The easiest way for now is to add print statements inside TestSuite#run:

def run
  runs = 0
  failures = 0
  test_names = public_methods(false).grep(/^test_/)

  test_names.each do |test|
    setup
    begin
      send(test)
      print "."
    rescue AssertionError
      failures = failures + 1
      print "F"
    end
    runs = runs + 1
    teardown
  end

  puts

  Report.new(runs, failures)
end

That works, but the method is getting a little unwieldy. It does too much: it selects which methods to run, it gathers statistics, it prints to the terminal. The test suite itself shouldn't need to know how to do all that. First, we'll introduce a collaborator that hides the exception handling:

class TestResult
  def self.from(&block)
    begin
      yield
      TestSuccess.new
    rescue AssertionError
      TestFailure.new
    end
  end
end

class TestSuccess
  def success?
    true
  end
end

class TestFailure
  def success?
    false
  end
end

We use it like this:

class TestSuite
  def run
    runs = 0
    failures = 0
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      result = TestResult.from do
        setup
        send(test)
        teardown
      end

      if result.success?
        print "."
      else
        failures = failures + 1
        print "F"
      end
      runs = runs + 1
    end

    puts

    Report.new(runs, failures)
  end
end

Notice that I've included the setup and tear-down in the block. It's a matter of precaution, in case they contain assertions as well. Also, running the test and displaying the result used to be interleaved and now they are done one after the other, allowing us to extract the reporting code. The report class becomes:

class Report
  attr_reader :runs, :failures

  def initialize
    @runs = 0
    @failures = 0
  end

  def add_result(result)
    if result.success?
      print "."
    else
      @failures = @failures + 1
      print "F"
    end
    @runs = @runs + 1
  end
end

Here is the improved test runner:

class TestSuite
  def run
    report = Report.new
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      result = TestResult.from do
        setup
        send(test)
        teardown
      end
      report.add_result(result)
    end
    report
  end
end

(Commit d707825)

One of the things we haven't tested is the report output. This requires a small modification of the report class. We will pass the input/output stream to the constructor. That way we can use $stdout when we want to display something in the terminal and an instance of StringIO when we need to capture the output for tests. In our test case, we are going to simulate a suite that has one failing and one passing test. The resulting output should be an "F" followed by a dot. We reuse the DummySuite from before, but we capture the output.

output = StringIO.new
report = DummySuite.new.run(output)
assert_equal(2, report.runs)
assert_equal(1, report.failures)
assert_equal("F.", output.string)

We modify Report as described above:

class Report
  attr_reader :runs, :failures

  def initialize(io)
    @io = io
    @runs = 0
    @failures = 0
  end

  def add_result(result)
    if result.success?
      io.print "."
    else
      @failures = @failures + 1
      io.print "F"
    end
    @runs = @runs + 1
  end

  private

  attr_accessor :io
end

Finally, we update TestSuite#run to use the new API:

class TestSuite
  def run(io = $stdout)
    report = Report.new(io)
    # ...
  end
end

(Commit 3aff6fc)

By just writing "F" when a test fails we lose valuable information about the reason of the failure. We can store the error and print a nice summary of all errors when the test suite finishes running. Let's make a test test suite (you read that right) to test the summary.

suite = Class.new(TestSuite) do
  def test_1
    assert(false, "failure1")
  end

  def test_2
    assert(false, "failure2")
  end
end
output = StringIO.new
report = suite.new.run(output)
assert(output.string.include?("failure1"), "Report does not include error details")
assert(output.string.include?("failure2"), "Report does not include error details")
assert(output.string.include?("2 runs, 2 failures"), "Report does not include statistics")

I created an anonymous class, because we don't really care about reusing it. To implement this, we first need to store the error messages for failed tests:

class TestResult
  def self.from(&block)
    begin
      yield
      TestSuccess.new
    rescue AssertionError => error
      TestFailure.new(error)
    end
  end
end

class TestFailure
  attr_reader :error

  def initialize(error)
    @error = error
  end

  def success?
    false
  end
end

Then we gather them in the report and print them at the end:

class Report
  attr_reader :runs

  def initialize(io)
    @io = io
    @runs = 0
    @errors = []
  end

  def add_result(result)
    if result.success?
      io.print "."
    else
      @errors << result.error
      io.print "F"
    end
    @runs = @runs + 1
  end

  def failures
    @errors.count
  end

  def summarize
    io.puts
    @failures.each do |failure|
      io.puts
      io.puts failure.message
      io.puts failure.backtrace
      io.puts
    end
    io.puts
    io.puts "#{runs} runs, #{failures} failures"
  end

  private

  attr_accessor :io
end

Finally, we call summarize from TestSuite#run:

class TestSuite
  def run(io = $stdout)
    # ...
    report.summarize
    report
  end
end

We also need to update an assertion in the previous test case from

assert_equal("F.", output.string)

assert(output.string.include?("F."))

(Commit 175d0a4)

Test suites testing test suites

Now that we can organize tests and see them fail, we should refactor.

Include the library in a new file where the new test suites will reside:

# test.rb
require_relative './raisin'

We'll be porting our tests into that suite. What this gives is an opportunity to organize our existing tests, but also test the test suite functionality using itself. I added some tests, making sure to see each one fail before fixing it and getting to the next:

class NothingRaised < StandardError
end

class AssertionTests < TestSuite
  def test_true
    assert(true)
  end

  def test_truthy
    assert("")
    assert([])
    assert("foo")
    assert(Object.new)
  end

  def test_false
    assert_error { assert(false) }
  end

  def test_falsy
    assert_error { assert(nil) }
  end

  def test_equal
    assert_equal("foo", 'foo')
    assert_equal(1, 1)
  end

  def test_not_equal
    assert_error { assert_equal(1, 2) }
    assert_error { assert_equal("foo", "bar") }
  end

  def assert_error(&block)
    raised = false
    begin
      yield
    rescue AssertionError
      raised = true
    end
    raise NothingRaised unless raised
  end
end

AssertionTests.new.run

We make another suite to test the output of running a test suite:

class OutputTests < TestSuite
  def test_statistics
    suite = Class.new(TestSuite) do
      def test_equality
        assert_equal(1, 2)
      end

      def test_the_truth
        assert(true)
      end
    end

    output = StringIO.new
    report = suite.new.run(output)
    assert_equal(2, report.runs)
    assert_equal(1, report.failures)
    assert(output.string.include?("2 runs, 1 failures"),
           "Report does not include statistics")
  end

  def test_summary
    suite = Class.new(TestSuite) do
      def test_1
        assert(false, "failure1")
      end

      def test_2
        assert(false, "failure2")
      end
    end
    output = StringIO.new
    suite.new.run(output)
    assert(output.string.include?("failure1"),
           "Report does not include error details")
    assert(output.string.include?("failure2"),
           "Report does not include error details")
  end
end

OutputTests.new.run

(Commit 3d47bcf. I also grouped the assertions in a module in commit cfcdd41)

Notice that we're instantiating and running the test suites individually. That's because executing TestSuite.run will run all registered suites, including the tests suites that fail on purpose because they are part of a test case.

Since we run the inner test suites directly, we can remove them from TestSuite's suites. That way they won't be run automatically. It's a bit dirty, but it gets the job done.

class TestSuite
  def self.unregister(suite)
    @suites.delete(suite)
  end
end

Let's define a test helper that we'll use everywhere so that we don't forget to unregister any inner test suites:

def define_suite(&block)
  suite = Class.new(TestSuite, &block)
  TestSuite.unregister(suite)
  suite
end

Example usage:

class ReportingTests < TestSuite
  def test_statistics
    suite = define_suite do
      def test_equality
        assert_equal(1, 2)
      end

      def test_the_truth
        assert(true)
      end
    end

    output = StringIO.new
    report = suite.new.run(output)
    assert_equal(2, report.runs)
    assert_equal(1, report.failures)
    assert(output.string.include?("2 runs, 1 failures"),
           "Report does not include statistics")
  end
end

After we replace all previous inner test suite instantiations with our helper, we can replace AssertionTests.new.run and ReportingTests.new.run with a single TestSuite.run.

(Commit 6fa98df)

Auto-runner

It would be cool to not have to invoke TestSuite.run at all. With Minitest, you can require mintiest/autorun and executing the script automatically runs the tests within. Let's implement that!

The at_exit method takes a block that will get executed when the program exits. The end of the program is a good place to run tests, because it guarantees that all the production and test code has been loaded.

require_relative './raisin'

module Raisin
  @@at_exit_registered ||= false

  def self.autorun
    unless @@at_exit_registered
      at_exit { TestSuite.run }
      @@at_exit_registered = true
    end
  end
end

Raisin.autorun

In the code above, we make sure that we only register the callback only once in case that file is required multiple times. Let's remove that TestSuite.run from the bottom of our test file and replace the require_relative './raisin' by require_relative './autorun'. Run it:

$ ruby test.rb
......

6 runs, 0 failures
..

2 runs, 0 failures

It works!

(Commit 2a53359)

Better reports

Our test output is useful, but it could be better. We don't want it broken down by class, we want to combine the progress reports and print a grand summary at the end. We would also like failures to be reported where the assertion was made and not at the line of the library internals where the exception was raised.

First, we change TestSuite#run to take the report as argument, TestSuite.run to pass it in and we update our tests:

class TestSuite
  def self.run(io = $stdout)
    @suites.each do |suite|
      suite.new.run(Report.new(io))
    end
  end

  def run(report)
    # ...
  end
end

# in the tests
suite = define_suite { #.... }
output = StringIO.new
suite.new.run(Report.new(output))

Then we make only one report and pass to the different test suites. In the end we print the summary.

class TestSuite
  def self.run(io = $stdout)
    report = Report.new(io)

    @suites.each do |suite|
      suite.new.run(report)
    end

    report.summarize
  end

  def run(report)
    test_names = public_methods(false).grep(/^test_/)

    test_names.each do |test|
      result = TestResult.from do
        setup
        send(test)
        teardown
      end
      report.add_result(result)
    end
    report
  end
end

Since the run instance method no longer creates a report by itself and it does not invoke report.summarize we need to add those steps to the tests to make them pass.

class ReportingTests < TestSuite
  def test_statistics
    suite = define_suite do
      def test_equality
        assert_equal(1, 2)
      end

      def test_the_truth
        assert(true)
      end
    end

    output = StringIO.new
    report = suite.new.run(Report.new(output))
    report.summarize

    assert_equal(2, report.runs)
    assert_equal(1, report.failures)
    assert(output.string.include?("2 runs, 1 failures"),
           "Report does not include statistics")
  end

  def test_summary
    suite = define_suite do
      def test_1
        assert(false, "failure1")
      end

      def test_2
        assert(false, "failure2")
      end
    end

    output = StringIO.new
    report = Report.new(output)
    suite.new.run(report)
    report.summarize

    assert(output.string.include?("failure1"),
           "Report does not include error details")
    assert(output.string.include?("failure2"),
           "Report does not include error details")
  end
end

The test setup is a little heavy, but I like it more than the alternatives I can think of. We could declare TestResults instead of a test suite and feed those to the report (with report.add_result(result)). That involves too many library internals. We also cannot use the TestSuite.run facade, because at this point we're already in code executed by it. That would effectively run a loop until we reach the call stack depth limit. Besides, we're already verifying that reports are wired up correctly through visual inspection: running our own test suite produces output. It's the same with registering test suites through inheritance: because we always make a test fail first, we can tell something is wrong if we don't see the failure.

(Commit 9f45081)

Let's also exclude raisin internals from the backtrace. First we'll organize the project files like this:

$ tree
.
├── lib
│   ├── raisin
│   │   └── autorun.rb
│   └── raisin.rb
├── README.md
└── test
    └── raisin.rb

Don't forget to change the require_relative paths. Also note test.rb is now test/raisin.rb.

(Commit 17454c3)

Then we exclude backtrace lines referring to lib/raisin.

class Report
  def summarize
    io.puts
    @errors.each do |failure|
      io.puts
      io.puts failure.message
      io.puts filter(failure.backtrace)
      io.puts
    end
    io.puts
    io.puts "#{runs} runs, #{failures} failures"
  end

  private

  def filter(backtrace)
    backtrace.rejct { |line| line =~ /lib\/raisin/ }
  end
end

(Commit 088fbd2)

Random execution order

A nice feature for a test framework is the execution of test cases in a random order. This helps improve the quality of your tests by detecting dependencies. The tests should pass no matter the execution order. Another aspect of this is that if a certain ordering makes the tests fail, the programmer should be able to reproduce that particular run. To make the process deterministic we'll take a random seed as an optional command line argument and generate a default if none is given.

First, we introduce a new entry point:

module Raisin
  def self.run(args)
    TestSuite.run
  end
end

We modify Raisin.autorun to call it with the command-line arguments:

module Raisin
  @@at_exit_registered ||= false

  def self.autorun
    unless @@at_exit_registered
      at_exit { Raisin.run(ARGV) }
      @@at_exit_registered = true
    end
  end
end

We parse the arguments and pass them to the test suite runner.

require 'optparse'

module Raisin
  def self.run(args)
    options = RunOptions.parse(args)
    TestSuite.run(options)
  end
end

class RunOptions
  attr_reader :seed

  def self.parse(arguments = [])
    program_options = {}

    parser = OptionParser.new do |options|
      options.on('-h', '--help', 'Display this help') do
        puts options
        exit
      end

      options.on('--seed SEED', Integer, 'Set random seed') do |value|
        program_options[:seed] = value.to_i
      end
    end

    parser.parse!(arguments)

    new(program_options.fetch(:seed, Random.new_seed))
  end

  private

  def initialize(seed)
    @seed = seed
  end
end

There are a few things going on here. OptionParser is from the standard library and does the heavy lifting of parsing arguments and pretty printing help. program_options.fetch(:seed, Random.new_seed) makes sure we always know what the random seed is.

Then we shuffle the test order using a random number generator seeded with that seed.

class TestSuite
  def self.run(io = $stdout, options)
    report = Report.new(io, options)

    @suites.each do |suite|
      suite.new.run(report, options)
    end

    # ...
  end

  def run(report, options)
    # ...

    test_names.shuffle(random: Random.new(options.seed)).each do |test|
      # ...
    end

    # ...
  end
end

We could also srand at the beginning of the program. However, that could interfere with the code being tested if that also contains calls to srand. Furthermore, passing in the seed makes the code more explicit and testing it easier.

Finally, we let the user know how to repeat that specific test order:

class RunOptions
  def invocation_command
    ['ruby', $PROGRAM_NAME, '--seed', seed].join(" ")
  end
end

class Report
  def initialize(io, options)
    # ...
    @options = options
  end

  def summarize
    # ...
    io.puts
    io.puts 'Rerun the tests in the same order with:'
    io.puts options.invocation_command
  end

  private

  attr_reader :options
end

Testing the feature can be as simple as finding a seed that will run tests in a specific order and checking that order:

def test_order
  suite = define_suite do
    def test_1
      assert(false)
    end

    def test_2
      assert(true)
    end
  end

  output = StringIO.new
  options = RunOptions.parse(%w[--seed 2])
  report = Report.new(output, options)
  suite.new.run(report, options)

  assert_equal('.F', output.string)
end

The natural output order here would be "F." and we check it is reversed with a specific seed. Removing the seed (options = RunOptions.parse([])) will sometimes fail the test, but with a seed of 2 it always passes.

(Commit 991a00e)

Cleanup

Before we wrap up, let's put all classes and modules in the Raisin module. That way we don't pollute the global namespace. The tests need updating as well to prefix everything in our library with the top-level namespace.

(Commit a5b7220)

Final words

The cloc utility counts 151 lines of Ruby in the lib/ directory and 96 lines in test/. That is not a lot for a fully functional test library.

We're missing a few useful assertions, like a test that two floating point numbers are very close (we cannot use assert_equal because of machine precision). This and other assertions can be easily built on top of the assert primitive.

I also left out test doubles on purpose. I may revisit the project to add implement them later.

Another thing to implement is a more helpful comparison of expected and actual input for assert_equal.

All in all, writing this library satisfied my own curiosity about self-tested code. I hope the post inspired you to write your own toy test library!

See the final code.

Elm Single Page Application Setup

2016-08-03T00:00:00Z

Update (2018): While the ideas in this post are still valid elm-live will get you started a lot faster. Also check out the Elm guide section on minification.

elm-reactor is good for iterating quickly on self-contained programs. You run elm-reactor, you navigate to the source file that contains the entry point to your program and you are up and running. elm-reactor automatically compiles your program and embeds it in an HTML page. You refresh that page when you want to try out modifications to your program or read the compiler errors.

When you start a bigger project you may need to customize that bootstrap page to, say, include external CSS or JavaScript. The Elm 0.17 reactor gives you no control over that. You need to compile your program to a JavaScript file with elm-make using the --output flag and then reference that in your custom HTML. Now there is an extra step in the development workflow.

When I started developing a single page application in Elm, I decided to make this workflow as nice as possible for myself by automating the build process. I came up with something that works well for me and that I want to share. Before we get into the specifics, however, let's see what needs my tooling solves.

Requirements and assumptions

Use make. This is a matter of personal preference. You can find other resources if using Gulp or Webpack is more suitable for your project.
Static, self-sufficient build artifact: everything must be contained within a single directory that can be served by a production server or a CDN.
Compression: minifying the JavaScript output of the Elm compiler is an easy win for loading speed.
External CSS: elm-css is an interesting way to handle styling, but the warning about it being a beta release helped me decide to stick to what I know for now. This should also be minified.
External JS: some things are currently not doable directly in Elm, like programmatically setting focus to a form field. JavaScript interoperability solves this.
Images and web fonts: no processing is necessary for these assets, but they need to be copied to the output directory.
Development/production build parity: since static files are going to be served in production, you might as well serve the exact same files in development.
Automatic rebuild on file change to reduce the number of manual steps to what we have with elm-reactor. The Elm compiler is fast after the first build since it keeps unchanged modules in cache, therefore this is relatively fast. The minification phase is a potential bottleneck depending on the app size.
Automatic browser refresh: switching to your browser window and refreshing the page are two extra steps that you needn't take.
Node.js is installed. The easiest way to install Elm is through npm and there are a lot of npm packages that address our build needs, making Node.js a good starting point.

Bootstrapping Elm

In a new project directory, create the npm configuration file package.json:

npm init

Install Elm as an npm package.

npm install --save-dev elm

I usually create the Makefile at this point and add a single target: setup. That is the convention I use for installing dependencies in projects that use make.

setup:
	npm install

Make sure to document in the project's README that the project is set up with make setup (or just npm install if you didn't add a setup target in the Makefile).

Hello, world!

Create a src/ directory to hold your Elm program. Write a simple main module in src/Main.elm for testing:

module Main exposing (main)

import Html

main =
    Html.text "Hello, world!"

Compile it with:

./node_modules/.bin/elm-make src/Main.elm

This will ask to install some Elm packages before it can compile. It will automatically create the default elm-package.json. Update it to point to src/ as a source directory. Otherwise, once you start adding new modules, elm make will complain about not finding them. It should look something like this (the important bit is "source-directories"):

{
    "version": "1.0.0",
    "summary": "Project summary",
    "repository": "https://github.com/user/project.git",
    "license": "BSD3",
    "source-directories": [
        "src/"
    ],
    "exposed-modules": [],
    "dependencies": {
        "elm-lang/core": "4.0.3 <= v < 5.0.0",
        "elm-lang/html": "1.1.0 <= v < 2.0.0"
    },
    "elm-version": "0.17.1 <= v < 0.18.0"
}

Now we can add a compilation step to the Makefile.

BIN = ./node_modules/.bin
SRC = src
BUILD = build

build: build-directory js

build-directory:
	mkdir -p $(BUILD)

js:
	$(BIN)/elm make src/Main.elm --output $(BUILD)/app.js

I declared some variables at the top, because we are going to use these paths a lot. Now running make compiles our program and writes it to build/app.js. This is everything we need for the Elm part even as the project grows. The compiler will bundle all imported modules.

Let's add our custom src/index.html that imports app.js.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Elm app</title>
  </head>
  <body>
  </body>
  <script src="app.js"></script>
  <script>Elm.Main.fullscreen();</script>
</html>

The corresponding Makefile section simply copies that file to the build directory.

html:
	cp $(SRC)/index.html $(BUILD)/index.html

When adding new build steps, don't forget to add them as dependencies to the build target in the Makefile. Right now, it should look like this:

build: build-directory js html

Automatic rebuilds on file changes

The chokidar-cli package provides a command-line utility that watches files for changes and runs a command in reaction. In this case, we want to run make when files in the src/ or assets/ directories are modified. src/ contains the source code and all the stylesheets, web fonts and images are stored in assets/.

watch:
	@$(BIN)/chokidar "$(SRC)" "assets" -c "make"

Cool! We can use our elm-reactor-like workflow by running make watch and opening build/index.html. Whenever we save a file, we can see the changes by refreshing the browser page. One difference is that the compiler errors are displayed in the terminal instead of the browser.

Now is a good time to document the effects of make and make watch in your README.

Live reloading

Figwheel provides a wonderful development experience for ClojureScript projects. We can get close to it thanks to BrowserSync.

build: build-directory html js css fonts images
	$(BIN)/browser-sync reload

browser-sync:
	$(BIN)/browser-sync start --server $(BUILD)

Running make browser-sync will open the app in your browser and automatically refresh it after each build (note that we added the command browser-sync reload to the build target). What's more, if you access the same URL from another browser, even other devices, all instances will be refreshed without manual intervention.

Depending on the way your application derives its state you may lose state between reloads, but that is already the case with manual reloads. How you can achieve seamless reloads is a topic for another time.

Minification

The Google Closure compiler requires Java, so I opted for UglifyJs, which gives good results. I enabled name mangling and compression. This reduces the file size significantly (from 164Kb down to 64Kb for the Hello World example). Gzip reduces the size even further (down to 19Kb), but gzipping can be done by the web server.

js:
	$(BIN)/elm make $(SRC)/Main.elm \
		--output $(BUILD)/app.js
	$(BIN)/uglifyjs --compress --mangle \
		--output $(BUILD)/app.min.js \
		$(BUILD)/app.js 2> /dev/null
	mv $(BUILD)/app.min.js $(BUILD)/app.js

I redirect the UglifyJS error stream to /dev/null, because it just contains warnings about the minification of things in the Elm standard library: "Side effects in initialization of unused variable", "Dropping unused variable", "Dropping unused function". Ignoring warnings is generally not a good idea, but in this case:

The dead code elimination phase doesn't drop any of the unused variables with "side effects in initialization".
Minification with --compress --mangle doesn't impair the program. Maybe more advanced options do.
Even if something went wrong, we would notice since we are using the production build in development.

Stylesheets

clean-css works great for my purposes. It takes your concatenated CSS files as input and creates a minified stylesheet. If you need Sass, node-sass has a command line interface that can be used in a similar way.

CSS = assets/stylesheets

css:
	cat $(CSS)/*.css | \
		$(BIN)/cleancss --output $(BUILD)/style.css

Import the stylesheet in the <head> section of src/index.html:

<link rel="stylesheet" type="text/css" href="style.css" />

Final result

The package.json configuration file looks something like this:

{
  "name": "elm-spa",
  "version": "1.0.0",
  "private": true,
  "description": "Elm single page application",
  "dependencies": {},
  "devDependencies": {
    "browser-sync": "^2.14.0",
    "chokidar-cli": "^1.2.0",
    "clean-css": "^3.4.18",
    "elm": "^0.17.1",
    "uglify-js": "^2.7.0"
  },
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "UNLICENSED"
}

The build directory contains the following after I run make:

$ tree -L 1 build
build
├── app.js
├── fonts
├── images
├── index.html
├── setup.js
└── style.css

2 directories, 4 files

setup.js contains the JavaScript code that embeds the Elm program and sets up ports. In the final Makefile below you can see that I also compress setup.js.

BIN = ./node_modules/.bin
SRC = src
BUILD = build
CSS = assets/stylesheets
FONTS = assets/fonts
IMAGES = assets/images
HTML = src/index.html

build: build-directory html js css fonts images

build-directory:
	mkdir -p $(BUILD)

html:
	cp $(HTML) $(BUILD)/index.html

js:
	$(BIN)/elm make $(SRC)/Main.elm \
		--output $(BUILD)/app.js
	$(BIN)/uglifyjs --compress --mangle \
		--output $(BUILD)/app.min.js \
		$(BUILD)/app.js 2> /dev/null
	mv $(BUILD)/app.min.js $(BUILD)/app.js
	$(BIN)/uglifyjs --compress --mangle \
		--output $(BUILD)/setup.js \
		$(SRC)/setup.js 2> /dev/null

css:
	cat $(CSS)/*.css | \
		$(BIN)/cleancss --output $(BUILD)/style.css

fonts:
	cp -r $(FONTS) $(BUILD)

images:
	cp -r $(IMAGES) $(BUILD)

watch:
	$(BIN)/chokidar "$(SRC)" "assets" -c "make"

browser-sync:
	$(BIN)/browser-sync start \
		--server $(BUILD) --files $(BUILD)

setup:
	npm install

Make sure to use tabs for indentation or make will fail with a cryptic error message.

Possible improvements

Deployment automation

It would be good to be able to deploy the app in a single step by running

make deploy

This depends on where you deploy, so I left it out.

Cleaner `make watch` logs

By default make prints a command before executing it. It also prints "Entering directory" and "Leaving directory" messages. All of this creates a lot of noise and we only care about compiler errors. There are ways to silence make. For instance, prefixing a Makefile line with @ prevents make from printing it before execution. I find it best to leave the defaults for manual builds. For builds triggered by file changes during development I completely disable make output with the -s flag:

watch:
	$(BIN)/chokidar "$(SRC)" "assets" -c "make -s"

I also like to add some blank lines to distinguish the output of consecutive builds:

build: build-directory html js css fonts images
	@$(BIN)/browser-sync reload
	@echo -e "\n\n\n\n"

API Response Serialization with Plain Old Ruby Objects and ActiveSupport

2016-07-30T00:00:00Z

If you are building an API application with Ruby on Rails, you will notice that the jbuilder gem is suggested in the default Gemfile. Jbuilder allows you to create JSON using a builder domain specific language. Some people prefer ActiveModel::Serializers which has its own DSL, is more declarative and handles structural conventions for you. This is convenient if you want your API to conform to the JSON API specification or a custom format.

While these gems (and others) each have their trade-offs, in some projects, they are more of a hindrance than help. I have another serialization strategy for these cases.

The Ruby standard library supports encoding primitives to JSON and ActiveSupport adds a layer on top of that to allow serialization of any Ruby object by first serializing it to a Hash. In my code, I build a hash directly:

class PostSerializer
  def initialize(post)
    @post = post
  end

  def serialize
    {
      title: post.title,
      author: post.author
    }
  end

  private

  attr_reader :post
end

Then in a controller, I would have something like:

def show
  post = Post.find(params[:id])
  render json: PostSerializer.new(post).serialize.to_json
end