Find My Team

Introducing Crosby

Image of engineering blog author: Dan MatthewsDan Matthews

Recently, we've been working on migrating email from our 'Classic' codebase into our 'NextJenn' architecture. If you've ever worked with ActionMailer emails, you know that this is a bigger task than it sounds.

One issue in particular has proven very tricky. Due in part to differing regulations in different geographic locations (I'm looking at you, Canada) and in part to the team level customization possible, there are dozens of different permutations for many of the several dozen emails that our platform sends. Verifying each email would take more time than our QA staff has at their disposal. Our team needed a solution to compare (or diff) the outputs of each individual email permutation in the 'Classic' codebase to it's contemporary in 'NextJenn'.

Crosby is that solution.

Why Crosby?

Like many good open source projects, I decided to name this one after a historical figure from Quality Assurance Land. Philip Crosby (6/18/26 - 8/18/01) was a quality control expert in the aerospace industry. While working at The Martin Company, Crosby is credited with developing the Zero Defects concept.

According to the Wikipedia Article, "Zero Defects seeks to directly reverse the attitude that the amount of mistakes a worker makes doesn't matter since inspectors will catch them before they reach the customer."

I could not think of a more effective way to describe our email migration challenge. Those of us who are taking on this task need to catch all mistakes. We can not rely on our 'inspectors', the QA team, to catch them for us.

Approach

Early on, we figured that the best way to compare the emails generated by each platform would be a diff. This raises one fairly decent sized challenge:

Common Output Format

In our 'Classic' platform, we're using a version of ActionMailer 2. Some of the emails were originally written in ActionMailer 1. The 'NextJenn' platform uses the most recent version of ActionMailer 4.

Here are segments of output from the same email in ActionMailer 2 & 4:

<!-- ActionMailer 2 -->

<table bgcolor=3D"#ffffff" width=3D"100%" border=3D"0" cellpadding=3D"0" =
cellspacing=3D"0" align=3D"center" class=3D"mobileCenter" name=3D"0">
  <tr>
    <td height=3D"35">&nbsp;</td>
  </tr>
</table>

<!-- ActionMailer 4 -->

<table bgcolor="#ffffff" width="100%" border="0" cellpadding="0" cellspacing="0" align="center" class="mobileCenter" name="0">
  <tr>
    <td height="35">&nbsp;</td>
  </tr>
</table>

Obviously, if we try to diff these two code segments, we'll see changes on any line that's longer than 74 characters or contains an equal sign. Spacing is also an issue. If we replace tabs with spaces during the migration, the lines will not match. We also have potential issues with HTML nesting / line breaks.

Solution (from Crosby::Exporter):

    def commonize_html(str)
      commonize_text(str)
      compact(Nokogiri::HTML::Document.parse(commonize_text(str)).to_s)
    end

    def commonize_text(str)
      str
        .gsub(/([^=])=\n/, '\1') # single quotes required for encoding
        .gsub(/=3D/, "=")
    end

    def compact(str)
      str
        .each_line
        .reject(&:blank?)
        .map{ |line|
          line
            .gsub(/\s+/, " ")
            .gsub(/^ /, "")
            .gsub(/ $/, "\n")
            .gsub(/></, ">\n<")
        }
        .join
    end

Nokogiri helps us ensure that the HTML output is generated in a common way. String#gsub is used to reverse the 74 character line breaks, equal sign encoding issue, and a number of whitespace issues in the HTML. We'll likely be adjusting or adding to this block of code as we discover more inconsistencies.

Generation via RSpec

Most of our main email use cases are tested using RSpec, making it an ideal place to generate our output files. In 'Classic', we're still on RSpec 1. In 'NextJenn' we're using a mix of RSpec 2 & 3. The Crosby::RSpecHelper module provides a method, #crosby_email, to all example groups regardless of RSpec version. This method generates the output file for an email while creating and returning the Mail::Message instance.

so this:

mailer = ExampleMailer.notify(arg1, arg1)
# or in ActionMailer 2
mailer = ExampleMailer.create_notify(arg1, arg2)

becomes this:

mailer = crosby_email(ExampleMailer, :notify, arg1, arg2)

As we run our tests, we get the output files for free. Read more...

UUID and Output Files

In most of our use cases, the mailer classes and method names are changed during migration as a result of built up technical debt. UUIDs were introduced to allow common file names (and therefore, automatic comparison) for each mailer. We also created a configuration variable, app_name, to allow for distinction between different platforms or applications. Both the UUID and app_name are used to generate the output file's name.

By default, output files are created in the ./tmp/crosby directory. As with app_name, this value can be set via a configuration variable, export_path:

Crosby.config do |c|
  c.app_name = "ExampleApp"
  c.export_path = "../crosby_files"
end

The default value of app_name is randomly generated on load. While this is helpful for getting started quickly, it also means that a new crosby file will be generated each time you run a test. In most cases, you'll want to set this value in your Crosby.config block.

If you're using the RSpec helper, a UUID will be auto-generated using the arguments passed to #crosby_email. This ensures that the same UUID is created for each unique email call in your tests. I recommend not relying on these defaults. Instead, set the UUID in each test case using the @crosby_uuid instance variable:

@crosby_uuid = "example_mailer_notify"
mailer = crosby_email(ExampleMailer, :notify, arg1, arg2)

Skipping Output File Generation

I ran into an issue as soon as I started thinking about the world outside my local dev environment. At the time of this writing, we use Codeship for continuous integration testing. Codeship wouldn't like it very much if Crosby tried to create a new directory automatically, particularly given my config settings, which tell it to do so in the parent directory.

Luckily, Codeship does allow us to configure environment variables. Setting ENV["SKIP_CROSBY"] will cause file and directory creation to be skipped.

SKIP_CROSBY=true bundle exec rspec spec/mailers/example_mailer_spec.rb

Generating the Diffs

As mentioned above, the end goal is to look at email diffs. The easiest way to do this is to include the crosby:diff task in your application's Rakefile.

# Rakefile
require "crosby/tasks"

Crosby.config do |c|
  c.export_path = "../crosby_files"
end

bundle exec rake crosby:diff

This task will look for every UUID in the configured export directory and run a diff from the perspective of the current application. Read more about diffs...

Wrapping up

The first release version of Crosby was only completed a few days ago, before I had even encountered the need for skipping file creation. It's highly likely that this project will get some upgrades here and there over the next several months. If you are a Ruby dev and would like to contribute, head on over to Github and fork the repository!