Proof-of-concept: Unit tests and OO principals in Node.js

TL;DR

For the last few weeks I’ve been trying to make a significant back-end change to Webmaker.org’s Login server.  I was told that, because of issues with scaling across servers, we needed to switch the technology we use to store user accounts from a NoSQL solution (MongoDB) to a more standard relational database (MySQL).  

I’m nearing the end of the marathon now, and looking back on the process has led to some observations about big back-end revisions, unit-testing and software architecture in Node.js, as well as some general thoughts about storage solutions.

About those observations?

I originally baked database logic into the login server, instead of into its own object with a distinct unchanging interface. Later, to encourage reusability, I created wrapper methods to encapsulate the logic that interacted with the database. Also, I had written unit tests that had little practical demonstration of their purpose – until we had to switch to MySQL.

I could see errors immediately, and isolate where they came from since I had encapsulated the database helper object’s purpose and function so well. Major, major win with a drastic reduction in development & testing time.

The full scoop

OO Principals, Express.js Architecture and Unit Tests

In its infancy, I was responsible for the development of the Login server’s approach to persistent data storage. Since I lacked experience, I was happy to be told which storage back-end to use. After all, relational databases could get horrendously complicated, and even though I’m fairly good at normalization, the thought of engineering a production-grade schema was daunting.

So, when my fantastic co-developers at CDOT & Mozilla strongly suggested MongoDB, I was happy to try it. Using a Node.js wrapper module called MongooseJS, I spun up the initial logic for the RESTful API calls that would need to access or modify our database.

If you’re familiar with Node.js (and specifically ExpressJS) routes, you know that when a particular route is hit, say “POST /user”, a callback function is executed that determines which response to send to the client.

The logic that interacted with the database was baked into these callbacks, meaning that the RESTful calls were, at first, highly coupled with the database technology I was using:

exports.get = function ( req, res ) { 
  var id = req.params.id,
      query = {},
      field = "email";

  // Parse out field type
  if ( id.match( /^\d+$/g ) ) {
    field = "_id";
  } else if ( id.match( /^[^@]+$/g ) ) {
    field = "username";
  } 
  query[ field ] = id;

  UserHandle.findOne( query, callback );
};

Here “UserHandle” is a direct link to Mongoose, which is a direct link to the MongoDB instance. “findByID” (line 14) is a Mongoose method, and the logic before that call is parsing data into a form Mongoose can accept. This makes this “GET” request logic irreconcilably tied to the storage solution.

It might have been the cries of my college professor’s ghost, or my own instinct for good design, but something about this irked me. I built an object that would expose a clearly defined set of methods for interacting with whatever database lay underneath it:

  /**
   * Model Access methods
   */  
  return {
    /**
     * getUser( id, callback )
     * -
     * id: username, email or _id
     * callback: function( err, user )
     */      
    getUser: function( id, callback ) {
      var query = {},
          field = "email";

      // Parse out field type
      if ( id.match( /^\d+$/g ) ) {
        field = "_id";
      } else if ( id.match( /^[^@]+$/g ) ) {
        field = "username";
      } 
      query[ field ] = id;

      model.findOne( query, callback );
    },
    ... 
    // Other access methods
    ...
  };

See how much good there is there? Lines 6-9 create a definition that
can be relied on not to change, and all of the backend-specific calls (line 23) and database specific logic (lines 12-21) are contained and can be modified without changing how the helper object is used.

It allowed me to modify the “GET” logic I previewed earlier into a form that wouldn’t need to be changed if the database scheme was updated:

  controller.get = function ( req, res ) {
    var id = req.params.id;

    UserHandle.getUser( id, function ( err, user ) {
      // Response logic omitted for brevity
    });
  };

See how much cleaner that is? Beautiful.

Around the same time my supervisor (the talented Dave Humphrey) was insisting that we add unit testing to the server. I’ve already covered the benefits of unit testing in Node.js in a previous post, but it was all theoretical.

It was almost as if he knew they would be helpful later! Almost as if major revisions to underlying technology happen all the time! Well we must have been on the same wavelength, because my application of OO principles combined with the unit tests I built with his assistance are the only reason my marathon is near its end.

Wasn’t that easy?

Implementing MySQL repeatedly broke the functionality of the Login server. It would whine, and complain, and break, and regurgitate error stacks a mile high – but it was all visible. It was all readable, and most importantly it was all traceable through the combination of extensive unit tests and the isolation of changing components through OO principals.

When I made a change, it was to my “UserHandle” object’s implementation, not to its design. This made for a change in one, very predictable place. Then, after making the change, I had immediate feedback from the test suite about whether or not the world came crashing down as a result:

Ahhhh…. Now that feels good!