Writing a Node Module

This example assumes that you have a Github account. If you don't have one, you can get one at (suprise!) Github. It also assumes that you have installed Node and npm. It will also simplify your life to have the github gem installed

Create the basic structure

# Create a directory
$ mkdir sleep-sort
# cd into it
$ cd sleep-sort
# Initialize the Git repo
$ git init

OK, now I have an initialized repository and I need something to put in it. I like Readme Driven Development and, in that spirit, let's create a Readme.

# Create a Readme file with my favorite editor
$ vi Readme.md
$ cat Readme.md 
# Sleep Sort

`sleep-sort` works by creating a task for every number to be sorted. The task
will "sleep" n number of milliseconds and then push the number onto an array.
When all tasks are done, the array is returned sorted.

# Add all files to the repo
$ git add .
# Commit 
$ git commit -m 'Added Readme'

I have setup a local repository and committed our Readme. Now, I need to create our remote repository on Github. With the Github gem, this is trivial.

# Create the remote repo at Github with the Github gem.
$ gh create-from-local
Counting objects: 3, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 359 bytes, done.
Total 3 (delta 0), reused 0 (delta 0)
To git@github.com:andersjanmyr/sleep-sort.git
 * [new branch]      master -> master

The above creates the remote repository, pushes the local changes, and sets up the origin. I can of course do all the above manually, but why?

Making the module

Even though this is only going to be a small module, I want to create directories for the different files I plan to use.

# Create lib, test, and bin dirs
$ mkdir lib test bin

I created a lib dir for the source files, a test dir for the tests, and a bin dir for the shell command, sleep-sort, to go in.

Lets also create the files.

$ touch lib/main.js test/main.js
$ git add lib test
$ git commit -m 'Added initial files'

OK, the directories exists and the files exist, it's time to make the module. To do this I need to create a package.json file that contains meta-information about the module. An easy way to do this is by using npm init.

$ npm init
Package name: (sleep-sort) 
Description: Implementation of the sleep-sort algorithm.
Package version: (0.0.0) 0.0.1
Project homepage: (none) 
Project git repository: (git://github.com/andersjanmyr/sleep-sort.git) 
Author name: Anders Janmyr
Author email: (none) anders@email.com
Author url: (none) http://anders.janmyr.com
Main module/entry point: (none) lib/main.js
Test command: (none) mocha test/*.js
What versions of node does it run on? (~0.6.14) 
About to write to /Users/andersjanmyr/Projects/sleep-sort/package.json

{
  "author": "Anders Janmyr <anders@email.com> (http://anders.janmyr.com)",
  "name": "sleep-sort",
  "description": "Implementation of the sleep-sort algorithm.",
  "version": "0.0.1",
  "repository": {
    "type": "git",
    "url": "git://github.com/andersjanmyr/sleep-sort.git"
  },
  "main": "lib/main.js",
  "scripts": {
    "test": "mocha test/*.js"
  },
  "engines": {
    "node": "~0.6.14"
  },
  "dependencies": {},
  "devDependencies": {}
}

Is this ok? (yes) yes

npm init guesses some of the information, so that I don't have to enter it myself. When the process is done I have a package.json file in the root directory of our project, lets add it to git and commit it.

$ git add package.json
$ git commit -m 'Initial package.json'

Note that I added lib/main.js as the main file. This will be the file that is required when Node requires the module later on. I also added the test script mocha test/*.js, this script can be invoked by running npm test.

But, in order for this to work, I need to add mocha as a development dependency in package.json. And while I am at it, I will also add should.js as a dependency. should.js provides should-style assertions.

//package.json
{
 ...
 "devDependencies": {
   // Use any version
   "mocha": "",
   "should": ""
 }
}

Install the dependencies with npm install

$ npm install
npm http GET https://registry.npmjs.org/should
npm http GET https://registry.npmjs.org/mocha
...
should@0.6.0 ./node_modules/should 
mocha@1.0.0 ./node_modules/mocha 
|-- growl@1.5.0
|-- debug@0.6.0
|-- commander@0.5.2
|-- diff@1.0.2
|-- jade@0.20.3

Try the command out, by running npm test and watch it fail since I don't have any proper tests yet.

npm install created a directory called node_modules containing all the dependencies. I don't want to check this into source control so I add a .gitignore file.

$ echo node_modules/ > .gitignore
$ git add package.json .gitignore
$ git commit -m 'Added and installed dependencies and ignored them.'

Some Code

That was a lot of stuff without writing a single line of code. Enough of that. I'll do it TDD style and write a test first.

// test/main.js
var should = require('should');
var sleepsort = require('../lib/main');

describe('sleepsort', function() {
    describe('with no arguments', function() {
        it('returns an empty array', function() {
            var result = sleepsort();
            result.should.eql([]);
        });
    });
});

If I run this, npm test I get an error since I haven't implemented a module in lib/main.js. Time to do that. Since I am only going to be exporting a single function, sleepsort, from this module I will export it via module.exports, instead of exports.sleepsort.

function sleepsort() {
  return [];
}


module.exports = sleepsort;

The above code gets the first test passing. Notice that I required the function directly with require('sleepsort'), in the test above, this works because of the above mentioned export method.

$ npm test
> sleep-sort@0.0.1 test /Users/andersjanmyr/Projects/sleep-sort
> mocha test/*.js
  .
    OK 1 tests complete (2ms)

# Add the files, and commit them.
$ git commit -a 'Added a first running test for sleepsort'

Now when I have a running test, it's time to set up continuous integration.

Continuous Integration

The easiest way to do this is to use Travis-CI. I need to do three things (a more complete description can be found here)

1. Sign in to (or sign up with) Travis with my Github account

2. Go to the Travis profile page

Find my repository, sleep-sort, and turn it on. This will turn on the Travis commit hook on Github, and it will run my tests, every time I push to Github. Travis will run the command I specified in package.json under the scripts/test entry, by running npm test.

3. Add a .travis.yml

Create a .travis.yml, like this.

language: node_js
node_js:
  - 0.6
  - 0.7

I'm telling Travis to run the tests with both version 0.6 (stable) and 0.7 (unstable). Then commit this file, and push it to Github.

$ git add .travis.yml
$ git commit -m 'Added .travis.yml to run agains node 0.6 and 0.7.'
$ git push

After a while you will get an email, telling you that the tests were run successfully. Relax, enjoy the beauty of Node, of Travis, of life!

Additionaly, if you want to show off that you are a responsible citizen that uses continuous integration you can add a Travis status image to the Readme. It looks like this:

The Binary

Now I have everything setup with testing and CI. I have a function that can sort empty arrays, instantly :), but I don't yet have a script to run it. Let's create bin/sleep-sort

#!/usr/bin/env node
var sleepsort = require('../lib/main');

var args = process.argv.slice(2);
console.log(sleepsort(args));

I also need to register the script as a binary in package.json. Add the following line to package.json.

  "bin": { "sleep-sort": "./bin/sleep-sort" },

And make it executable, try it, add it and commit it.

$ chmod a+x bin/sleep-sort
$ bin/sleep-sort
[]
$ git add bin
$ git commit -m 'Added a binary to the module'

Beautiful, time for another break :)

The Rest of the Code

That was a lot of work for a function that is only able to sort an empty array. And, there is something fishy going on. That function I implemented seems very synchronous, isn't Node supposed to be asynchronous?

Well, gosh! Let me fix that right away.

First I change the test. Since the code now is going to be asynchronous I will take advantage of some nifty features of mocha. If I give the callback function of it an argument done, mocha will inject a function that I can call when the test is done. Like this:

    describe('with an empty array argument', function() {
        it('calls the callback with an empty array', function(done) {
            var result = sleepsort([], function(result) {
                result.should.eql([]);
                done();
            });
        });
    });

If I hadn't called done, mocha would timeout and the test would be marked as a failure. Simple, yet powerful. With a new failing test, I update the actual code.

function sleepsort(array, callback) {
    return process.nextTick(function() {callback([]);});
}

process.nextTick tells Node that I want the function to be called next time the loop comes around. And the test is passing again, and now I can sort an empty array asynchronously. How about that!

Commit it and push it! Then, rejoice as Travis verifies it and emails that everything is fine.

Time to add one more test, a single element array.

describe('with a single element array', function() {
    it('calls the callback with a single element array', function(done) {
        var result = sleepsort([1], function(result) {
            result.should.eql([1]);
            done();
        });
    });
});

The test fails and I am ready to fix it. Here is a function that will pass the test.

function sleepsort(array, callback) {
    if (!array || array.length === 0)
        return process.nextTick(function() {callback([]);});
    var result = [];
    function appendResult(n) {
        return function() {
            result.push(n);
            callback(result);
        };
    }
    for(var i = 0; i < array.length; i++) 
        setTimeout(appendResult(array[i]), array[i]);
}

Take an extra look at appendResult. It's a function that creates another function and returns it. The created function is the function that will be called when the timeout fires.

Now I have a function that sorts single element arrays too. Let's see if it can sort more than that. One more test.

describe('with an unsorted two element array', function() {
    it('calls the callback with a sorted two element array', function(done) {
        var result = sleepsort([2, 1], function(result) {
            result.should.eql([1, 2]);
            done();
        });
    });
});

Our test doesn't pass, I need to make sure the callback is only called when the result is complete. How do I know that it is complete? I know that if the array of the result is the same length as the input array. Here is the final function.

function sleepsort(array, callback) {
    if (!array || array.length === 0)
        return process.nextTick(function() {callback([]);});
    var result = [];
    function appendResult(n) {
        return function() {
            result.push(n);
            if (array.length === result.length) 
                callback(result);
        };
    }
    for(var i = 0; i < array.length; i++) 
        setTimeout(appendResult(array[i]), array[i]);
}

Now, the last thing I need to do is to update the script to use the asynchronous code. That is simple, I just give console.log as a the callback parameter to sleepsort.

#!/usr/bin/env node
// bin/sleep-sort
var sleepsort = require('../lib/main');

var args = process.argv.slice(2);
sleepsort(args, console.log);

Done. The program is working. I have a sorting algorithm with linear complexity working! Linear to the highest value to be sorted. :D

With this mighty fine algorithm running, I need to publish it for the world to see it. How do I do that? With npm of course.

Publishing the Module

Before I publish, I try it out locally by installing it.

$ npm install . # from the project dir
$ sleep-sort 5 3 7 9 1
[1,3,5,7,9]

It works on my machine :). It may not on yours, depending on how your path is setup. If it doesn't, you may try to install it globally with npm install -g ., then try again.

Now, when it is working, it is time to publish. If this is the first time you publish something with npm, you have to add yourself as a user before you publish.

$ npm adduser
...

Before I publish, I bump the version up to 1.0.0, in package.json, to show that this package is ready for production.

$ npm publish .
npm http PUT https://registry.npmjs.org/sleep-sort
npm http 409 https://registry.npmjs.org/sleep-sort
npm http GET https://registry.npmjs.org/sleep-sort
npm http 200 https://registry.npmjs.org/sleep-sort
npm http PUT https://registry.npmjs.org/sleep-sort/1.0.1/-tag/latest
npm http 201 https://registry.npmjs.org/sleep-sort/1.0.1/-tag/latest
npm http GET https://registry.npmjs.org/sleep-sort
npm http 200 https://registry.npmjs.org/sleep-sort
npm http PUT https://registry.npmjs.org/sleep-sort/-/sleep-sort-1.0.1.tgz/-rev/4-9dcdcb2bab4176ca5aad6f13c479994e
npm http 201 https://registry.npmjs.org/sleep-sort/-/sleep-sort-1.0.1.tgz/-rev/4-9dcdcb2bab4176ca5aad6f13c479994e
+ sleep-sort@1.0.1

That is all. The source code can be found on Github, if you want to play with it yourself.