A Critique of the Thoughtworks Tech Radar on Javascript Testing

The Thoughtworks' Tech Radar has come out again and there is no change in the recommendation on Javascript testing.

The radar recommends to "Adopt Jasmine paired with Node.js". This is very specific advice. It's not "Adopt Javascript testing paired with Node.js" but a specific tool, Jasmine. Compare this with more general advice such as "Adopt CSS Frameworks" or "Promises for asynchronous programming". Nothing specific there and, hence, nothing wrong.

There is no motivation as to why we should adopt Jasmine but, if we look at the TTF from October 2012(PDF), we can read:

The cream of the crop for out-of-browser testing is currently Jasmine. Jasmine paired with Node.js is the go-to choice for robust testing of both client- and serverside JavaScript.

"The cream of the crop"? This is certainly not the case! Jasmine is an elegant testing framework similar Ruby's RSpec but, it is not "the cream of the crop"! It has one major drawback. It sucks at testing asynchronous code! And, since asynchronous code is a central theme in Javascript programming we should not use Jasmine! There are at least two better alternatives:

• Buster.js
• Mocha

Example

In this example I want to test a simple asynchronous sort function called sleepsort, you can read more about it in Writing a Module.

The function signature looks like this:

// Sleepsort takes an array of numbers and calls callback(result)
// with the sorted array as the only argument
function sleepsort(array, callback) {
...
}

Testing this with Jasmine will look something like this:

it('sorts the array', function () {
  runs(function() {
    this.result;
    sleepsort([1, 3, 2, 7], function(result) {
      this.result = result;
    });
  });

  waits(1000); // Wait one second then check the rsult

  runs(function () {
    expect(this.result).toEqual([1, 2, 3, 7]);
  });

});

This is horrible! Compare this to Mocha, with should.js assertions:

it('Sorts the array', function(done) {
  sleepsort([1, 3, 2, 7], function(result) {
    result.should.eql([1, 2, 3, 7]);
    done();
  });
});

Thank you, Sir, may I have another!

Indeed you may, here it is with Buster.js, with Object style syntax:

"Sorts the array": function (done) {
  sleepsort([1, 3, 2, 7], function(result) {
    assert.equals(result, [1, 2, 3, 7]);
    done();
  });
}

Both Mocha and Buster.js send in a done function to call when the execution has finished. Elegant, and above all, crystal clear.

It is very obvious to me that Jasmine no longer is "the cream of the crop" of Javascript testing. But, what do I know, perhaps it was a copy-paste error.

I've done my duty, now I can go to bed!

Javascript Conditionals

As we all know, Javascript is a very flexible language. In the article I will show different ways to execute conditional code by using some common idioms from Javascript and general object-oriented techniques.

Default values

Javascript does not support default values for arguments and it is common to use an if statement or a conditional expression to set default values.

function swim(direction, speed, technique) {
  // Default value with if statement
  if (!direction) direction = 'downstream';

  // Default value with conditional operator
  var speedInMph = speed ? speed : 2;
}

I usually prefer to use an or-expression instead. The short-circuiting or, ||, avoids the repetition of the conditional operator and is, in my opinion, more readable. Another advantage of avoiding repetition is that a slow executing function condition such as fastestSwimmer() will avoid the performance penalty of calling the function twice.

  // Default value with or.
  var swimTechnique = technique || 'crawl';

  // Function is only invoked once
  var siwmmer = fastestSwimmer() || 'Michael Phelps';
}

Naturally, this technique is not limited to default arguments, it can be used to set default values from object literals too.

options = { kind: 'Mountain Tapir', }
var kind = options['kind'] || 'Baird Tapir';

A simple, yet useful, technique.

Update 2013-04-13, as Jeffery mentions in a comment, the technique only works for values that are not falsy. If values such as 0 or false are acceptable values, you will have to explicitly test for undefined instead.

Call callback if present

Another idiom in Javascript, especially in Node, is passing callbacks to other functions. But, sometimes the callbacks past in are optional. In this case we can use short-circuiting and, &&, instead.

function updateStatistics(data, callback) {
  var result = doSomethingWithData(data);
  // Call the callback if it is defined
  if (callback) return callback(result);
}

function (data, callback) {
  var result = doSomethingWithData(data);
  // The last evaluated value of the `&&` is returned
  return callback && callback(result);
}

I, personally, prefer the first form with the explicit if because I think it communicates my intent better but, it is good to know about the technique anyway.

Update 2013-04-13, A better use for the technique is for testing for the presence of objects before getting their properties.

function callService(url, options) {
  ajaxCall(url, options && options.callback);
}

Lookup Tables

If you have code that behaves differently based on the value of a property, it can often result in conditional statements with multiple else ifs or a switch cases.

if (kind === 'baird')
  bairdBehavior();
else if (kind === 'malayan')
  malayanBehavior();
else if (kind === 'mountain')
  mountainBehavior();
else if (kind === 'lowland')
  lowlandBehavior();
else
  throw new Error('Invalid kind ' + kind);

I find this kind of code ugly and I don't think it looks any better with a switch statement. I prefer to use a lookup table if there is more than two options.

var kinds = {
  baird: bairdBehavior,
  malayan: malayanBehavior,
  mountain: mountainBehavior,
  lowland: lowlandBehavior
};

var func = kinds[kind];
if (!func)
  throw new Error('Invalid kind ' + kind);
func();

I find this code a lot clearer since makes it clear that the else clause handles an exceptional case and that the normal cases works similarly.

Missing Objects

If similar conditionals appear in multiple places in my code, it is a sign that I am missing an object somewhere. Since Javascript is duck typed I can use the same technique as above to create objects instead of just functions.

var kinds = {
  baird: { act: bairdBehavior, info: bairdInfo },
  malayan: { act: malayanBehavior, info: malayanInfo },
  mountain: { act: mountainBehavior, info: mountainInfo },
  lowland: { act: lowlandBehavior, info: lowlandInfo },
};

var tapir = kinds[kind];
if (!tapir)
  throw new Error('Invalid kind of tapir ' + kind);

I prefer to have this kind of code on the borders of my application. That way the code inside my core domain doesn't have to deal with complicated conditional logic. Polymorphism for the win!

Null Objects

If I notice that in many places I have to check fornulls, it is usually a sign that I haven't handled the special null case properly. In the example above I have handled it properly since I throw an Error if the kind of tapir does not exist. But sometimes it is not an error when the value is missing.

// If a non existant kind is used, tapir will become null
var tapir = kinds[kind];

// In other places of the code
if (tapir)
  tapir.act();

// Somewhere else
if (tapir)
  return tapir.info();

This type of code is rather unpleasant and it is time to break out the Null Object.

var tapir = kinds[kind];
// If a non existant kind is used I use a Null Object
if (!tapir)
  tapir = { act: doNothing, info: unknownTapirInfo }

// In other places of the code the conditionals are gone.
tapir.act();

// Somewhere else, no special case here.
return tapir.info();

Null Objects are not appropriate everywhere but, I often find it very enlightening to have them in mind when I write code.

Summary

There are a lot of elegant ways to deal with conditional code in Javascript. I didn't even mention inheritance since it works similarly to the object approach I showed above. But if I need multiple instances of something I would of course use polymorphism through inheritance instead.

Web Workers

I recently wrote a program, Word Maestro, which requires extensive calculations in Javascript. The calculations, permutations and searching, are very CPU intensive and hangs the GUI when performed in the foreground.

Web workers to the rescue! Web workers are supported by most moderns browsers with the exception of IE (Surprise!). IE10 release candidate supports them, but it is not very wide spread yet. More info can be found at Can I Use

How do Web Workers Work?

A web worker is just a plain Javascript file with anything in it. If you start an empty file empty-worker.js it will start up just fine and do absolutely nothing.

// empty-worker.js

To start a web worker you create a new Worker and give the constructor a URL as the only parameter. The URL must come from the same domain as the page loading the Worker.

// This code goes inside a script tag or in a file loaded by <script src>
// Start the empty worker, which does nothing.
var worker = new Worker('empty-worker.js');

In order to have any use for our worker we need it to communicate with us. The way a worker communicates is be sending messages. The method that does this is called postMessage(object). It takes any type of argument, primitives as well as arrays and objects.

// eager-worker.js
postMessage('I am eager for work!');
// self.postMessage('I am eager for work!'); // Safest way

It is also possible to prefix the call with this or self , they both refer to the same WorkerGlobalScope. self is the safest way since that will not change with the calling context the way this does.

Our eager-worker.js starts up and posts a message and we need to receive it. We can do that by setting the onmessage property on our worker reference.

// In script tag or file loaded by script tag
var worker = new Worker('empty-worker.js');
worker.onmessage = function(event) {
  console.log(event);
};

Reloading the page will result in the following output in the console. Notice that the data sent by the worker is available in the data property of the MessageEvent

MessageEvent {ports: Array[0], data: "I am eager for work!", source: null, lastEventId: "", origin: ""}

An alternative way of attaching a listener to the workers is to use addEventListener('message', listener). Adding the event listener this way has the advantage of allowing us to attach multiple listeners to the same worker. I have not had the need for this yet.

worker.addEventListener('message', function(event) {
  console.log('One', event.data);
});
worker.addEventListener('message', function(event) {
  console.log('Two', event.data);
});

Reloading the page with the above code, will result in two lines in the console log. Notice that I am only logging the data part of the event.

One, I am eager for work!
Two, I am eager for work!

Our eager-worker.js is really eager to work so he keeps on telling his boss that he want to work every second.

// eager-worker.js
setInterval(function() {
  postMessage('I am eager for work!');
}, 1000);

This of course annoys the boss tremendously so he decides to tell the worker to do something by sending him a message with postMessage.

// main.js
var worker = new Worker("eager-worker.js");
worker.onmessage = function(event) {
  console.log(event.data);
};
worker.postMessage('Stop bugging me and do something!');

Our eager-worker.js is not listening yet, so the boss can scream all he wants without any success. Let's change that by implementing the onmessage method in the worker as well. addEventListener also works.

// eager-worker.js
postMessage('I am eager for work!');

var timer = setInterval(function() {
  postMessage('I am eager for work!');
}, 1000);

onmessage = function(event) {
  clearInterval(timer);
  postMessage('Alright Boss!');
};

Now the output is less annoying.

I am eager for work!
Alright Boss!

Now that we know the basics of web workers, lets look of some other interesting issues that come up.

Debugging Web Workers

If you try to use console.log in your web workers you will get an error messages such as this:

Uncaught ReferenceError: console is not defined

So this is an issue with web workers, it is not possible to use console or alert to debug them.

It is not a big problem because with Chrome it is possible to debug workers. In the lower right corner of the source tab of the Chrome Developer Tools, there is a Workers panel.

Checking the checkbox Pause on start will open up a new inspector window which allow us to debug the worker just as if it was a normally loaded script. Nice!

Errors

If there are script errors in the web worker, it will send back an error event instead of a message event. The errors can be handled via the onerror property of by subscribing to the error event.

worker.onerror(function(event) {
  console.log(event);
});
// or
worker.addEventListener('error', function(event) {
  console.log(event);
});

The above code will result in an event that looks like this, showing you the filename and line number to handle the message.

ErrorEvent {lineno: 6, filename: "http://localhost/web-workers/eager-worker.js", message: "Uncaught ReferenceError: missing is not defined", clipboardData: undefined, cancelBubble: false…}

Web Worker Script Loading

A web worker can load additional scrips with importScripts(URL, ...). The URLs can be relative and, if so, are relative to the file doing the importing.

importScripts('../data/swedish-word-list.js', 'word-maestro.js', 'messageHandler.js')

A larger example

In this example I will show how easy it is to create a delegating worker that allows me to call normal methods on an object.

The messages are sent using a simple protocol with an object containing two properties.

// The message object
var message = {
  method: 'The method I wish to call',
  args:   ['An array of arguments']
}

main.js starts the delegating-worker.js and sends messages to it.

// main.js
var worker = new Worker("delegating-worker.js");
worker.onmessage = function(event) {
  console.log(event.data);
};

setInterval(function() {
  // Call the method echo with the argument ['Work']
  worker.postMessage({method: 'echo', args: ['Work']});
}, 4200);

setInterval(function() {
  // Call the method ohce with the argument ['Work']
  worker.postMessage({method: 'ohce', args: ['Work']});
}, 1100);

The delegating-worker.js loads the external script echo.js, which declares the variable Echo in the global worker scope. In the onmessage method I unpack the event and delegate the method call to Echo via apply. I use apply since I want to allow a variable number of arguments. The reply is sent back to main.js along with the method that was called.

// Declares Echo
importScripts('echo.js');

onmessage = function(event) {
  var method = event.data.method;
  var args = event.data.args;

  // I use apply since to allow a variable number of arguments
  var reply = Echo[method].apply(Echo, args);
  self.postMessage({method: method, reply: reply});
};

The Echo service is a simple object with two methods.

var Echo = {
  // Return the word recieved.
  echo: function(word) {
    return word;
  },
  // Reverse the word and return it.
  ohce: function(word) {
    return word.split('').reverse().join('');
  }
};

Structuring the code in this way makes it easy to reuse the functionality in a non worker context.

Limitations of Web Workers

Since web workers are working in the background, they do not have access to the DOM, window, document or even the console. Any communication with these objects will have to be done by sending messages back to the main script

Checking for Web Worker support

Checking for Web Worker support is easy, just check if window.Worker is defined and show an error page or use an alternative solution if it is not.

function workersSupported() {
  return window.Worker;
}

if (!workersSupported()) {
  window.location = './unsupported-browser.html';
}

Wrap up

Using workers is easy, if you want to see a more thorough example, check out the source code (in Coffeescript) for Word-Maestro.

jQuery Changes From 1.4.2 to 1.6

jQuery is a powerful library and it is possible to get by without using any of the new features. That’s why many of us just upgrade to a new version assuming that it is mostly bug and performance fixes. This is not the case. jQuery 1.4.2 was released in February 2010 and it’s been one and a half years and a number of releases since then.

I was going to write about the changes in 1.5 and 1.6, but I have noticed that many people have missed some of the new features of the 1.4 releases. And by the way, all examples are written in Coffeescript.

Selected changes from 1.4.2+

delegate()

Most people know about live() and how it can be used to attach listeners to elements that don’t yet exist in the DOM. live() has a younger brother that was born in 1.4.2 and he is called delegate(). delegate() is more powerful. It gives you more precision in where to attach the listener.

$('.main-content').find('section').delegate 'p', 'click', -> 
  $(this).addClass 'highlight' 

As you can see above, delegate(), unlike live(), can be chained like normal jQuery calls.

jQuery.now(), jQuery.type() and jQuery.parseXML()

Nothing special here, just some utilities that are good to know about.

$.now() is (new Date()).getTime() 
 
$.type('hello') is 'string' 
 
xml = """ 
<rss version='2.0'> 
<channel> 
<title>RSS Title</title> 
</channel> 
</rss> 
""" 
xmlDoc = $.parseXML xml 
($(xmlDoc).find 'title').text() is 'RSS Title' 

All these utilities are interesting, but the truly good thing that came out of jQuery-1.5+ is Deferred().

Deferred()

With the release of jQuery 1.5 the internal implementation of $.ajax was changed to use Deferred(), and, even better, the implementation was deemed so useful, that it became part of the public API.

Here is how you can use it via the $.ajax method.

Declare a function hex that calls the /hex url on the server, which will return a hex value between 00 and FF.

  hex = -> 
    $.ajax { url: '/hex' } 

Call the function multiple times, and you get a new Deferred back for each call. Notice the lack of handlers for the calls.

  red = hex() 
  green = hex() 
  blue = hex() 

Attach handlers to each of the Deferreds to do something useful with the returned value. Notice the use of done instead of success. success is deprecated and will be removed in 1.8.

  red.done (hh)-> 
    $("#r").css 'background-color', "##{hh}0000" 
  green.done (hh)-> 
    $("#g").css 'background-color', "#00#{hh}00" 
  blue.done (hh)-> 
    $("#b").css 'background-color', "#0000#{hh}" 

I am not limited to adding one handler, I can attach as many as I like. Here I attach another one for logging the success of the red-call.

  red.done (hh) -> 
    console.log(hh) 

If this was all there was to it, I would be happy, but it is not by using the $.when method I get a new Deferred object that orchestrates multiple Deferreds in a very simple way. Let me create a color-Deferred that waits for the others to return and then calls a new callback when they are all done.

  color = $.when(red, green, blue) 
 
  color.done (r, g, b) -> 
    $("#color").css 'background-color', "##{r[0]}#{g[0]}#{b[0]}" 

The results from the requests are given in the same order as the Deferred objects are passed in. The results are given as an array with three arguments [ data, ‘success’ , deferred ].

You can see the example, a simple Sinatra app, in action on Heroku and the source code can be found on Github

Very nice! Apart from the examples I have shown, there are methods working with Deferred. Methods for creating, jQuery.Deferred(), resolving, resolve(), resolveWith(), and rejecting, reject(), rejectWith().

To attach handlers to the events, you can use done() for resolve, fail for reject, always() for both resolve and reject. You can also use then(doneCallback, failCallback) to attach both a done and a fail handler with one call.

Deferreds are really cool, check them out here

A Not Very Short Introduction To Node.js

Node.js is a set of asynchronous libraries, built on top of the Google V8 Javascript Engine. Node is used for server side development in Javascript. Do you feel the rush of the 90's coming through your head. It is not the revival of LiveWire, Node is a different beast. Node is a single threaded process, focused on doing networking right. Right, in this case, means without blocking I/O. All the libraries built for Node use non-blocking I/O. This is a really cool feature, which allows the single thread in Node to serve thousands of request per second. It even lets you run multiple servers in the same thread. Check out the performance characteristics of Nginx and Apache that utilize the same technique.

The graph for memory usage is even better.

Read more about it at the Web Faction Blog

OK, so what's the catch? The catch is that all code that does I/O, or anything slow at all, has to be called in an asynchronous style.

// Synchronous 
var result = db.query("select * from T"); 
// Use result 
 
// Asynchronous 
db.query("select * from T", function (result) { 
    // Use result 
}); 

So, all libraries that deal with IO has to be re-implemented with this style of programming. The good news is that even though Node has only been around for a couple of years, there are more than 1800 libraries available. The libraries are of varying quality but the popularity of Node shows good promise to deliver high-quality libraries for anything that you can imagine.

History

Node is definitely not the first of its kind. The non-blocking select() loop, that is at the heart of Node, dates back to 1983.

Twisted appeared in Python (2002) and EventMachine in Ruby (2003).

This year a couple of newcomers appeared.

Goliath, which builds on EventMachine, and uses fibers to allow us to program in an synchronous style even though it is asynchronous under the hood.

And, the Async Framework in .Net, which enhances the compiler with the keywords async and await to allow for very elegant asynchronous programming.

Get Started

This example uses OSX as an example platform, if you use something else you will have to google for instructions.

# Install Node using Homebrew 
$ brew install node
==> Downloading http://nodejs.org/dist/node-v0.4.7.tar.gz
######################################################################## 100.0% 
==> ./configure --prefix=/usr/local/Cellar/node/0.4.7 
==> make install
==> Caveats
Please add /usr/local/lib/node to your NODE_PATH environment variable to have node libraries picked up. 
==> Summary
/usr/local/Cellar/node/0.4.7: 72 files, 7.5M, built in 1.2 minutes

When installed you have access to the node command-line command. When invoked without arguments, it start a REPL.

$ node
> function hello(name) {
... return 'hello ' + name; 
... }
> hello('tapir') 
'hello tapir' 
> 

When invoked with a script it runs the script.

// hello.js 
setTimeout(function() { 
    console.log('Tapir'); 
}, 2000); 
console.log('Hello'); 
 

$ node hello.js
Hello
... 
Tapir

Networking

As I mentioned above, Node is focused on networking. That means it should be easy to write networking code. Here is a simple echo server.

// Echo Server 
var net = require('net'); 
 
var server = net.createServer(function(socket) { 
    socket.on('data', function(data) { 
        console.log(data.toString()); 
        socket.write(data); 
    }); 
}); 
server.listen(4000); 
 

And here is a simple HTTP server.

// HTTP Server 
var http = require('http'); 
var web = http.createServer(function(request, response) { 
  response.writeHead(200, { 
    'Content-Type': 'text/plain' 
  }); 
  response.end('Tapirs are beautiful!\n'); 
}); 
web.listen(4001); 
 

Quite similar. A cool thing is that the servers can be started from the same file and node will, happily, serve both HTTP and echo requests from the same thread without any problems. Let's try them out!

# curl the http service 
$ curl localhost:4001 
Tapirs are beautiful! 
 
# use netcat to send the string to the echo server 
$ echo 'Hello beautiful tapir' | nc localhost 4000 
Hello beautiful tapir
 

Modules

Node comes with a selection of built in modules. Ryan Dahl says that they try to keep the core small, but even so the built-in modules cover a lot of useful functionality.

• net - contains tcp/ip related networking functionality.
• http - contains functionality for dealing with the HTTP protocol.
• util - holds common utility functions, such as log, inherits, pump, ...
• fs - contains filesystem related functionality, remember that everything should be asynchronous.
• events - contains the EventEmitter that is used for dealing with events in a consistent way. It is used internally but it can be used externally too.

An example

Here is an example of a simple module.

// module tapir.js 
 
// require another module 
var util = require('util'); 
 
function eat(food) { 
  util.log('eating '+ food); 
} 
 
// export a function 
exports.eat = eat; 
 

As you can see it looks like a normal Javascript file and it even looks like it has global variables. It doesn't. When a module is loaded it is wrapped in code, similar to this.

var module = { exports: {}}; 
(function(module, exports){ 
  // module code from file 
  ... 
})(module, module.exports); 
 

As you can see the code is wrapped in a function and an empty object with an export property is sent into it. This is used by the file to export only the functions that it want to publish.

The require function works in symphony with the module and it returns the exported functions to the caller.

Node Package Manager, npm

To allow simple handling of third-party packages, Node uses npm. It can be installed like this:

$ curl http://npmjs.org/install.sh | sh
... 
 

And used like this:

$ npm install -g express
mime@1.2.1 /usr/local/lib/node_modules/express/node_modules/mime
connect@1.4.0 /usr/local/lib/node_modules/express/node_modules/connect
qs@0.1.0 /usr/local/lib/node_modules/express/node_modules/qs
/usr/local/bin/express -> /usr/local/lib/node_modules/express/bin/express
express@2.3.2 /usr/local/lib/node_modules/express
 

As you can see, installing a module also installs its dependencies. This works because a module can be package with meta-data, like so:

// express/package.json 
{ 
  "name": "express", 
  "description": "Sinatra inspired web development framework", 
  "version": "2.3.2", 
  "author": "TJ Holowaychuk <tj@vision-media.ca>", 
  "contributors": [ 
    { "name": "TJ Holowaychuk", "email": "tj@vision-media.ca" }, 
    { "name": "Guillermo Rauch", "email": "rauchg@gmail.com" } 
  ], 
  "dependencies": { 
    "connect": ">= 1.4.0 < 2.0.0", 
    "mime": ">= 0.0.1", 
    "qs": ">= 0.0.6" 
  }, 
  "keywords": ["framework", "sinatra", "web", "rest", "restful"], 
  "repository": "git://github.com/visionmedia/express", 
  "main": "index", 
  "bin": { "express": "./bin/express" }, 
  "engines": { "node": ">= 0.4.1 < 0.5.0" } 
} 
 

The package.json contains information about who made the module, its dependencies, along with some additional information to enable better searching facilities.

Npm installs the modules from a common respository, which contains more than 1800 modules.

Noteworthy Modules

Express is probably the most used of all third-party modules. It is a Sinatra clone and it is very good, just like Sinatra.

// Create a server 
var app = express.createServer(); 
app.listen(4000); 
 
// Mount the root (/) and redirect to index 
app.get('/', function(req, res) { 
  res.redirect('/index.html'); 
}); 
 
// Handle a post to /quiz 
app.post('/quiz', function(req, res) { 
  res.send(quiz.create().id.toString()); 
}); 

Express uses Connect to handle middleware. Middleware is like Rack but for Node (No wonder that Node is nice to work with when it borrows its ideas from Ruby :)

connect( 
      // Add a logger 
      connect.logger() 
      // Serve static file from the current directory 
    , connect.static(__dirname) 
      // Compile Sass and Coffescript files, on the fly 
    , connect.compiler({enable: ['sass', 'coffeescript']}) 
      // Profile all requests 
    , connect.profiler() 
  ).listen(3000); 

Another popular library is Socket.IO. It handles the usual socket variants, such as WebSocket, Comet, Flash Sockets, etc...

var http = require('http'); 
var io = require('socket.io'); 
 
server = http.createServer(function(req, res){...}); 
server.listen(80); 
 
// socket.io attaches to an existing server 
var socket = io.listen(server); 
socket.on('connection', function(client){ 
  // new client is here! 
  client.on('message', function(){ ... }) 
  client.on('disconnect', function(){ ... }) 
}); 

MySql has a library for Node.

client.query( 
  'SELECT * FROM ' + TEST_TABLE, 
  // Note the callback style 
  function(err, results, fields) { 
    if (err) { throw err; } 
 
    console.log(results); 
    console.log(fields); 
    client.end(); 
  } 
); 

And Mongoose can be used for accessing MongoDB.

// Declare the schema 
var Schema = mongoose.Schema
  , ObjectId = Schema.ObjectId; 
 
var BlogPost = new Schema({ 
    author    : ObjectId
  , title     : String
  , body      : String
  , date      : Date
}); 
 
// Use it 
var BlogPost = mongoose.model('BlogPost'); 
 
// Save 
var post = new BlogPost(); 
post.author = 'Stravinsky'; 
instance.save(function (err) { 
  // 
}); 
 
// Find 
BlogPost.find({}, function (err, docs) { 
  // docs.forEach 
}); 

Templating Engines

Everytime a new platform makes its presence, it brings along a couple of new templating languages and Node is no different. Along with the popular ones from the Ruby world, like Haml and Erb (EJS in Node), comes some new ones like Jade and some browser templating languages like Mustache and jQuery templates. I'll show examples of Jade and Mu (Mustache for Node).

I like Jade, because it is a Javascript dialect of Haml and it seems appropriate to use if I'm using Javascript on the server side.

!!! 5 
html(lang="en") 
  head
    title= pageTitle
    script(type='text/javascript') 
      if (foo) { 
        bar() 
      } 
  body
    h1 Jade - node template engine
    #container 
      - if (youAreUsingJade) 
        p You are amazing
      - else 
        p Get on it! 

I'm not really sure if I like Mustache or not, but I can surely see the value of having a templating language which works both on the server side and in the browser.

<h1>{{header}}</h1> 
{{#bug}}
{{/bug}}
 
{{#items}}
  {{#first}}
    <li><strong>{{name}}</strong></li> 
  {{/first}}
  {{#link}}
    <li><a href="{{url}}">{{name}}</a></li> 
  {{/link}}
{{/items}}
 
{{#empty}}
  <p>The list is empty.</p> 
{{/empty}}

Testing

Node comes with assertions built in, and all testing frameworks build on the Assert module, so it is good to know.

assert.ok(value, [message]); 
assert.equal(actual, expected, [message]) 
assert.notEqual(actual, expected, [message]) 
assert.deepEqual(actual, expected, [message]) 
assert.strictEqual(actual, expected, [message]) 
assert.throws(block, [error], [message]) 
assert.doesNotThrow(block, [error], [message]) 
assert.ifError(value) 
assert.fail(actual, expected, message, operator) 
 
// Example 
// assert.throws(function, regexp) 
assert.throws( 
  function() { throw new Error("Wrong value"); }, 
  /value/ 
); 

Apart from that there are at least 30 different testing frameworks to use. I have chosen to use NodeUnit since I find that it handles asynchronous testing well, and it has a nice UTF-8 output that looks good in the terminal,

// ./test/test-doubled.js 
var doubled = require('../lib/doubled'); 
 
// Exported functions are run by the test runner 
exports['calculate'] = function (test) { 
    test.equal(doubled.calculate(2), 4); 
    test.done(); 
}; 
 
// An asynchronous test 
exports['read a number'] = function (test) { 
    test.expect(1); // Make sure the assertion is run 
 
    var ev = new events.EventEmitter(); 
    process.openStdin = function () { return ev; }; 
    process.exit = test.done; 
 
    console.log = function (str) { 
        test.equal(str, 'Doubled: 24'); 
    }; 
 
    doubled.read(); 
    ev.emit('data', '12'); 
}; 
 

Deployment

There are already a lot of platforms providing Node as a service (PaaS , Platform as a Service). Most of them are using Heroku style deployment by pushing to a Git remote. I'll show three alternatives that all provide free Node hosting.

Joyent (no.de)

Joyent, the employers of Ryan Dahl, give you ssh access so that you can install the modules you need. Deployment is done by pushing to a Git remote.

$ ssh node@my-machine.no.de
$ nmp install express
$ git remote add node node@andersjanmyr.no.de:repo
$ git push node master
Counting objects: 5, done. 
Delta compression using up to 2 threads. 
Compressing objects: 100% (3/3), done. 
Writing objects: 100% (3/3), 321 bytes, done. 
Total 3 (delta 2), reused 0 (delta 0) 
remote: Starting node v0.4.7... 
remote: Successful
To node@andersjanmyr.no.de:repo
  8f59169..c1177b0  master -> master

Nodester

Nodester, gives you a command line tool, nodester, that you use to install modules. Deployment by pushing to a Git remote.

$ nodester npm install express
$ git push nodester master
Counting objects: 5, done. 
Delta compression using up to 2 threads. 
Compressing objects: 100% (3/3), done. 
Writing objects: 100% (3/3), 341 bytes, done. 
Total 3 (delta 2), reused 0 (delta 0) 
remote: Syncing repo with chroot
remote: From /node/hosted_apps/andersjanmyr/1346-7856c14e6a5d92a6b5374ec4772a6da0.git/. 
remote:    38f4e6e..8f59169  master     -> origin/master
remote: Updating 38f4e6e..8f59169
remote: Fast-forward
remote:  Gemfile.lock |   10 ++++------
remote:  1 files changed, 4 insertions(+), 6 deletions(-) 
remote: Checking ./.git/hooks/post-receive
remote: Attempting to restart your app: 1346-7856c14e6a5d92a6b5374ec4772a6da0
remote: App restarted.. 
remote: 
remote: 
remote:     \m/ Nodester out \m/ 
remote: 
remote: 
To ec2-user@nodester.com:/node/hosted_apps/andersjanmyr/1346-7856c14e6a5d92a6b5374ec4772a6da0.git
   38f4e6e..8f59169  master -> master

Cloud Foundry

Cloud Foundry is one of the most interesting platforms in the cloud. It was genius by VM Ware to open source the platform, allowing anyone to set up their own cloud if they wish. If you don't want to setup your own Cloud Foundry Cloud, you can use the service hosted at cloundfoundry.com.

With Cloud Foundry, you install the modules locally and then they are automatically deployed as part of the vmc push. Push in this case does not mean git push, but instead, copy all the files from my local machine to the server.

$ npm install express  # Install locally 
mime@1.2.1 ./node_modules/express/node_modules/mime
connect@1.4.0 ./node_modules/express/node_modules/connect
qs@0.1.0 ./node_modules/express/node_modules/qs
express@2.3.0 ./node_modules/express
 
$ vmc push
Would you like to deploy from the current directory? [Yn]: Y
Application Name: snake
Application Deployed URL: 'snake.cloudfoundry.com'? 
Detected a Node.js Application, is this correct? [Yn]: 
Memory Reservation [Default:64M] (64M, 128M, 256M, 512M, 1G or 2G) 
Creating Application: OK
Would you like to bind any services to 'snake'? [yN]: 
Uploading Application: 
  Checking for available resources: OK
  Packing application: OK
  Uploading (1K): OK
Push Status: OK
Staging Application: OK
Starting Application: ........OK

Tools

There are of course a bunch of tools that come with a new platform, Jake, is a Javascript version of Rake, but I am happy with Rake and I don't see the need to switch. But, there are some tools that I cannot live without when using Node.

Reloaders

If you use the vanilla node command then you have to restart it every time you make a change to a file. That is awfully annoying and there are already a number of solutions to the problem.

# Nodemon watches the files in your directory and reloads them if necessary 
$ npm install nodemon
nodemon@0.3.2 ../node_modules/nodemon
 
$ nodemon server.js 
30 Apr 08:21:23 - [nodemon] running server.js 
... 
# Saving the file 
30 Apr 08:22:01 - [nodemon] restarting due to changes... 
 
# Alternative 
$ npm install supervisor
$ supervisor server.js 
DEBUG: Watching directory '/evented-programming-with-nodejs/.  

Debuggers

Another tool that it is hard to live without is a debugger. Node comes with one built in. It has a gdb flavor to it and it is kind of rough.

$ node debug server.js
debug> run
debugger listening on port 5858 
connecting...ok
break in #<Socket> ./server.js:9 
    debugger; 
 
debug> p data.toString(); 
tapir
 

// Javascript 
var echo = net.createServer(function(socket) { 
  socket.on('data', function(data) { 
      debugger; // <= break into debugger 
      socket.write(data); 
  }); 
}); 

If you want a GUI debugger, it is possible to use the one that comes with Chrome by installing the node-inspector. It is started similarly to the built in debugger, but the --debug is an option instead of a subcommand.

$ node-inspector & 
visit http://0.0.0.0:8080/debug?port=5858 to start debugging
 
$ node --debug server.js debugger listening on port 5858 

After that you can just fire up Chrome on the URL, http://0.0.0.0:8080/debug?port=5858 and you can debug the node process just as if it was running in the browser.

Idioms

Idioms, patterns, techniques, call it what you like. Javascript code is littered with callbacks, and event more so with Node. Here are some tips on how to write good asynchronous code with Node.

Return on Callbacks

It is easy to forget to escape from the function after a callback has been called. An easy way to remedy this problem is to call return before every call to a callback. Even though the value is never used by the caller, it is an easy pattern to recognize and it prevents bugs.

 
function doSomething(response, callback) { 
  doAsyncCall('tapir', function(err, result) { 
    if (err) { 
      // return on the callback 
      return callback(err); 
    } 
    // return on the callback 
    return callback(null, result); 
  }); 
} 

Exceptions in Callbacks

Exceptions that occur in callbacks cannot be handled the way we are used to, since the context is different. The solution to this is to pass along the exception as a parameter to the callback. In Node the convetion is to pass the error as the first parameter into the callback.

function insertIntoTable(row, function(err, data) { 
  if (err) return callback(err); 
  ... 
 
  // Everything is OK 
  return callback(null, 'row inserted'); 
} 

Parallel Execution

If you have multiple tasks that need to be finished before you take some new action, this can be handled with a simple counter. Here is an example of a simple function that starts up a bunch of functions in parallel and waits for all of them to finish before calling the callback.

// Do all in parallel 
function doAll(collection, callback) { 
  var left = collection.length; 
  collection.forEach(function(fun) { 
    fun(function() { 
      if (--left == 0) callback(); 
    }); 
  }); 
}; 
 
// Use it 
var result = []; 
doAll([ 
  function(callback) { 
    setTimeout(function() {result.push(1); callback();}, 2000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(2); callback();}, 3000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(3); callback();}, 1000 )} 
  ], function() { return result; } 
 
// returns [3, 1, 2] 

Sequential Execution

Sometimes the ordering is important. Here is a simple function that makes sure that the calls are executed in sequence. It uses recursion to to make sure that the calls are handled in the correct order. It also uses the Node function process.nextTick() to prevent the stack from getting to large for large collections. Similar results can be obtained with setTimeout() in browser Javascript. It can be seen as a simple trick to achieve tail recursion.

function doInSequence(collection, callback) { 
    var queue = collection.slice(0); // Duplicate 
 
    function iterate() { 
      if (queue.length === 0) return callback(); 
      // Take the first element 
      var fun = queue.splice(0, 1)[0]; 
      fun(function(err) { 
        if (err) throw err; 
        // Call it without building up the stack 
        process.nextTick(iterate); 
      }); 
    } 
    iterate(); 
} 
 
 
var result = []; 
doInSequence([ 
  function(callback) { 
    setTimeout(function() {result.push(1); callback();}, 2000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(2); callback();}, 3000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(3); callback();}, 1000 )} 
  ], function() { return result; }); 
 
// Returns [1, 2, 3] 

Library Support for Asynchronous Programming

If you don't want to write these functions yourself, there are a few libraries that can help you out. I'll show two version that I like.

Fibers

Fibers are also called co-routines. Fibers provide two functions, suspend and resume, which allows us to write code in a synchronous looking style. In the Node version of fibers, node-fibers, suspend and resume are called yield() and run() instead.

require('fibers'); 
var print = require('util').print; 
 
function sleep(ms) { 
    var fiber = Fiber.current; 
    setTimeout(function() { fiber.run(); }, ms); 
    yield(); 
} 
 
Fiber(function() { 
    print('wait... ' + new Date + '\n'); 
    sleep(1000); 
    print('ok... ' + new Date + '\n'); 
}).run(); 
print('back in main\n'); 

Fibers are a very nice way of writing asynchronous code but, in Node, they have one drawback. They are not supported without patching the V8 virtual machine. The patching is done when you install node-fibers and you have to run the command node-fibers instead of node to use it.

The async Library

If you don't want to use the patched version of V8, I can recommend the async library. Async provides around 20 functions that include the usual 'functional' suspects (map, reduce, filter, forEach...) as well as some common patterns for asynchronous flow control (parallel, series, waterfall...). All these functions assume you follow the Node convention of providing a single callback as the last argument of your async function.

async.map(['file1','file2','file3'], fs.stat, function(err, results){ 
    // results is now an array of stats for each file 
}); 
 
async.filter(['file1','file2','file3'], path.exists, function(results){ 
    // results now equals an array of the existing files 
}); 
 
async.parallel([ 
    function(){ ... }, 
    function(){ ... } 
], callback); 
 
async.series([ 
    function(){ ... }, 
    function(){ ... } 
], callback); 

Conclusion

Node is definitely an interesting platform. The possibility to have Javascript running through the whole stack, from the browser all the way down into the database (if you use something like CouchDB or MongoDB) really appeals to me. The easy way to deploy code to multiple, different cloud providers is also a good argument for Node.