The Tapir's Tale

Finding with Git

2012-01-25T17:18:00.000+01:00

Git is an amazing version control system that never loses anything, but sometimes it can be hard to find out where things are. Most of the time it is going to be git log that is our friend, but not all the time.

Where is my file?

Sometimes you know that you have a file in your repository but, you don't know exactly where it is. git ls-files is the answer.

# Find all files with the name security in the path.
$ git ls-files | grep security
lib/dynamo-db/security.js
test/security-test.js

Obviously, you can use any particular grep options you prefer, like -i to ignore case.

In what files does word exist?

If you want to find information inside files git grep is your friend. git grep works similar to a recursive grep grep -r . but it only searches files that are managed by git.

# Find all lines matching *crypt* in current version.
$ git grep crypt
lib/dynamo-db/security.js:var crypto = require('crypto');
lib/dynamo-db/security.js:   var hmac = crypto.createHmac('sha256', key);

# Also give me the line numbers
 git grep -n crypt
lib/dynamo-db/security.js:2:var crypto = require('crypto');
lib/dynamo-db/security.js:15:   var hmac = crypto.createHmac('sha256', key);

# List only the file names
 git grep -l crypt
lib/dynamo-db/security.js

# Also list how many times (count) it matched.
$ git grep -c crypt
lib/dynamo-db/security.js:2

It is also possible to give versions to grep to find out what has changed between revisions.

# Find all files with lines matching *type* in revisions `master` and `8f0fb7f`.
git grep -l type  master 8f0fb7f
master:lib/dynamo-db/index.js
master:lib/dynamo-db/security.js
master:package.json
8f0fb7f:lib/dynamo-db/index.js
8f0fb7f:package.json

Maybe this is not that impressive. Most of the above can be accomplished with standard grep, find, ack, and friends. But Git is a version control system. How do I find out about things that happened in the past?

Who deleted my file?

You know how it is, you are working in some project and, your dog gets sick and you have to stay home from work, when you come back someone has deleted your file! Where is it and who did it? git log to the rescue.

git log shows you the commit logs. It is your eye into the past.

# When (in what commit) was my file deleted?
$ git log --diff-filter=D -- test/create-table-test.js
commit ba6c4d8bc165b8fb8208979c3e5513bd53477d51
Author: Anders Janmyr <anders@janmyr.com>
Date:   Wed Jan 25 09:46:52 2012 +0100

    Removed the stupid failing test.

Looks like I found the culprit, was I working from home? But is the file really deleted here. To get some more information about files add the --summary option.

# When (in what commit) was my file deleted?
$ git log --diff-filter=D --summary -- test/create-table-test.js
commit ba6c4d8bc165b8fb8208979c3e5513bd53477d51
Author: Anders Janmyr <anders@janmyr.com>
Date:   Wed Jan 25 09:46:52 2012 +0100

    Removed the stupid failing test.

 delete mode 100644 test/create-table-test.js

Yes, it looks like the file is really deleted. Stupid bastard! Let me break this command down starting with the last command.

test/create-table-test.js - The filename has to be the relative path to the file (from your current directory).
-- - The double-dash is used to tell Git that this is not a branch or an option.
--summary - Show me what files were deleted or added. --name-status is similar.
--diff-filter - This is a real beauty, it allows me to limit the log to show me only the specified kind of change, in this case D, for Deleted. Other options are: Added (A), Copied (C), Modified (M) and Renamed (R)

When was a file added?

This uses the same technique as above, but I will vary it since I don't want to type the full path of the file.

# Find out when the integration tests where added.
$ git log --diff-filter=A --name-status |grep -C 6 integ
commit 09420cfea8c7b569cd47f690104750fec358a10a
Author: Anders Janmyr <anders@janmyr.com>
Date:   Tue Jan 24 16:23:52 2012 +0100

    Extracted integration test

A integration-test/sts-test.js

commit 205db3965dec6c2c4c7b2bb75387a591d49e1951
Author: Anders Janmyr <anders@janmyr.com>
Date:   Sat Jan 21 10:03:59 2012 +0100

As you can see here I am using --name-status as a variation on --summary, it uses the same notation as the --diff-filter.

I am using grep -C 6 to get some context around the found element, in this case six lines before and after the match. Very useful!

Who changed that line?

As you probably know it is to see who has done something in a file by using git blame

$ git blame test/security-test.js
205db396 (Anders Janmyr 2012-01-21 10:03:59 +0100  1) var vows = require('vows'),
205db396 (Anders Janmyr 2012-01-21 10:03:59 +0100  2)     assert = require('assert');
205db396 (Anders Janmyr 2012-01-21 10:03:59 +0100  3) 
09420cfe (Anders Janmyr 2012-01-24 16:23:52 +0100  4) var access = 'access';
09420cfe (Anders Janmyr 2012-01-24 16:23:52 +0100  5) var secret = 'secret';
90b65208 (Anders Janmyr 2012-01-21 11:58:21 +0100  6) 
205db396 (Anders Janmyr 2012-01-21 10:03:59 +0100  7) var security = new Security({
90b65208 (Anders Janmyr 2012-01-21 11:58:21 +0100  8)   access: access,
90b65208 (Anders Janmyr 2012-01-21 11:58:21 +0100  9)   secret: secret
205db396 (Anders Janmyr 2012-01-21 10:03:59 +0100 10) });
...

Every line gets annotated with the commit who introduced it and by whom. Very helpful.

Who deleted that line?

Another feature, which is not as well known, is git blame --reverse. It allows you to see the file as it was before, annotated to show you where it has been changed.

# Check the what lines have been changed since the last 6 commits.
$ git blame --reverse head~6..head security-test.js
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100  1) var vows = require('vows'),
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100  2)     assert = require('assert');
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100  3) 
093c13e9 (Anders Janmyr 2012-01-24 17:26:09 +0100  4) var Security = require('dynamo-db').Security;
ba6c4d8b (Anders Janmyr 2012-01-25 09:46:52 +0100  5) 
^b96c68b (Anders Janmyr 2012-01-21 12:33:50 +0100  6) var access = process.env['S3_KEY'];
^b96c68b (Anders Janmyr 2012-01-21 12:33:50 +0100  7) var secret = process.env['S3_SECRET'];
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100  8) 
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100  9) var security = new Security({
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100 10)   access: access,
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100 11)   secret: secret
558b8e7f (Anders Janmyr 2012-01-25 16:53:52 +0100 12) });

In the output you can see that most of the lines are still the same at HEAD (558b8e7f). But the fourth and fifth line 093c13e9 and ba6c4d8b don't exist anymore. And the sixth and seventh lines ^b96c68b have been changed after this commit.

What commits contain the string?

Another thing I find very useful is to find out when certain words or sentences are removed or added. For this you can use git log -S<string> or git log -G<regex>

# Find commits that modified the string aws and display the full diff.
$ git log -Saws --diff-filter=M --patch
commit b96c68b839f204b310b79570bc3d27dc93cff588
Author: Anders Janmyr <anders@janmyr.com>
Date:   Sat Jan 21 12:33:50 2012 +0100

    We have a valid request, tjohoo

diff --git a/lib/dynamo-db/security.js b/lib/dynamo-db/security.js
index bee6936..8471527 100644
--- a/lib/dynamo-db/security.js
+++ b/lib/dynamo-db/security.js
@@ -2,6 +2,7 @@
 var crypto = require('crypto');
 var _ = require('underscore');
 var request = require("request");
+var xml2js = require('xml2js');
 
 function Security(options) {
   this.options = options;
@@ -23,7 +24,7 @@ mod.timestamp = function() {
 mod.defaultParams = function() {
   return {
     AWSAccessKeyId: this.options.access,
-    Version: '2010-05-08',
+    Version: '2011-06-15',
     Timestamp: this.timestamp(),
     SignatureVersion: 2,
     SignatureMethod: 'HmacSHA256'
@@ -57,9 +58,10 @@ mod.url = function(host, path, params) {
 
 mod.makeRequest = function(method, host, path, params, callback) {
   var extParams = _.extend({}, this.defaultParams(), params);
-  var signedParams = this.signedParams('GET', 'iam.amazonaws.com', '/', extParams);
-  console.log(extParams,signedParams);
-  return request({ method: method, url: this.url(host, path, signedParams) },
+  var signedParams = this.signedParams(method, host, path, extParams);
+  var url = this.url(host, path, signedParams);
+  console.log(url,signedParams);
+  return request({ method: method, url: url },
...

That's all folks!

7 Reasons to go to Øredev 2011

2011-10-04T17:37:00.002+02:00

I’m am looking forward to Øredev 2011 more than I have looked forward to any of the previous ones. The reason for this is that Øredev has finally become a leading conference for dynamic programming.

Øredev has always been good in the enterprise sphere led by Java, .Net and mobile tracks, but it has been weaker in the area of dynamic programming languages. Last year was better than before, but this year is going to be great. Here are some speakers that you should not miss.

Yehuda Katz

Yehuda Katz was one of the driving forces behind the great Rails 3 refactoring, that made sure that Rails will remain the most productive web development environment for many years to come. Yehuda just released a new book, Rails 3 in Action, Its the first book about Rails 3.1, including the awesome Asset Pipeline, streaming, and reversible migrations.

Yehuda has also been involved with jQuery and written jQuery in Action.

He will be speaking about Rails and Sproutcore.

Felix Geisendörfer

Felix Geisendörfer is a core Node.js developer and he will of course be talking about Node. Node.js is a set of Javascript libraries that runs on top of the Google V8 virtual machine. What is interesting about Node, apart from being server-side Javascript is that it uses asynchronous programming as the default. This default makes Node extremely interesting for developing solutions involving multiple open connections, such as websockets, and for streaming video and audio. Node is definitely part of the future of the web. I have written extensively about it in the past.

Corey Haines

Cory Haines is a legend in the TDD community. He is also famous for his Code Retreats. He will give one workshop, Improving your TDD, and two presentations, Fast Ruby on Rails Tests and Come introduce yourself to the concepts and fundamental technique behind TDD

Ilya Grigorik

Ilya Grigorik is the founder of PostRank, that was recently acquired by Google. He is now working on Social Analytics at Google. At PostRank he used Ruby to perform analysis on very large amounts of data. While doing this he developed Goliath, a high-performance non-blocking web server using Ruby 1.9 and fibers. He will be talking about Goliath, Concurrency and Machine Learning

I can recommend that you follow Ilya on Twitter since his tweets has the highest signal-to-noise ratio I know of.

And, finally, make sure to check out Vim Golf, a really cool way to become a Vim wizard.

Trevor Burnham

Coffeescript is the new way to write Javascript without actually writing it :). Coffeescript is an elegant language, created by Jeremy Ashkenas, with features from Ruby and Python. The language is very pure and removes a lot of clutter. Coffeescript is compiled into good, efficient Javascript. Trevor Burnham has written a book, CoffeeScript: Accelerated JavaScript Development, on the subject and he will be giving two presentations about it, CoffeeScript: Design Patterns for the New JavaScript and Transforming Data into Pixels: Visualization with Canvas and CoffeeScript

Charles Nutter

Charles Nutter is the man behind JRuby. He has also create another language called Mirah, which he will be talking about in Have you tried Mirah yet?.

While doing all this he has obviously learned a thing or two about the JVM and about bytecodes. Who could be better to teach us about the internals of the JVM. Charles will be giving another talk about this in What the JVM Does With your Bytecode when Nobody’s Looking.

Simon Peyton Jones

Even though this list of people is mostly about dynamic programming languages, it has to include Simon Peyton Jones.

Haskell is one of the most statically typed languages there is. It is, probably, also the most elegant programming language in the world. It is purely functional, has lazy evaluation, pattern matching, and currying by default. Even if you never use Haskell in a real-life project learning Haskell will be worth your while. If you want to get a good introduction to Haskell I can highly recommend Programming in Haskell by Graham Hutton.

Summary

As you can see, this years Øredev is looking better than ever before and I have only included a select part of it in this post. Missing it should be considered professional misconduct!

jQuery Changes From 1.4.2 to 1.6

2011-08-09T14:17:00.004+02:00

jQuery is a powerful library and it is possible to get by without using any of the new features. That’s why many of us just upgrade to a new version assuming that it is mostly bug and performance fixes. This is not the case. jQuery 1.4.2 was released in February 2010 and it’s been one and a half years and a number of releases since then.

I was going to write about the changes in 1.5 and 1.6, but I have noticed that many people have missed some of the new features of the 1.4 releases. And by the way, all examples are written in Coffeescript.

Selected changes from 1.4.2+

`delegate()`

Most people know about live() and how it can be used to attach listeners to elements that don’t yet exist in the DOM. live() has a younger brother that was born in 1.4.2 and he is called delegate(). delegate() is more powerful. It gives you more precision in where to attach the listener.

$('.main-content').find('section').delegate 'p', 'click', -> 
  $(this).addClass 'highlight'

As you can see above, delegate(), unlike live(), can be chained like normal jQuery calls.

`jQuery.now()`, `jQuery.type()` and `jQuery.parseXML()`

Nothing special here, just some utilities that are good to know about.

$.now() is (new Date()).getTime() 
 
$.type('hello') is 'string' 
 
xml = """ 
<rss version='2.0'> 
<channel> 
<title>RSS Title</title> 
</channel> 
</rss> 
""" 
xmlDoc = $.parseXML xml 
($(xmlDoc).find 'title').text() is 'RSS Title'

All these utilities are interesting, but the truly good thing that came out of jQuery-1.5+ is Deferred().

`Deferred()`

With the release of jQuery 1.5 the internal implementation of $.ajax was changed to use Deferred(), and, even better, the implementation was deemed so useful, that it became part of the public API.

Here is how you can use it via the $.ajax method.

Declare a function hex that calls the /hex url on the server, which will return a hex value between 00 and FF.

  hex = -> 
    $.ajax { url: '/hex' }

Call the function multiple times, and you get a new Deferred back for each call. Notice the lack of handlers for the calls.

  red = hex() 
  green = hex() 
  blue = hex()

Attach handlers to each of the Deferreds to do something useful with the returned value. Notice the use of done instead of success. success is deprecated and will be removed in 1.8.

  red.done (hh)-> 
    $("#r").css 'background-color', "##{hh}0000" 
  green.done (hh)-> 
    $("#g").css 'background-color', "#00#{hh}00" 
  blue.done (hh)-> 
    $("#b").css 'background-color', "#0000#{hh}"

I am not limited to adding one handler, I can attach as many as I like. Here I attach another one for logging the success of the red-call.

  red.done (hh) -> 
    console.log(hh)

If this was all there was to it, I would be happy, but it is not by using the $.when method I get a new Deferred object that orchestrates multiple Deferreds in a very simple way. Let me create a color-Deferred that waits for the others to return and then calls a new callback when they are all done.

  color = $.when(red, green, blue) 
 
  color.done (r, g, b) -> 
    $("#color").css 'background-color', "##{r[0]}#{g[0]}#{b[0]}"

The results from the requests are given in the same order as the Deferred objects are passed in. The results are given as an array with three arguments [ data, ‘success’ , deferred ].

You can see the example, a simple Sinatra app, in action on Heroku and the source code can be found on Github

Very nice! Apart from the examples I have shown, there are methods working with Deferred. Methods for creating, jQuery.Deferred(), resolving, resolve(), resolveWith(), and rejecting, reject(), rejectWith().

To attach handlers to the events, you can use done() for resolve, fail for reject, always() for both resolve and reject. You can also use then(doneCallback, failCallback) to attach both a done and a fail handler with one call.

Deferreds are really cool, check them out here

Tips from Rails Anti-Patterns

2011-06-05T15:49:00.003+02:00

Another good Ruby book is out, Rails Anti-Patterns. The book is loaded with good tips on everything from following the Law of Demeter to cleaning up your views with the use of helper methods.

Here are some things I picked up from the book.

`delegate` can take a `:prefix` argument

The delegate method from active_support is used for delegating calls to another object without having to write out the full delegating methods. It can take a prefix option to customize the name of the delegating methods.

class Address
  @attr_accessors :street, :zip
end

class Person
  attr_reader :address
  # prefix => true, results in the model name being used as prefix
  delegate :street, :zip, :to => :address, :prefix => true
  #@person.address_street, @person.address_zip
end

class Person
  attr_reader :billing_address, :delivery_address
  # prefix => string, uses the string as prefix
  delegate :street, :zip, :to => :address, :prefix => delivery
  #@person.delivery_street, @person.delivery_zip
  delegate :street, :zip, :to => :address, :prefix => billing
  #@person.billing_street, @person.billing_zip
end

Transaction Scope

The code executed in the ActiveRecord callbacks execute in the same transaction as the actual call to save, create, update, or delete. Knowing this helps to eliminate unneccessary explicit transactions.

# Using a before filter
class Drink
  before_create :remove_ingredients_from_bar

  def remove_ingredients
    ingredients.each do |ingredient|
      Bar.remove(ingredient)
    end
  end
end

# Is better than using an explicit transaction
class Drink
  def create_drink
    transaction do
      remove_ingredients
      create
    end
  end

  def remove_ingredients
    ingredients.each do |ingredient|
      Bar.remove(ingredient)
    end
  end
end

Association Methods

It is possible to add methods directly on the activerecord associations. This is especially handy if the method uses information from both sides of the relation.

class Drink
  #has_field :minimum_drinking_age
end

class Customer
  #has_field :age

  has_many :drinks do
    def allowed
      # proxy_owner is the object defining the relation, Customer
      where(['minimum_drinking_age < ?', proxy_owner.age])
    end
  end
end

When to make a model active

If there is no user interface for adding, removing, or managing data, there is no need for an active model. A denormalized column populated by a hash or array of possible values is fine.

This is really just the application of the KISS principle, Keep It Simple Stupid, but I have never seen it as clearly described before reading this book.

Haml []

A nice feature of Haml that I didn’t know about, is the [] operator. When given an object, such as [record], [] acts as a combination of div_for and content_for, outputting a tag with the id and class attributes set appropriately.

-# This Haml
%span[@team]

<!-- Results in this HTML -->
<span class='team' id='team_1'></span>

RESTful actions

When using resources in Rails there are seven methods that are used.

index, create, show, update, delete, edit, and new. The first five naturally map to get(collection), post(collection), get(singular), put(singular), and delete(singular), but what isn’t as obvious is that.

The new and edit actions are really just different ways of representing the show action.

This is of course obvious when you think about it, but once again Chad and Tammer has written it down in plain simple English.

Rake Tasks

How should you treat your application specific Rake tasks on order to test them easily. Once again the solution is very simple.

Write the domain specific code as a class method on the appropriate model associated with the task.

Then all you have to do is call the method from the task.

task :fill_bar_with_ingredients do
  Bar.fill_with_ingredients
end

It is not always appropriate to add this functionality to the existing models. This is a clue that another model is needed in your domain.

task :fill_bar_with_ingredients do
  LiquorStore.order_ingredients
end

Database Index

Since most applications have far more reads than writes, you should add indexes to every field that appears in a WHERE clause or an ORDER clause. You should also add indexes for every combination of fields that are used that are combined with AND.

As always, don’t follow this advice blindely, if a table only has three rows then perhaps the index is overkill…

Conclusion

Apart from these tips, there are tons of other useful information, making this book a must-read if you are doing Rails development.

Ruby, an Exceptional Language

2011-05-25T11:35:00.001+02:00

Based on the book Exceptional Ruby by Avdi Grimm, I have developed a strategy for how I should deal with exceptions in Ruby.

Being a very dynamic language, Ruby allows very flexible coding techniques. Exceptions are not an exception :).

When I am developing a library in Ruby I typically create one Error module and one StdError class. The Error module is a typical tag module and does not contain any methods.

Tag Module

# Tag module for the Tapir library 
module Tapir
  module Error
  end 
end

The reason for the tag module is that I can use it to tag exceptions occurring inside my library without having to wrap them in a nested exception.

module Tapir
  class Downloader
 
    def self.get url
      HTTP.get url
    rescue StandardError -> error   # Rescue the error 
      # Namespace the error by tagging it with ::Tapir::Error 
      error.extend(::Tapir::Error)  
      raise                         # And raise it again 
    end 
 
  end 
end 
 
 
# Client usage 
begin 
  Tapir::Downloader.get 'http://non.existent.url/' 
rescue Tapir::Error => error
  puts "Stupid tapir, gave me error #{error.message}" 
end

This is beautiful. I am scoping an internal error as my own. Since Ruby is dynamic there is no need to declare a new class that wraps all the methods in the StandarError I have access to them anyway. Duck typing for the win!

A Nested Exception Class

In some cases the tag module is not enough. Perhaps the exception was not created by another exception. In that case I need a real class since it is not possible to raise modules. But while I am at it I usually make the class a nested exception in order to simplify wrapping of other exceptions if the need comes up. This is how I do that.

module Tapir
  # I usually call the class `StdError` since it prevents the user of 
  # the library from rescuing the global `StandardError`. 
  class StdError < StandardError 
    extend Error             # Extend the Error tag module 
    attr_reader :original    # Add an accessor for the original, if one exists 
   
    # Create the error with a message and an original that defaults to 
    # the exception that is currently active, in this thread, if one exists 
    def initialize(msg, original=$!) 
      super(msg) 
      @original = original; 
    end 
  end 
end 
 
 
# Client Usage 
begin 
  Tapir.do_something_that_fails
rescue Tapir::Error => error      # rescue the tag module 
  puts "Bad tapir #{error.message}, due to #{error.original.message}" 
end 
 
# or if I want to be more specific 
begin 
  Tapir.do_something_that_fails
rescue Tapir::StdError => error   # rescue the specific error 
  puts "Bad tapir #{error.message}, due to #{error.original.message}" 
end

Notice that I don’t have to wrap the exception explicitly, since I default the Exception to the last error that is stored in $!.

Now the only reason for me to want to create a Tapir::StdError apart from it being misuse of my library is if I want to add additional information to the exception that already occurred. In that case I may also want to extend the Tapir::StdError and create an exception with additional fields.

module Tapir
  
  # Create a specific exception to add more information for the client 
  class TooOldError < StdError
    attr_reader :age, :max_age
 
    def initialize(msg, original=$!, age, max_age) 
      super(msg, original) 
      @age, @max_age = age, max_age
    end 
  end 
end 
 
# Client usage 
begin 
  tapir.mate(other_tapir) 
rescue TooOldError => error
  # Use the specific error properties 
  puts "Hey, your are #{error.age}, that is too damn old!" 
end

Throw – Catch

Ruby also has an alternative to raise and rescue called throw and catch.

They should not be used as an alternative to exceptions, instead they are escape continuations that should be used to escape from nested control structures across method calls. Powerful! Here is an example from Sinatra

# Here is the throw 
 
   # Pass control to the next matching route. 
    # If there are no more matching routes, Sinatra will 
    # return a 404 response. 
    def pass(&block) 
      throw :pass, block
    end 
 
# and here is where it is caught 
 
    def process_route(pattern, keys, conditions) 
      ... 
      catch(:pass) do 
        conditions.each { |cond| 
          throw :pass if instance_eval(&cond) == false } 
        yield 
      end 
    end 
 
 
# Allowing usage such as 
 
  get '/guess/:who' do 
    pass unless params[:who] == 'Frank' 
    'You got me!' 
  end 
 
  get '/guess/*' do 
    'You missed!' 
  end

Lovely!

Wrap up

This is how I use exceptions in Ruby now, thanks to ideas from the book. Other good ideas from the book are the three guarantees:

The weak guarantee, if an exception is raised, the object will be in a consistent state.
The strong guarantee, if an exception is raised, the object will be left in its initial state.
The nothrow guarantee, no exceptions will be raised by this method.

And a nice way of categorizing exceptions based on three different usages by the client. (My categories are not exactly the same as Avdis)

User Error, the client has used the library wrong.
Internal Error, something is wrong with the library. We are looking into the problem…
Transient Error, something is now working right now, but the same call may succeed in a while. It is a good idea to provide a period after whick the call will probably succeed. the client to try again.

It is a great book which contains a lot more information than I covered here. Get it, it is well worth the money.

A Not Very Short Introduction To Node.js

2011-05-15T09:49:00.011+02:00

Node.js is a set of asynchronous libraries, built on top of the Google V8 Javascript Engine. Node is used for server side development in Javascript. Do you feel the rush of the 90's coming through your head. It is not the revival of LiveWire, Node is a different beast. Node is a single threaded process, focused on doing networking right. Right, in this case, means without blocking I/O. All the libraries built for Node use non-blocking I/O. This is a really cool feature, which allows the single thread in Node to serve thousands of request per second. It even lets you run multiple servers in the same thread. Check out the performance characteristics of Nginx and Apache that utilize the same technique.

The graph for memory usage is even better.

Read more about it at the Web Faction Blog

OK, so what's the catch? The catch is that all code that does I/O, or anything slow at all, has to be called in an asynchronous style.

// Synchronous 
var result = db.query("select * from T"); 
// Use result 
 
// Asynchronous 
db.query("select * from T", function (result) { 
    // Use result 
});

So, all libraries that deal with IO has to be re-implemented with this style of programming. The good news is that even though Node has only been around for a couple of years, there are more than 1800 libraries available. The libraries are of varying quality but the popularity of Node shows good promise to deliver high-quality libraries for anything that you can imagine.

History

Node is definitely not the first of its kind. The non-blocking select() loop, that is at the heart of Node, dates back to 1983.

Twisted appeared in Python (2002) and EventMachine in Ruby (2003).

This year a couple of newcomers appeared.

Goliath, which builds on EventMachine, and uses fibers to allow us to program in an synchronous style even though it is asynchronous under the hood.

And, the Async Framework in .Net, which enhances the compiler with the keywords async and await to allow for very elegant asynchronous programming.

Get Started

This example uses OSX as an example platform, if you use something else you will have to google for instructions.

# Install Node using Homebrew 
$ brew install node
==> Downloading http://nodejs.org/dist/node-v0.4.7.tar.gz
######################################################################## 100.0% 
==> ./configure --prefix=/usr/local/Cellar/node/0.4.7 
==> make install
==> Caveats
Please add /usr/local/lib/node to your NODE_PATH environment variable to have node libraries picked up. 
==> Summary
/usr/local/Cellar/node/0.4.7: 72 files, 7.5M, built in 1.2 minutes

When installed you have access to the node command-line command. When invoked without arguments, it start a REPL.

$ node
> function hello(name) {
... return 'hello ' + name; 
... }
> hello('tapir') 
'hello tapir' 
>

When invoked with a script it runs the script.

// hello.js 
setTimeout(function() { 
    console.log('Tapir'); 
}, 2000); 
console.log('Hello');

$ node hello.js
Hello
... 
Tapir

Networking

As I mentioned above, Node is focused on networking. That means it should be easy to write networking code. Here is a simple echo server.

// Echo Server 
var net = require('net'); 
 
var server = net.createServer(function(socket) { 
    socket.on('data', function(data) { 
        console.log(data.toString()); 
        socket.write(data); 
    }); 
}); 
server.listen(4000);

And here is a simple HTTP server.

// HTTP Server 
var http = require('http'); 
var web = http.createServer(function(request, response) { 
  response.writeHead(200, { 
    'Content-Type': 'text/plain' 
  }); 
  response.end('Tapirs are beautiful!\n'); 
}); 
web.listen(4001);

Quite similar. A cool thing is that the servers can be started from the same file and node will, happily, serve both HTTP and echo requests from the same thread without any problems. Let's try them out!

# curl the http service 
$ curl localhost:4001 
Tapirs are beautiful! 
 
# use netcat to send the string to the echo server 
$ echo 'Hello beautiful tapir' | nc localhost 4000 
Hello beautiful tapir

Modules

Node comes with a selection of built in modules. Ryan Dahl says that they try to keep the core small, but even so the built-in modules cover a lot of useful functionality.

net - contains tcp/ip related networking functionality.
http - contains functionality for dealing with the HTTP protocol.
util - holds common utility functions, such as log, inherits, pump, ...
fs - contains filesystem related functionality, remember that everything should be asynchronous.
events - contains the EventEmitter that is used for dealing with events in a consistent way. It is used internally but it can be used externally too.

An example

Here is an example of a simple module.

// module tapir.js 
 
// require another module 
var util = require('util'); 
 
function eat(food) { 
  util.log('eating '+ food); 
} 
 
// export a function 
exports.eat = eat;

As you can see it looks like a normal Javascript file and it even looks like it has global variables. It doesn't. When a module is loaded it is wrapped in code, similar to this.

var module = { exports: {}}; 
(function(module, exports){ 
  // module code from file 
  ... 
})(module, module.exports);

As you can see the code is wrapped in a function and an empty object with an export property is sent into it. This is used by the file to export only the functions that it want to publish.

The require function works in symphony with the module and it returns the exported functions to the caller.

Node Package Manager, npm

To allow simple handling of third-party packages, Node uses npm. It can be installed like this:

$ curl http://npmjs.org/install.sh | sh
...

And used like this:

$ npm install -g express
mime@1.2.1 /usr/local/lib/node_modules/express/node_modules/mime
connect@1.4.0 /usr/local/lib/node_modules/express/node_modules/connect
qs@0.1.0 /usr/local/lib/node_modules/express/node_modules/qs
/usr/local/bin/express -> /usr/local/lib/node_modules/express/bin/express
express@2.3.2 /usr/local/lib/node_modules/express

As you can see, installing a module also installs its dependencies. This works because a module can be package with meta-data, like so:

// express/package.json 
{ 
  "name": "express", 
  "description": "Sinatra inspired web development framework", 
  "version": "2.3.2", 
  "author": "TJ Holowaychuk <tj@vision-media.ca>", 
  "contributors": [ 
    { "name": "TJ Holowaychuk", "email": "tj@vision-media.ca" }, 
    { "name": "Guillermo Rauch", "email": "rauchg@gmail.com" } 
  ], 
  "dependencies": { 
    "connect": ">= 1.4.0 < 2.0.0", 
    "mime": ">= 0.0.1", 
    "qs": ">= 0.0.6" 
  }, 
  "keywords": ["framework", "sinatra", "web", "rest", "restful"], 
  "repository": "git://github.com/visionmedia/express", 
  "main": "index", 
  "bin": { "express": "./bin/express" }, 
  "engines": { "node": ">= 0.4.1 < 0.5.0" } 
}

The package.json contains information about who made the module, its dependencies, along with some additional information to enable better searching facilities.

Npm installs the modules from a common respository, which contains more than 1800 modules.

Noteworthy Modules

Express is probably the most used of all third-party modules. It is a Sinatra clone and it is very good, just like Sinatra.

// Create a server 
var app = express.createServer(); 
app.listen(4000); 
 
// Mount the root (/) and redirect to index 
app.get('/', function(req, res) { 
  res.redirect('/index.html'); 
}); 
 
// Handle a post to /quiz 
app.post('/quiz', function(req, res) { 
  res.send(quiz.create().id.toString()); 
});

Express uses Connect to handle middleware. Middleware is like Rack but for Node (No wonder that Node is nice to work with when it borrows its ideas from Ruby :)

connect( 
      // Add a logger 
      connect.logger() 
      // Serve static file from the current directory 
    , connect.static(__dirname) 
      // Compile Sass and Coffescript files, on the fly 
    , connect.compiler({enable: ['sass', 'coffeescript']}) 
      // Profile all requests 
    , connect.profiler() 
  ).listen(3000);

Another popular library is Socket.IO. It handles the usual socket variants, such as WebSocket, Comet, Flash Sockets, etc...

var http = require('http'); 
var io = require('socket.io'); 
 
server = http.createServer(function(req, res){...}); 
server.listen(80); 
 
// socket.io attaches to an existing server 
var socket = io.listen(server); 
socket.on('connection', function(client){ 
  // new client is here! 
  client.on('message', function(){ ... }) 
  client.on('disconnect', function(){ ... }) 
});

MySql has a library for Node.

client.query( 
  'SELECT * FROM ' + TEST_TABLE, 
  // Note the callback style 
  function(err, results, fields) { 
    if (err) { throw err; } 
 
    console.log(results); 
    console.log(fields); 
    client.end(); 
  } 
);

And Mongoose can be used for accessing MongoDB.

// Declare the schema 
var Schema = mongoose.Schema
  , ObjectId = Schema.ObjectId; 
 
var BlogPost = new Schema({ 
    author    : ObjectId
  , title     : String
  , body      : String
  , date      : Date
}); 
 
// Use it 
var BlogPost = mongoose.model('BlogPost'); 
 
// Save 
var post = new BlogPost(); 
post.author = 'Stravinsky'; 
instance.save(function (err) { 
  // 
}); 
 
// Find 
BlogPost.find({}, function (err, docs) { 
  // docs.forEach 
});

Templating Engines

Everytime a new platform makes its presence, it brings along a couple of new templating languages and Node is no different. Along with the popular ones from the Ruby world, like Haml and Erb (EJS in Node), comes some new ones like Jade and some browser templating languages like Mustache and jQuery templates. I'll show examples of Jade and Mu (Mustache for Node).

I like Jade, because it is a Javascript dialect of Haml and it seems appropriate to use if I'm using Javascript on the server side.

!!! 5 
html(lang="en") 
  head
    title= pageTitle
    script(type='text/javascript') 
      if (foo) { 
        bar() 
      } 
  body
    h1 Jade - node template engine
    #container 
      - if (youAreUsingJade) 
        p You are amazing
      - else 
        p Get on it!

I'm not really sure if I like Mustache or not, but I can surely see the value of having a templating language which works both on the server side and in the browser.

<h1>{{header}}</h1> 
{{#bug}}
{{/bug}}
 
{{#items}}
  {{#first}}
    <li><strong>{{name}}</strong></li> 
  {{/first}}
  {{#link}}
    <li><a href="{{url}}">{{name}}</a></li> 
  {{/link}}
{{/items}}
 
{{#empty}}
  <p>The list is empty.</p> 
{{/empty}}

Testing

Node comes with assertions built in, and all testing frameworks build on the Assert module, so it is good to know.

assert.ok(value, [message]); 
assert.equal(actual, expected, [message]) 
assert.notEqual(actual, expected, [message]) 
assert.deepEqual(actual, expected, [message]) 
assert.strictEqual(actual, expected, [message]) 
assert.throws(block, [error], [message]) 
assert.doesNotThrow(block, [error], [message]) 
assert.ifError(value) 
assert.fail(actual, expected, message, operator) 
 
// Example 
// assert.throws(function, regexp) 
assert.throws( 
  function() { throw new Error("Wrong value"); }, 
  /value/ 
);

Apart from that there are at least 30 different testing frameworks to use. I have chosen to use NodeUnit since I find that it handles asynchronous testing well, and it has a nice UTF-8 output that looks good in the terminal,

// ./test/test-doubled.js 
var doubled = require('../lib/doubled'); 
 
// Exported functions are run by the test runner 
exports['calculate'] = function (test) { 
    test.equal(doubled.calculate(2), 4); 
    test.done(); 
}; 
 
// An asynchronous test 
exports['read a number'] = function (test) { 
    test.expect(1); // Make sure the assertion is run 
 
    var ev = new events.EventEmitter(); 
    process.openStdin = function () { return ev; }; 
    process.exit = test.done; 
 
    console.log = function (str) { 
        test.equal(str, 'Doubled: 24'); 
    }; 
 
    doubled.read(); 
    ev.emit('data', '12'); 
};

Deployment

There are already a lot of platforms providing Node as a service (PaaS , Platform as a Service). Most of them are using Heroku style deployment by pushing to a Git remote. I'll show three alternatives that all provide free Node hosting.

Joyent (no.de)

Joyent, the employers of Ryan Dahl, give you ssh access so that you can install the modules you need. Deployment is done by pushing to a Git remote.

$ ssh node@my-machine.no.de
$ nmp install express
$ git remote add node node@andersjanmyr.no.de:repo
$ git push node master
Counting objects: 5, done. 
Delta compression using up to 2 threads. 
Compressing objects: 100% (3/3), done. 
Writing objects: 100% (3/3), 321 bytes, done. 
Total 3 (delta 2), reused 0 (delta 0) 
remote: Starting node v0.4.7... 
remote: Successful
To node@andersjanmyr.no.de:repo
  8f59169..c1177b0  master -> master

Nodester

Nodester, gives you a command line tool, nodester, that you use to install modules. Deployment by pushing to a Git remote.

$ nodester npm install express
$ git push nodester master
Counting objects: 5, done. 
Delta compression using up to 2 threads. 
Compressing objects: 100% (3/3), done. 
Writing objects: 100% (3/3), 341 bytes, done. 
Total 3 (delta 2), reused 0 (delta 0) 
remote: Syncing repo with chroot
remote: From /node/hosted_apps/andersjanmyr/1346-7856c14e6a5d92a6b5374ec4772a6da0.git/. 
remote:    38f4e6e..8f59169  master     -> origin/master
remote: Updating 38f4e6e..8f59169
remote: Fast-forward
remote:  Gemfile.lock |   10 ++++------
remote:  1 files changed, 4 insertions(+), 6 deletions(-) 
remote: Checking ./.git/hooks/post-receive
remote: Attempting to restart your app: 1346-7856c14e6a5d92a6b5374ec4772a6da0
remote: App restarted.. 
remote: 
remote: 
remote:     \m/ Nodester out \m/ 
remote: 
remote: 
To ec2-user@nodester.com:/node/hosted_apps/andersjanmyr/1346-7856c14e6a5d92a6b5374ec4772a6da0.git
   38f4e6e..8f59169  master -> master

Cloud Foundry

Cloud Foundry is one of the most interesting platforms in the cloud. It was genius by VM Ware to open source the platform, allowing anyone to set up their own cloud if they wish. If you don't want to setup your own Cloud Foundry Cloud, you can use the service hosted at cloundfoundry.com.

With Cloud Foundry, you install the modules locally and then they are automatically deployed as part of the vmc push. Push in this case does not mean git push, but instead, copy all the files from my local machine to the server.

$ npm install express  # Install locally 
mime@1.2.1 ./node_modules/express/node_modules/mime
connect@1.4.0 ./node_modules/express/node_modules/connect
qs@0.1.0 ./node_modules/express/node_modules/qs
express@2.3.0 ./node_modules/express
 
$ vmc push
Would you like to deploy from the current directory? [Yn]: Y
Application Name: snake
Application Deployed URL: 'snake.cloudfoundry.com'? 
Detected a Node.js Application, is this correct? [Yn]: 
Memory Reservation [Default:64M] (64M, 128M, 256M, 512M, 1G or 2G) 
Creating Application: OK
Would you like to bind any services to 'snake'? [yN]: 
Uploading Application: 
  Checking for available resources: OK
  Packing application: OK
  Uploading (1K): OK
Push Status: OK
Staging Application: OK
Starting Application: ........OK

Tools

There are of course a bunch of tools that come with a new platform, Jake, is a Javascript version of Rake, but I am happy with Rake and I don't see the need to switch. But, there are some tools that I cannot live without when using Node.

Reloaders

If you use the vanilla node command then you have to restart it every time you make a change to a file. That is awfully annoying and there are already a number of solutions to the problem.

# Nodemon watches the files in your directory and reloads them if necessary 
$ npm install nodemon
nodemon@0.3.2 ../node_modules/nodemon
 
$ nodemon server.js 
30 Apr 08:21:23 - [nodemon] running server.js 
... 
# Saving the file 
30 Apr 08:22:01 - [nodemon] restarting due to changes... 
 
# Alternative 
$ npm install supervisor
$ supervisor server.js 
DEBUG: Watching directory '/evented-programming-with-nodejs/.

Debuggers

Another tool that it is hard to live without is a debugger. Node comes with one built in. It has a gdb flavor to it and it is kind of rough.

$ node debug server.js
debug> run
debugger listening on port 5858 
connecting...ok
break in #<Socket> ./server.js:9 
    debugger; 
 
debug> p data.toString(); 
tapir

// Javascript 
var echo = net.createServer(function(socket) { 
  socket.on('data', function(data) { 
      debugger; // <= break into debugger 
      socket.write(data); 
  }); 
});

If you want a GUI debugger, it is possible to use the one that comes with Chrome by installing the node-inspector. It is started similarly to the built in debugger, but the --debug is an option instead of a subcommand.

$ node-inspector & 
visit http://0.0.0.0:8080/debug?port=5858 to start debugging
 
$ node --debug server.js debugger listening on port 5858

After that you can just fire up Chrome on the URL, http://0.0.0.0:8080/debug?port=5858 and you can debug the node process just as if it was running in the browser.

Idioms

Idioms, patterns, techniques, call it what you like. Javascript code is littered with callbacks, and event more so with Node. Here are some tips on how to write good asynchronous code with Node.

Return on Callbacks

It is easy to forget to escape from the function after a callback has been called. An easy way to remedy this problem is to call return before every call to a callback. Even though the value is never used by the caller, it is an easy pattern to recognize and it prevents bugs.

 
function doSomething(response, callback) { 
  doAsyncCall('tapir', function(err, result) { 
    if (err) { 
      // return on the callback 
      return callback(err); 
    } 
    // return on the callback 
    return callback(null, result); 
  }); 
}

Exceptions in Callbacks

Exceptions that occur in callbacks cannot be handled the way we are used to, since the context is different. The solution to this is to pass along the exception as a parameter to the callback. In Node the convetion is to pass the error as the first parameter into the callback.

function insertIntoTable(row, function(err, data) { 
  if (err) return callback(err); 
  ... 
 
  // Everything is OK 
  return callback(null, 'row inserted'); 
}

Parallel Execution

If you have multiple tasks that need to be finished before you take some new action, this can be handled with a simple counter. Here is an example of a simple function that starts up a bunch of functions in parallel and waits for all of them to finish before calling the callback.

// Do all in parallel 
function doAll(collection, callback) { 
  var left = collection.length; 
  collection.forEach(function(fun) { 
    fun(function() { 
      if (--left == 0) callback(); 
    }); 
  }); 
}; 
 
// Use it 
var result = []; 
doAll([ 
  function(callback) { 
    setTimeout(function() {result.push(1); callback();}, 2000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(2); callback();}, 3000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(3); callback();}, 1000 )} 
  ], function() { return result; } 
 
// returns [3, 1, 2]

Sequential Execution

Sometimes the ordering is important. Here is a simple function that makes sure that the calls are executed in sequence. It uses recursion to to make sure that the calls are handled in the correct order. It also uses the Node function process.nextTick() to prevent the stack from getting to large for large collections. Similar results can be obtained with setTimeout() in browser Javascript. It can be seen as a simple trick to achieve tail recursion.

function doInSequence(collection, callback) { 
    var queue = collection.slice(0); // Duplicate 
 
    function iterate() { 
      if (queue.length === 0) return callback(); 
      // Take the first element 
      var fun = queue.splice(0, 1)[0]; 
      fun(function(err) { 
        if (err) throw err; 
        // Call it without building up the stack 
        process.nextTick(iterate); 
      }); 
    } 
    iterate(); 
} 
 
 
var result = []; 
doInSequence([ 
  function(callback) { 
    setTimeout(function() {result.push(1); callback();}, 2000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(2); callback();}, 3000 )}, 
  function(callback) { 
    setTimeout(function() {result.push(3); callback();}, 1000 )} 
  ], function() { return result; }); 
 
// Returns [1, 2, 3]

Library Support for Asynchronous Programming

If you don't want to write these functions yourself, there are a few libraries that can help you out. I'll show two version that I like.

Fibers

Fibers are also called co-routines. Fibers provide two functions, suspend and resume, which allows us to write code in a synchronous looking style. In the Node version of fibers, node-fibers, suspend and resume are called yield() and run() instead.

require('fibers'); 
var print = require('util').print; 
 
function sleep(ms) { 
    var fiber = Fiber.current; 
    setTimeout(function() { fiber.run(); }, ms); 
    yield(); 
} 
 
Fiber(function() { 
    print('wait... ' + new Date + '\n'); 
    sleep(1000); 
    print('ok... ' + new Date + '\n'); 
}).run(); 
print('back in main\n');

Fibers are a very nice way of writing asynchronous code but, in Node, they have one drawback. They are not supported without patching the V8 virtual machine. The patching is done when you install node-fibers and you have to run the command node-fibers instead of node to use it.

The `async` Library

If you don't want to use the patched version of V8, I can recommend the async library. Async provides around 20 functions that include the usual 'functional' suspects (map, reduce, filter, forEach...) as well as some common patterns for asynchronous flow control (parallel, series, waterfall...). All these functions assume you follow the Node convention of providing a single callback as the last argument of your async function.

async.map(['file1','file2','file3'], fs.stat, function(err, results){ 
    // results is now an array of stats for each file 
}); 
 
async.filter(['file1','file2','file3'], path.exists, function(results){ 
    // results now equals an array of the existing files 
}); 
 
async.parallel([ 
    function(){ ... }, 
    function(){ ... } 
], callback); 
 
async.series([ 
    function(){ ... }, 
    function(){ ... } 
], callback);

Conclusion

Node is definitely an interesting platform. The possibility to have Javascript running through the whole stack, from the browser all the way down into the database (if you use something like CouchDB or MongoDB) really appeals to me. The easy way to deploy code to multiple, different cloud providers is also a good argument for Node.

Mastery

2011-05-08T10:52:00.007+02:00

Here is a wonderful story, told by Ken Robinson in his book The Element, about an 8 year old girl, Gillian Lynne, who everyone sees as a problem child.

Gillian and her mother went to a psychiatrist and the mother describes her daughter's difficulties with concentrating, sitting, and doing her homework.

The doctor listened to the mother and then he told Gillian that he wanted to talk to her mother alone. When they left the room, the doctor turned on some music on the radio. Outside the room, the doctor said to the mother "Just stand her and watch her!" As they watched, the girl started moving to the music with a grace that anyone would have been impressed by.

After watching for a few minutes, the doctor turned to the mother and said: "There is nothing wrong with your daughter, she's a dancer, take her to a dance school!"

Gillan grew up to become one of the most accomplished choreographers of our time.

This is a story about how people shine when they are allowed to do what they are meant to do, when they are in their element. But, what it does not take into account is work, hard work!

Thomas Jefferson put it nicely:

I find that the harder I work, the more luck I seem to have!

This post is not about talent, it is about mastery, the hard work of mastery!

Talent is not a requirement for Mastery, in fact talent can get in your way on the path of mastery, if it is too easy in the beginning we may not be able to put in the hours needed for true mastery when it get hard or boring.

Many of the ideas for this post come from the book called Mastery by George Leonard and he describes it well.

We fail to realize that mastery is not about perfection. It's about a process, a journey. The master is the one who stays on the path day after day, year after year. The master is the one who is willing to try, and fail, and try again, for as long as he or she lives.

Leonard identifies five keys to mastery. They are instruction, practice, surrender, intentionality, and the Edge.

Instruction

Instruction is vital. How do we know that what we are doing is the right thing. As a programmer I am always confronted with new things. How do I tell the good from the bad? After you reach a certain level of experience it gets easier to make an educated guess, but it is still just a guess. And before we have a lot of experience, we are liable to fall into traps all the time, (EJB anyone?).

My way of dealing with new things is to read books, papers, and blogs, listen to podcasts, watch screencasts, and if I can find the time, attend workshops at conferences. But, it is important to listen to critique as well as praise of a new technique. If it seems reasonable to you, by all means, give it a shot. And then, not until you have actually used a system for quite a while, can you tell if it is good or not for you. Because that is another aspect of mastery, we are all on our own path, what is right for me may not be right for someone else!

Practice

Practice is the true essence of Mastery. Without it, everything else falls to pieces. Practice is the path that build the foundation for mastery. Another quote from Leonard:

How do you best move toward mastery? To put it simply, you practice diligently, but you practice primarily for the sake of the practice itself.

An interesting fact about practice is how the normal learning curve looks.

Do you recognize it? Long periods where you practice and practice but you don't seem to get any better. Then suddenly there is a small improvement and you feel like you finally get it, and then after a little while you are back to normal again.

The more I know, the more I know that I don't know. --Socrates (misquoted:)

Surrender

What Leonard calls surrender I would call humility. A Zen Master would probably call it Beginners Mind.

Beginner's Mind. It refers to having an attitude of openness, eagerness, and lack of preconceptions when studying a subject, even when studying at an advanced level, just as a beginner in that subject would.

It can be very hard to give up your hard earned skills in order to grasp a new concept. But, sometimes, the only way to improve is to start over with a completely different approach.

How do we expect to learn something new if we are not willing to look like an idiot for awhile?

Intentionality

You gotta be careful if you don't know where you're going, because you might not get there. -- Yogi Berra

Intentionality means a lot of things. It means character, discipline, stamina or willpower. But it also means knowing what you want.

Do you know what you want? Unless you do, it is very hard to stay on the path. In my mind intentionality is not necessarily goals, it is more a direction. If I stop for a while and look around, I can tell if I am on the path, because if I am, I will, slowly, be moving in the direction I intend to go.

When you get to the top of the mountain, keep climbing. -- Chinese proverb

The Edge

Here's another quote that I like, from the book, Zorba:

No you're not free! The string that you are tied to is perhaps longer than others'. You're on a long piece of string boss, but you never cut the string in two, you need a touch of folly to do that... A man's head is like a grocer, it keeps accounts. It never risks all it has, always keeps something in reserve. It never breaks the string. But if a man never breaks the string, tell me what flavor is left in life? Nothing, but the flavor of weak camomile tea. Nothing like rum that makes you see life inside out!

In order to make the really big leaps like, for example, Einstein's Relativity Theory, we need to be a bit crazy. It is impossible to analytically reach certain conclusions. They can only be reached by thinking so far outside the box, that most people would consider us insane.

Conclusion

I will conclude with another quote from Leonard:

It is easy to get on the path of mastery. The real challenge lies in staying on it.

What Is New in EcmaScript 5?

2011-04-05T10:44:00.004+02:00

More and more browsers have implemented EcmaScript 5. That means that it is time to get up to speed on what it means for us as developers

Here is a summary of Douglas Crockfords talk on Ecmascript 5 at Scandev

New Syntax

It is now possible to use reserved words as property names and in unquoted literal keys.
A comma is allowed after the last pair in object literals and after the last element in array literals.
It is possible to break strings by inserting a backslash at the end of the line (not recommended by Doug).

object.return = 'please';
object = { throw: 'down', };
array = ['a', 'tapir', 'is', 'lovely', ];
string = "a tapir is \
beautiful"

Library Additions

JSON

JSON.parse and JSON.stringify are now part of the standard.

Date

Date has new methods, toISOString and toJSON It is also able to parse ISO strings.

Object

Object gets a bunch of new methods. But they are added on Object and not on Object.prototype.


// Creates an object with parent as prototype and properties from donor
Object.create(parent, donor);
// Meta properties of an object
var descriptor = {
value: "test",
writable: true, // Can the value be changed?
enumerable: true, // Will it appear in for-in and Object.keys(object)?
configurable: true, // Can the property be removed?
set: function(value) { test = value}, // Getter
get: function() { return test } // Setter
}
// Methods for manipulation the descriptors
Object.defineProperty(object, property, descriptor)
Object.defineProperties(object, descriptors)
Object.getOwnPropertyDescriptor(object, property)
Object.getPrototypeOf(object)
// Returns an array of enumerable properties
Object.keys(object)
// Returns an array of all properties
Object.getOwnPropertyNames(object)
// Prevents anyone from adding properties to the object, cannot be undone.
Object.preventExtensions(object)
Object.isExtensible(object)
// Prevents anyone from changing, properties or descriptors of the object.
// The values can still be changed
Object.seal(object)
Objcect.isSealed(object)
// Prevents any changes to the object.
Object.freeze(object)
Object.isFrozen(object)

Array

Arrays are extended with a bunch of new, useful methods, leading to less need for third-party libraries.


// Do all elements satisfy predicate?
Array.prototype.every(predicate)
// Return a new array with the elements that satisfy predicate?
Array.prototype.filter(predicate)
// Call action(element) for each element.
Array.prototype.forEach(action)
// What is the index of the first element that equals value?
Array.prototype.indexOf(value, fromIndex)
// What is the index of the last element that equal value?
Array.prototype.lastIndexOf(value, fromIndex)
// Create a new array by applying unaryFunc to each element
Array.prototype.map(unaryFunc)
// Reduces the elements of the array, by applying binaryFunc
// between the elements
// [a0, a1].reduce(+ , seed) == seed + a0 + a1
Array.prototype.reduce(binaryFunc, seed)
// Is at least one element satisfied by the predicate?
Array.prototype.some(predicate)

String

String.prototype.trim is added to string. It trims whiespace from the beginning and the end of a string.

Function

Function.prototype.bind(thisArg, arg, ...) is added to function. It creates a new function by binding the value of this to a specified object.


var tapir = {
method: function(name){
this.name = name;
}
};
setTimeout( tapir.method.bind(tapir, "Malayan"), 100 );

Strict Mode

A new mode called strict can be turned on by writing

function() {
"use strict";
}

It must be the first line in the function. It could also be the first line of the file, but that does not play well with the optimizing practices used in Javascript, like concatenation and minification.

In strict mode:

Variables must be declared before usage.
The with statement cannot be used.
eval is a reserved word.
arguments.caller and arguments.callee cannot be used.
etc.

Since IE6-IE9 does not implement strict model, Crockford thinks that they "must die" since they, "hinder the creation of useful mashups".

High Performance Javascript

2011-03-28T20:43:00.001+02:00

Here are some tips on high perfomance Javascript I have picked up. Most of it comes from the books High Performance Javascript by Nicholas C. Zakas and High Performance Web Sites by Steve Souders.

Loading

Load files at the end of the HTML page

Load the Javascript files right before the body, this will allow the page to render without having to wait for all the Javascript files.

Group files together

With normal loading, the files are loaded sequentially. Each file will be loaded and parsed before the next file starts to load. Merge them together into one large file. While you are at it, you should also minimize it. Tools to help you with this are:

Load files asynchronously

If normal loading with grouped files is not good enough, it is also possible to load the files asynchronously. This will also allow you to load files on demand. Tools for this are:

Variable Access

Literal values and local variables can be accessed very quickly. Array access and member access take longer. If the prototype chain or scope chain needs to be traversed, it will take longer the further up the chain the access is. Global variables are always the slowest to access because they are always last in the scope chain.

You can improve the performance of JavaScript code by storing frequently used, object members, array items, and out-of-scope variables, in local variables.

DOM

All DOM manipulation is slow.

Minimize DOM access
Use local variables to store DOM references you'll access repeatedly.
HTML collections represent the live, underlying document, so:
- Cache the collection length into a variable and use it when iterating
- Make a copy of the collection into an array for heavy work on collections

Reflow and rendering

The browser contains two trees, the DOM tree and a render tree. Whenever the DOM layout or geometry is changed the view will have to be re-rendered. This is known as reflow.

Reflow happens when:

Visible DOM elements are added or removed
Elements change position
Elements change size (margin, padding, border thickness, width, height, etc.)
Content is changed, (text changes or an image is replaced with one of a different size)
The page renders initially
The browser window is resized

Combine multiple DOM and style changes into a batch and apply them once. This can be done with documentFragments or by cloning the node.

// Create a document fragment 
var fragment = document.createDocumentFragment(); 
 
// Do something with framgment 
 
// Append the fragments children to the DOM 
document.getElementById('mylist').appendChild(fragment);

// Clone Node 
var old = document.getElementById('mylist'); 
var clone = old.cloneNode(true); 
 
// Do something with clone 
 
// Replace node with clone 
old.parentNode.replaceChild(clone, old);

Algorithms and Flow

Use algortithms with better complexity performance for large collections.

for-in loops are slower than for, while and do-while loops. Avoid for-in unless you need to iterate over a number of unknown object properties.
Lookup-tables are faster than multiple conditionals.
Recursion can be re-written with iteration if you get stackoverflow errors.

Strings and Regexes

Strings concatenation is quite fast in most browsers. In IE, you may need to use Array.join.

Regular expression can be improved by:

Focus on failing faster.
Start regexes with simple, required tokens.
Make quantified patterns and their following pattern mutually exclusive /"[^"]*"/.
Use noncapturing groups. (?:) instead of ().
Capture text to reduce postprocessing.
Expose required tokens /^(ab|cd)/ instead of /(^ab|^cd)
Resue regexes by assigning them to variables.
Split complex regexes into simpler pieces.

Responsiveness

The total amount of time that a single JavaScript operation should take is 100. If it takes longer it needs to be split up, this can be done using timers.

Two determining factors for whether a loop can be done asynchronously using timers:

Does the processing have to be done synchronously?
Does the data have to be processed sequentially?

// Function for processing an array in parallel 
function processArray(items, process, callback) { 
  var minTimeToStart = 25; 
  var copyOfItems = items.concat(); 
  setTimeout(function() { 
    process(copyOfItems.shift()); 
    if (copyOfItems.lengthÂ > 0) 
      setTimeout(arguments.callee, minTimeToStart); 
    else 
      callback(items); 
  }, minTimeToStart); 
} 
 
// Function for processing multiple tasks in parallel 
function processTasks(tasks, args, callback) { 
  var minTimeToStart = 25; 
  var copyOfTasks = steps.concat(); 
  setTimeout(function() { 
    var task = copyOfTasks.shift(); 
    task.apply(null, args || []); 
    if (copyOfTasks.length > 0) 
      setTimeout(arguments.callee, minTimeToStart); 
    else 
      callback(); 
  }, minTimeToStart); 
}

You should limit the number of high-frequency repeating timers in your web application. It is better to create a single repeating timer that performs multiple operations with each execution.

It is not recommended to have minTimeToStart less than 25 milliseconds, because there is a risk that the timers will fill up the queue.

// Timed version of process array, where each version is able to 
// process items from the array for up to 50 milliseconds. 
function timedProcessArray(items, process, callback) { 
  var minTimeToStart = 25; 
  var copyOfItems = items.concat(); 
  setTimeout(function() { 
    // (+) converts the Date object into a numeric representation 
    var start = +new Date(); 
    do { 
      process(copyOfItems.shift()); 
    } while (copyOfItems.length > 0 && (+new Date() - start < 50)); 
    if (copyOfItems.length > 0) 
      setTimeout(arguments.callee, minTimeToStart); 
    else 
      callback(items) 
  }, minTimeToStart); 
}

Newer browsers support web workers. Web workers does not run in the UI-thread and does not affect responsiveness at all. Their environment is limited to allow this to work. It is limited to:

A navigator object, which contains only four properties: appName, appVersion, user Agent, and platform.
A location object (same as on window, except all properties are read-only)
A self object that points to the global worker object
An importScripts() method that is used to load external JavaScript for use in the worker
All ECMAScript objects, such as Object, Array, Date, etc.
The XMLHttpRequest constructor
The setTimeout() and setInterval() methods
A close() method that stops the worker immediately

It is not possible to create a WebWorker from code. It needs to be started with its own javascript file. You can however communicate with it through events.

// Application code 
var worker = new Worker("code.js"); 
worker.onmessage = function(event) { 
  alert(event.data); 
}; 
worker.postMessage("Tapir"); 
 
// Worker code (code.js) 
//inside code.js 
importScripts("file1.js", "file2.js"); // importing some files 
 
self.onmessage = function(event) { 
  self.postMessage("Hello, " + event.data + "!"); 
};

Any code that takes longer than 100 milliseconds to run should be refactored to use webworkers to decrease the load on the UI-thread.

Ajax

Favor lightweight formats in general; the best is JSON and a character-delimited custom format. If the data set is large and parse time becomes an issue, use one of these two techniques:

JSON-P data, fetched using dynamic script tag insertion. This treats the data as executable JavaScript, not a string, and allows for extremely fast parsing. This can be used across domains, but shouldn't be used with sensitive data.

A character-delimited custom format, fetched using either XHR or dynamic script tag insertion and parsed using split(). This technique parses extremely large datasets slightly faster than the JSON-P technique, and generally has a smaller file size.

XML has no place in high-performance Ajax.

Cache data! The fastest Ajax request is one that you don't have to make. There are two main ways of preventing an unnecessary request:

On the server side, set HTTP headers that ensure your response will be cached in the browser.
On the client side, store fetched data locally so that it doesn't have be requested again.

Multipart XHR can be used to reduce the number of requests, and can handle different file types in a single response, though it does not cache the resources received.

Some more guidelines that will help your Ajax appear to be faster:

Reduce the number of requests you make, either by concatenating JavaScript and CSS files, or by using MXHR.
Improve the perceived loading time of your page by using Ajax to fetch less important files after the rest of the page has loaded.
Ensure your code fails gracefully and can handle problems on the server side.
Know when to use a robust Ajax library and when to write your own low-level Ajax code.

Programming Practices

Avoid the use of eval() and the Function() constructor.
Pass functions into setTimeout() and setInterval() instead of strings.
Use object and array literals when creating new objects and arrays.
Avoid doing the same work repeatedly.
Use lazy loading or conditional advance loading when browser-detection logic is necessary.
When performing mathematical operations, consider using bitwise operators that work directly on the underlying representation of the number.
Native methods are always faster than anything you can write in JavaScript.

Building and Deploying

The build and deployment process can have a tremendous impact on the performance of a JavaScript-based application. The most important steps in this process are:

Combining JavaScript files to reduce the number of HTTP requests
Minifying JavaScript files using the YUI Compressor
Serving JavaScript files compressed (gzip encoding)
Making JavaScript files cacheable by setting the appropriate HTTP response headers
Work around caching issues by appending a timestamp to filenames
Using a Content Delivery Network to serve JavaScript files;
All these steps should be automated using build tools

Tools

Minification

Profiling

Development

Firebug
Internet Explorer Developer Tools
Safari Web Inspector
Chrome Developer Tools

Proxies

Host Specific Routing Via the iPhone on OSX

2011-03-01T20:54:00.000+01:00

If you are working as a consultant, it is sometimes not easy to get proper network access when you are using your customer's network. The easiest way to solve this problem is to connect via your mobile phone. But, the 3G network is not always the fastest and it would be nice to use the normal network for certain tasks and the 3G network for others.

The problem in my case was that I wanted to connect to Github but, SSH traffic was blocked on the network where I was working. So what to do? I quick mail to the Jayway tech mailing list, lets me know that in Linux there is a route command, that can solve this problem. (Un)Fortunately I am not using Linux on my development machine, but it turns out there is a route(8) command in BSD too, with slightly different syntax.

Find the IP-address of the host I wish to connect to

# Find the IP-address of the host I wish to connect to 
$ nslookup github.com
Server:  195.58.103.21 
Address: 195.58.103.21#53 
 
Non-authoritative answer: 
Name: github.com
Address: 207.97.227.239

Find out the gateway of the interface of the cell phone

# Find out the gateway of the interface of the cell phone 
$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 
 inet 127.0.0.1 netmask 0xff000000 
 inet6 ::1 prefixlen 128 
 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 
... 
vmnet8: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 
 ether 00:50:56:c0:00:08 
 inet 192.168.105.1 netmask 0xffffff00 broadcast 192.168.105.255 
en3: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 
 ether 06:1e:64:00:34:1a 
 inet6 fe80::41e:64ff:fe00:341a%en3 prefixlen 64 scopeid 0xa 
 inet 172.20.10.3 netmask 0xfffffff0 broadcast 172.20.10.15 
 media: 10baseT/UTP
 status: active

In my case the interface I want to use is the last one, en3 with inet 172.20.10.3 but if you are unsure, you can open the network preference pane and look it up there.

Now I need to find out what the gateway for this interface is. I can do this with netstat -r.

# Show the routing table 
$ netstat -r
Routing tables
 
Internet: 
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            10.67.10.254       UGSc           10        0     en1
default            172.20.10.1        UGScI           0        0     en3
10.67.10/24        link#5             UCS             3        0     en1 
10.67.10.88        localhost          UHS             1      811     lo0
10.67.10.254       0:14:4f:69:da:13   UHLWI          10       41     en1    661 
10.67.10.255       link#5             UHLWbI          1       78     en1 
127                localhost          UCS             0        0     lo0
... 
 
Internet6: 
Destination        Gateway            Flags         Netif Expire
localhost          localhost          UH              lo0
fe80::%lo0         localhost          Uc              lo0
localhost          link#1             UHL             lo0 
...

Close to the top we find the entry for our interface en3.

default            172.20.10.1        UGScI           0        0     en3

Now we have the ip to the gateway that we wanted. So now the last step is to make sure that access to github.com is routed via this gateway.

#route command -host destination gateway 
$ sudo route add -host 207.97.227.239 172.20.10.1 
add host 207.97.227.239: gateway 172.20.10.1

That's it, I'm done. To make sure the routing is changed I check the routing table with netstat -r again, to make sure the new entry is in it.

$ netstat -r
Routing tables
 
Internet: 
Destination        Gateway            Flags        Refs      Use   Netif Expire
default            10.67.10.254       UGSc           30        0     en1
default            172.20.10.1        UGScI           0        0     en3
10.67.10/24        link#5             UCS             4        0     en1 
10.67.10.88        localhost          UHS             0      973     lo0
10.67.10.254       0:14:4f:69:da:13   UHLWI          27       11     en1    708 
10.67.10.255       ff:ff:ff:ff:ff:ff  UHLWbI          1       14     en1
127                localhost          UCS             0        0     lo0
... 
github.com         172.20.10.1        UGHS            0       43     en3
 
Internet6: 
Destination        Gateway            Flags         Netif Expire
localhost          localhost          UH              lo0
fe80::%lo0         localhost          Uc              lo0
localhost          link#1             UHL             lo0 
fe80::%en0         link#4             UC              en0

Sure enough, there it is, github.com. Finally I can now git pull and everything works :).

If I want to remove the entry, when I'm working elsewhere, I do that with

 
$ sudo route delete -host 207.97.227.239 172.20.10.1

Continuous Delivery

2011-02-26T15:55:00.001+01:00

I just finished the book, Continuous Delivery by Jez Humble and David Farley. It's a great book, but it is a bit wordy (512 pages), so here is the gist of it.

Continuous delivery builds on a simple foundation, continuous integration, automation, and version control. But in order to get it to work you must version control and automate everything. It's that simple! Version control and automate everything! Everything means different things to different people, but to Jez and David it means a lot.

You should version control:

Source code and test code
Assets, images, media files, base data, etc.
Build scripts
Deployment scripts
Secret configuration settings, like keys and passwords
Environments, development, production, test, staging, ...

The first three you are hopefully doing already, but the fourth is unusual and the fifth and sixth, I rarely see anywhere. Secret settings should be kept in version control too, just don't put them into the public repository. Here are some question to ask yourself:

Can you set up a complete development machine with one command?
Can you set up the deployment pipeline machine with one command?
Can you set up a fresh test environment with one command?
Can you set up a complete production environment with one command?
Can you upgrade the application, including data migrations, with one command?
Can you roll it back, in case it fails, with one command?

The Deployment Pipeline

Continuous Delivery uses a Continuous Itegration server to drive the Deployment Pipeline. In the simplest form of application, a deployment pipeline may have only one step. This step, runs all the tests and if all goes well it will deploy the application. But as soon as the application grows this will take too long, the single step needs to become a pipeline.

A deployment pipeline may have arbitrary many steps and they should be as automated as possible. A common setup of a deployment pipeline contains the following steps.

The Commit Stage (automatic)
The Integration Test Stage (automatic)
The Acceptance Test Stage (automatic)
Exploratory Testing Stage (manual)
Perfomance Testing Stage (automatic)
Deployment Stage (triggered manually)

The main reason for the deployment pipeline is to provide feedback as fast as possible. The commit stage typically only contains unit tests that run very fast. The steps after that only run if the step before succeeds.

The steps above may contain everything you need, too much, or in the wrong order. Feel free to remove and re-order as you please. If performance testing is cheaper than exploratory testing, in your case, switch them. There is no one-solution-fits-all when it comes to pipelines. Customize it, to fit your needs.

One important aspect of the deployment pipeline that I rarely see is that the final product should be created by the first commit stage and then resused by the other stages. Very often I see that the separate stages check out and build a new version from scratch, albeit a tagged version but still. The reuse of the same artifact through all the steps not only ensures that the same artifact has passed all the stages, it also speeds up the deployment pipeline, removing redundant work.

Another thing worth noting is that the manual steps described above should be as automatic as possible. If a tester want to test the latest build that has passed the acceptance test stage, he should be able to do it with one click. That one click may involve:

Create a fresh production like environment.
Install the selected artifact into the new envrionment.
Apply all the relevant configuration.

If the tester decides that the build should fail or succeed, he should be able to do that with one click. And, the feedback with the result should be fed back into the deploynment cycle so that the developers know that it failed, and why, or the operation guys knows that there is a new version ready to deploy.

Conclusion

The book is a good read and I highly recommend it. In the process of describing continuous delivery, it provides tons of useful tips on keeping your code and data under control. Among other things it describes.

Release strategies, such as canary relasing and blue-green deployment.
Test strategies for acceptance tests, capacity test, etc.
Why not to use long-lived feature branches and how to avoid it.
Dependency management
...

Dynamic For The Win

2011-01-09T21:50:00.001+01:00

Everything passes, everything. -- Anthony de Mello, Awareness

Take a look at what is happening around you. Relational databases are replaced by NoSQL databases. SOAP is replaced by REST, Javascript is dominating web development. The only place where dynamic is not winning is in the world of programming languages. But even if you take a look at programming languages, you can see that many of the largest most vital sites, such as Facebook and Wikipedia, are using dynamic languages. Why?

Dynamic is Better

It's not the strongest of the species that survives, nor the most intelligent, but those who are most responsive to change. -- Charles Darwin

What I mean when I say that dynamic is better is that dynamic systems are more responsive to change. And change, as they say, is the only thing that we can be sure of!

If I need to add a new field to a NoSQL database, I just do it.
If I need to add new field to my RESTful web service, I just do it.
If I need to alter the behavior of my Javascript UI, I change it, reload the page, done!

Soap vs. REST

REST is beating the hell out of SOAP nowadays, but you may have noticed that most "RESTful" web services aren't really RESTful. They don't follow the HATEOAS rule, they are just "Accessible over HTTP". So why the big fuss about "REST" if no one is using it as intended? Because there are no contracts for REST! You are not constrained by WSDL and all the other bloated specifications. You can just publish your services without contracts, write some documentation and be done with it.

Relational vs. NoSQL Databases

NoSQL is the new rave, and most people think that it is because they allow us to handle large amounts of data, fast. I don't think that is the main reason. I think the main reason is that they are schemaless. It is easy to add or remove fields without changing the data of all the other items of similar type. The problem with relational databases is not, in most cases, that they are slow, it is that they have a rigid contract that make them painful to work with. This effect is also, often, amplified by having O/R-mappers wrap the tables in static types.

Javascript vs. RIA Plugins

Javascript is the clear winner as the programming language of the web. Silverlight finally threw in the towel and Java is only living in ancient bank applets. Javascript is one of the most dynamic languages that you can imagine and it has won the battle of the web! Cool!

Programming Languages

In the area of programming languages, both client (not web) and server, dynamic is not winning, yet. Even though languages like C# and Objective-C are becoming more and more dynamic, they are still nowhere near Ruby and Python. Java has stagnated and Scala is even more typeful than Java, although type inference relieves you from having to declare most of the types. Clojure is an interesting dynamic JVM language but it suffers since you still have to compile the code.

But dynamic will win when it comes to programming languages too. Why? For the same reasons that dynamic is winning everywhere else. They are more suited for solving ever changing problems. The problems I try to solve everyday, always has to do with integrating with something else, and it is much easier to do with dynamic programming languages.

So why is dynamic not winning in the area of client and server programming? I think there are a few reasons.

Education, most of us have been taught programming at a university that propagates for statically typed, compiled languages. We have been taught that dynamic languages are bad and that they may be suitable for scripting but nothing else. It has been hammered into us, so hard that we have a hard time letting go of this.
Tradition, the folklore says that dynamic programming languages are too slow. The fact that they are not, has not changed this. You may remember that the argument was the same against Java fifteen years ago.
And then there is the main reason, dynamic is harder!

Dynamic is Harder

So why isn't everyone using dynamic programming langugages? Because, it is harder. It is easier to write really bad code in dynamic languages. But, on the other hand, it is also easier to write really good code in dynamic languages.

It is harder to develop large systems, because when you are using convention over configuration, it is really important to know the conventions and most people don't take the time to learn what they use properly, they rely on the IDE for guiding them down the right path. If you are suffering from IDEitis, the transition to dynamic languages is going to be tough until you realize that the help you get from the IDE is also what is holding you back.

The problem is moved from the programming language to me, the programmer, architect or whatever title I feel entitled to. If I am using a dynamic language and I screw up, it is my fault. If I don't organize the code in a way that can be understood by others, it is my fault. If I make changes to core classes, and the system stops working, it is my fault. I am responsible! I am in control!

An example, third party libraries in Ruby

I remember a comment from one of my colleagues, when he found out that I had a problem with a library (a Ruby Gem) that didn't behave the way I wanted it too.

Ah, you're using Ruby, then you can just fix it. -- Mattias Arthursson

Right on Mattias, I can just fix it!

Almost every Ruby Gem lives at Github. The flexibility this brings is tremendous. If a library is not working the way I think it should, I can just fix it. The process goes something like this:

Fork it
Make the appropriate changes
Push it back up to Github (to my fork)
Change the Gemfile, that holds my project dependencies, to use my fork instead.
Optionally, I may also send a pull requests of my patch to the maintainer of the library.
The maintainer takes a look at it and pulls it in, relieving me of the burden of keeping my fork up to date.

The process is so simple, it is almost seamless.

I'll finish with another quote.

It is vain to do with more, what can be done with less. -- William of Occam

It's a new year, it is time to prepare for the future, a dynamic future! Here are some books that I can recommend, they are about Ruby and Javascript, since they are my current favorites. But if you like other dynamic languages better, knock yourself out!

Books

The books are ordered, by increasing level of complexity.

Ruby

Rails

Javascript

jQuery

Review of Crafting Rails Applications, by José Valim

2010-12-30T12:00:00.002+01:00

José Valim is one of the newest members of the Rails core team. Apart from this he has also developed some good gems, Devise, Responders, and SimpleForm, that I use for almost every project.

And now he has written a book, a really good book, about advanced Rails programming techniques, called Crafting Rails Applications.

The book is only 180 pages long, and contains 7 chapters. I wish more books was this thin. The books is written in a clear concise way. José's lightweight test-driven approach clearly introduces every problem it intends to solve, and then solves the problem by introducing a new concept in the Rails framework. Most of the solutions are only a few lines of code, showing the power of Rails and also showing the power of having intimate knowledge of the framework. By hooking into the framework at appropriate places, José is able to provide elegant solutions to a range of problems.

I highly recommend the book to anyone interested in Rails.

If this was an ordinary book, this review could have ended here, but since the book is full of things I really wanted to know but didn't take the time to learn I decided to internalize them by describing parts of the book in my own words.

Chapter 1, Creating our own renderer

In this chapter we learn how to add a new renderer Rails. We add a renderer that handles generating PDF files with Prawn, allowing us to write code that looks like this.

class HomeController < ApplicationController 
  def index
    respond_to do |format| 
      format.html
      # This is the new capability, pdf rendering.
      format.pdf { render :pdf => "contents" }
    end 
  end
end

To allow the following we first have to register pdf as a mime type. We can do this with:

require "action_controller"
# Register a new mime type
Mime::Type.register "application/pdf", :pdf

# Registering the type allows us to write
respond_to do |format|
  format.pdf
end

With that out of the way all we have to do is to register a renderer, that will handle the actual rendering of the registered mime type. This is done with:

require "prawn" 

# Adds a new renderer to ActionController.Renderers
ActionController::Renderers.add :pdf do |filename, options|
  pdf = Prawn::Document.new pdf.text render_to_string(options) 
  send_data(pdf.render, :filename => "#{filename}.pdf",
      :type => "application/pdf", :disposition => "attachment")
end

This is all that is needed to add a rendering of a new type to Rails. José also goes into details about other points of interest where the rendering stack can be customized.

Chapter 2, Easy models with Active Model

Active Model is a Rails module that abstracts common behavior needed by different kinds om models such as Active Record, Active Resource and third party modules like Mongoid. Apart from giving developers of new models utility functionality. Active Model also provides tests for being Active Model compliant. To be compliant means to play well with the Rails view and controllers.

José implements a model for creating sending mail. In doing so he introduces us to a bunch of Active Model modules that will do our work for us.

Here are the modules in alphabetical order:

ActiveModel::AttributeMethods

When you include ActiveModel::AttributeMethods in your model, you get access to helper methods that will help you define additional methods for dealing with the attributes of your model. An example

class Tapir
  include ActiveModel::AttributeMethods
   
  # Define the attributes 
  attr_accessor 'name', 'color'

  # We want to create clear methods for every attribute
  attribute_method_prefix 'clear_'

  # We also want to have query methods for every attribute
  attribute_method_suffix '?'

  # Define the attribute methods that will call the protected methods below
  define_attribute_methods ['name', 'color']

protected
  # The name of this method corresponds to the prefix above
  def clear_attribute(attribute)
    send("#{attribute}=", nil)
  end

  # The name of this method corresponds to the suffix above
  def attribute?(attribute)
    send(attribute).present?
  end
end

   t = Tapir.new
 => #<Tapir:0x00000102769b90> 
> t.name?
 => false 
> t.name= 'kalle'
 => "kalle" 
> t.name?
 => true 
> t.clear_name
 => nil 
> t.name?
 => false 
> t.name

This is the gist of attribute methods, there is also an affix method that will allow you to add both a suffix and a prefix to your attributes. It is also worth taking a look inside the implmentation of this module, since the methods are defined on first usage through method_missing.

ActiveModel::Callbacks

ActiveModel::Callbacks provides methods for creating before_, after_, and `around_' callbacks to your model methods. An example:

class Tapir
  extend ActiveModel::Callbacks

  define_model_callbacks :snort

  def snort
    _run_snort_callbacks do
      snort_snort_snort
    end
  end

end

Then in inheriting classes I can use the callbacks as such:

class MountainTapir < Tapir
  before_snort :do_before_snort

  def before_snort
    i_am_so_cold
  end
end

ActiveModel::Conversion

This module assists with converting to, and representing a specific model instance. It contains three methods.

to_model, used for converting to an active model compliant model, in the default case it just returns self.
to_param, used for representing a model in routing.
to_key, used for representing a model from in a dom page.

ActiveModel::Naming and ActiveModel::Translation

ActiveModel::Naming contains one method model_name, that returns a subclass of String, that handles pluralization, etc. It has methods such as plural, human.

ActiveModel::Translation deals with I18N by adding functionality to human that handles the name lookup by keys using the preferred I18n.backend.

ActiveModel::Validations

The last, but not least of the models, ActiveModel::Validations, deals with validations. José demostrates how naming convetions and Rails constant lookup allow us to add new validators to Rails with ease. An example:

module Validators
  class TapirKindValidator < ActiveModel::EachValidator 
    def validate_each(record, attribute, value)
      unless [:baird, :mountain, :malayan, :lowland].include? value
        record.errors.add(attribute, :invalid, options) 
      end
    end
  end
end

If I now include my new validator wherever I want to use it. I can write code like:

class Tapir
  include Validators
  
  attr_accessor :kind
  validates :kind, :tapir_kind => true
end

t = Tapir.new :kind => :mexican
t.valid? 
# => false
t.errors[:kind] 
# ["is invalid"]

This example is unnecessarily complicated, since the validation could be fixed with:

class Tapir
    attr_accessor :kind
    validates_inclusion_of :kind, :in =>  [:baird, :mountain, :malayan, :lowland]
end

append_view_path

There is one more interesting thing in this chapter, append_view_path. It can be used to add a directory inside a gem to the view lookup path. This is essential to be able to deliver custom Rails components, Railties, with default views.

Chapter 3, Building a template management system

In the third chapter of the book,José takes on view resolvers. A resolver is responsible for looking up the appropriate template for a given request. The traditional resolver looks up the templates by path in the file system, but José guides us through creating a resolver that looks up the template in a database. Perfect for the core of a CMS system. I'm going to alter José's resolver to do the lookup in MongoDB instead.

# Most of this code is borrowed directly from Jos&eacute; Valim
# Only the Mongoid Part is written by me :)
require 'mongoid'
require 'singleton'

class MongoTemplate
  include Mongoid::Document
  field :body
  field :path
  field :locale
  field :format
  field :handler
  field :partial, :type => Boolean
  field :updated_at, :type => Date
    
  validates :body, :path, :presence => true
  validates :format, :inclusion => Mime::SET.symbols.map(&:to_s) 
  validates :locale, :inclusion => I18n.available_locales.map(&:to_s) 
  validates :handler, :inclusion => 
    ActionView::Template::Handlers.extensions.map(&:to_s)

  class Resolver < ActionView::Resolver 
    include Singleton
  protected
    def find_templates(name, prefix, partial, details)
      conditions = {
        :path => normalize_path(name, prefix),
        :locale => normalize_array(details[:locale]).first, 
        :format => normalize_array(details[:formats]).first,
        :handler.in => normalize_array(details[:handlers]), 
        :partial => partial || false
      }
   
      MongoTemplate.where(conditions).map do |record| 
        initialize_template(record)
      end 
    end
    
    # Normalize name and prefix, so the tuple ["index", "users"] becomes 
    # "users/index" and the tuple ["template", nil] becomes "template". 
    def normalize_path(name, prefix)
      prefix.present? ? "#{prefix}/#{name}" : name
    end
   
    # Normalize arrays by converting all symbols to strings.
    def normalize_array(array)
      array.map(&:to_s)
    end

    # Initialize an ActionView::Template object based on the record found.
    def initialize_template(record) 
      source = record.body 
      identifier = "MongoTemplate - #{record.id} - #{record.path.inspect}" 
      handler = ActionView::Template.registered_template_handler(record.handler)
      details = { 
        :format => Mime[record.format], 
        :updated_at => record.updated_at, 
        :virtual_path => virtual_path(record.path, record.partial)
      }
      ActionView::Template.new(source, identifier, handler, details)
    end
    
    # Make paths as "users/user" become "users/_user" for partials.
    def virtual_path(path, partial) 
      return path unless partial 
      if index = path.rindex("/")
        path.insert(index + 1, "_")
      else
        "_#{path}"
      end 
    end
  end
end

A Resolver is implemented by extending ActionView::Resolver and implementing the method find_templates(name, prefix, partial, details) There are a lot of extra information in the book, about caching etc. Obviously Rails caches the templates since it would be too slow to create a template every time. That is why we have the template as a singleton and that is why we clear the cache in the after_save hook above. The hook in Mongoid works exactly the same as in Active Record. Thank you Active Model!

Chapter 4, Multipart Emails with Markdown and Erb

In this chapter José walks us through three topics. The template handler API, Rails generators and Railties. I'm going to skip over the last two in this chapter. Generators are a large topic and they deserve a blog post of their own. José does a great job desribing how they work. The same goes for Railties and Rails Engines, they are definitely worthy of a blog post of their own.


# 
# Again most of the code is stolen from Jos&eacute; :)
#
module ScrambleHandler
  
  # Lookup the ERb handler
  def self.erb_handler
    @@erb_handler ||= ActionView::Template.registered_template_handler(:erb)
  end

  def self.call(template)
    # Call the erb handler, then call the new scrambler method.
    source = erb_handler.call(template)
    if template.formats.include?(:html)
      "Scrambler.from_html(begin;#{source};end).to_s"
    else
      "Scrambler.from_text(begin;#{source};end).to_s"
    end
  end

end

# Register our new handler, it handles index.html.scramble, show.html.scramble, ...
ActionView::Template.register_template_handler :scramble, ScrambleHandler

# This module scrambles text.
# Scrambler.from_text('Once upon a time a beautiful tapir came.')
# -> Ocne upon a time a baeuuitfl tpiar came.
module Scrambler
  def self.from_html html
    doc = Nokogiri::HTML(html) 
    doc.xpath('//text()').each do |node|
      node.content = from_text(node.content)
    end
    doc.to_html
  end

  def self.from_text text
    tokenize(text).map{|token| scramble(token)}.join('') 
  end

  def self.tokenize text
    return nil unless text
    text.scan(/(\p{Word}+|\W+|)/um).flatten
  end

  def self.scramble word
    return word if word =~ /\W/
    return word if word.size < 4
    arr = word.split(//)
    arr[0] + arr[1...-1].shuffle.join('') + arr[-1]
  end
end

Chapter 5, Publishing and subscribing to your applications events

In this chapter José uses the Notifications API to store notifcations in a MongoDB. He also shows how to use Rack middleware to configure what request to log. In order to configure the middleware he uses a Rails Engine. I'm just going to show how the Notifications API works here, and I'll let you read the rest in the book.

# Subscribe to all events and print them to the console
ActiveSupport::Notifications.subscribe do |*args|
  p ActiveSupport::Notifications::Event.new(*args)
end

# Instrument the rendering of Foo
ActiveSupport::Notifications.instrument(:render, :extra => :information) do
  render :text => "Foo"
end

Chapter 6, DRY controllers with Responders

In this chapter José goes through the implementation of responders. This implementation allows you to DRY up your controllers by factoring out common behavior.

def index
  @users = User.all
  respond_to do |format| 
    format.html # index.html.erb 
    format.xml { render :xml => @users }
  end
end

# becomes

def index 
  @users = User.all 
  respond_with(@users)
end


# and
def create 
  @user = User.new(params[:user])
  respond_to do |format| 
  if @user.save
    format.html { redirect_to(@user, :notice => 'User was successfully created.') }
    format.xml { render :xml => @user, :status => :created, :location => @user }
  else
    format.html { render :action => "new" } 
    format.xml { render :xml => @user.errors, :status => :unprocessable_entity }
  end 
end

# becomes
def create 
  @user = User.new(params[:user]) 
  flash[:notice] = 'User was successfully created.' if @user.save 
  respond_with(@user)
end

In this chapter we also learn how to replace the default generators with our own customized generators.

Chapter 7, Translatable app with I18n and Redis

In the final chapter José shows us how the I18N backed system works. He does this by creating a backend the uses Redis instead of the default YAML files. He also developes a simple Sinatra application that he hooks into the Rails routing with the mount method.

# Mount the Sinatra Translator::App at the /translator path.
mount Translator::App, :at => "/translator"

Rack applications normally get the full path inside ENV['PATH_INFO'], but when Rails mounts an application it removes the prefix before it sends it on /translator/en/pt/ becomes /en/pt/. The additional path is sent on in ENV['SCRIPT_NAME'] = '/translator'.

We also get a brief overview of Devise, the state-of-the-art authentication solution written by José himself.

Conclusion

One last thing, the book I have reviewed is a beta book, but the quality of it is higher than most published books I have seen. Congratulations to José and to the pragmatic bookshelf! There is not much more to say, get the book.

Java and Javascript in a Nutshell

2010-10-27T10:35:00.000+02:00

Java

Javascript

Choices, Habits and Awareness

2010-10-04T07:31:00.000+02:00

Every second, the human brain is bombarded with information, the conscious part of the brain is only able to handle a tiny percentage of this. The rest is handled by our unconscious.

A habit is an automatic reaction to a given situation. Habits allow us to do things automatically, without being consciously aware of what we are doing. But, to form a new habit takes time, and during that time we must choose to do what we want to make a habit of. And, to be able to make this choice we must be aware of the choice in the first place. We must be aware to be able to become unaware!

Here is an algorithm to illustrate this:

# Sequential
if not habit?(choice) or aware_of?(choice)
  make_decision(choice)
else
  automatic_response(choice)
end

Of course, it is not that simple. The brain is not sequential.

The brain is massively parallel and almost everything in it happens unconsciously. Every second our senses are fed with more than 11 million bits of information, while the conscious part of our brain is only able to process less than 50! All other information is filtered out by our unconscious. Scary! But, here comes the cool part, we can consciously change our filters by creating new habits.

Here is parallel version of the code above.

# Parallel, unconscious
if habit?(choice)
  automatic_response(choice)
end

# Parallel, conscious
if aware_of(choice)
  make_decision(choice) # Practice
end

Choices

Every single moment of our life, we are given choices. Some choices are obvious (the choices are, the answers may not be) such as:

Should I use Ruby or Python for my next project?
Do you take this woman to be your lawfully wedded wife?
Should I learn a new programming language that I have no hope, whatsoever, to use in a live project?
Are the kids allowed to play with fire?

Other choices are not so clear, such as:

Do I get up now or press snooze another time?
Should I step on the white lines when crossing the street?
Do I put the right foot in front of the left or jump on the left?
Should I remove the hand from the hot stove or not?
Should I react by being angry or not?

What's cool, and scary, with choices is that every time we make one, our brain changes. It literally changes physically every time. It actually changes every time we think a thought.

Every time we make a certain choice, that path through the brain gets wider, and thus the choice gets easier to make. It is easier to make the same decision, over and over, than it is to make new decisions all the time.

Another thing about choices is that it is often painful to make them. The reason for this, I believe, is that we often know, intuitively, what the right choice is, but that choice involves more work. This is why we should form good habits, because habits free us from the agony to choose.

Habits

When we have made a decision many times; when a path through the brain has become wide enough, we don't have to make a choice any more! We have formed a habit. Psychologists say that it takes 21 days. We have both good habits and bad ones. And most of the habits we have, we are unaware of.

A great thing about good habits is that they are performed by the unconscious part of the brain. They allow us to do things without consciously thinking about them. And since we are not consciously thinking about them, it is almost as if they do themselves. :D

A sad thing about bad habits is that they are performed by the unconscious part of the brain. We do things without consciously thinking about them. And since we are not consciously thinking about them, it is almost as if they do themselves. :(

That is why we should nurture our habits. We should create habits of things that we want to do, like writing our tests first, and complimenting people when they do good. We should make habits of boring things, that we have to do, like doing the dishes and writing documentation. Once they become habits they are not boring anymore.

This is also why we should be grateful when someone points out our bad habits, because they help us become aware of something that we may want to change.

Awareness

The key to forming new good habits and getting rid of bad ones, is awareness. We cannot change what we are not aware of. We cannot train what we are not aware of. But what are we aware of? We are aware of things we are trained to be aware of. Our awareness is learned, it is dependent on our habits! Ah, recursion!

When we are creating habits, we define ourselves.

Are you aware that there is a programming language designed for programmer happiness?

Are you aware that touch typing is a habit? That sitting in front of a computer eight hours a day without being able to touch type sucks? If you're not, read this blog post by blogger extraordinaire, Steve Yegge. I wasn't aware two years ago but now I type 50 WPM (which is not great, but I get better every day).

Are you aware of that your habits form your thoughts, that they form you?

We are what we repeatably do. --Aristotele

Conclusion

Can you imagine a day without googling for code-related things? When you don't have to think at all about the signatures of methods, the key combinations that moves you to the right position, or the inner workings of the authentication library. Can you imagine how good you would feel, and how fast you would work? Can you imagine eight hours of continuous flow?

Linus Torvalds, allegedly, wrote the bare bones of Git in two weeks. How could he do this? Because he has been living and breathing C and Linux, for the last 20 years!

So, where am I going with this? I want flow! I want eight hours of continuous flow while I'm working. In order to achieve this, I have to train my brain to know, really know, unconsciously know, the tools of my trade.

I am going to select a small number of tools that resonates with the way I am and want to work.

I have previously been a generalist when it comes to programming. I am proficient in many programming languages, but I have yet to become a true expert in one. With a true expert I mean someone who doesn't have to lookup anything. Everything, the language constructs, the built-in libraries, the major third-party libraries, the tools, should be intuitively known, habitual.

I aim to become an expert in the languages Javascript and Ruby.

This doesn't mean that I'll give up other programming languages. I will still dabble with interesting programming languages like Clojure, Scala, Mirah, etc., but my main focus will be on Ruby and Javascript.

Why Ruby and Javascript? Because they are languages that resonates with my brain. They are dynamic, interpreted and give lightning-fast feedback. They also allow me to express my concepts clearly and succinctly.

Why Ruby?

2010-09-29T18:47:00.000+02:00

Emancipate yourself from mental slavery, none but ourselves can free our minds. --Bob Marley, Redemption Song

There are a number of reasons to love Ruby and I will share some of them here. Ruby is influenced by some of the greatest languages ever invented: Perl, probably the most pragmatic language in the world, hell, even God used it :)

Ruby is also influenced by Smalltalk and Lisp. Both these languages are still living, vibrant languages despite their respectable age, 40 and 60 years respectively.

When Matz chose parts from the above programming languages his primary design goal was Programmer Happiness. He succeeded.

Here are some features that I love about the language.

Parallel Assignment

You can assign many values on the same line by separating them by commas. They are assigned in parallel. No need for temporary variables.

# Swap a and b, without a temporary variable
a, b = b, a

Multiple Return Values

If you want to return more than one value in Ruby all you need to do is to wrap it in an array. Then you can get the values by using commas. Also note, that return is not required in Ruby, the last evaluated expression is returned.

def div(a, b)
  [a/b, a%b]
end

a, b = div(5, 2)
# a = 2, b = 1

String Interpolation

Double-quoted strings can be interpolated.

kid = 'Kid'
"Hello #{kid} ##{1+3}"
# => Hello Kid #4

Here documents

You have seen them in other scripting languages and they are as good as they look. No need for String concatenation or embedded newlines (\n).

sql = <<-SQL
SELECT allocations.project_id as project_id, year, week, allocations.id as allocation_id
  FROM weeks
  LEFT OUTER JOIN allocations ON allocations.start_date < weeks.end_date
    AND allocations.end_date > weeks.start_date
    AND allocations.project_id = ?
  WHERE weeks.start_date >= ? AND weeks.end_date <= ?
SQL

Higher Order Functions

Higher order functions are easily expressed with Ruby blocks.

# Map converts an array of values to a new array of values
["a", "b", "c"].map { |item| item.upcase }
# => ["A", "B", "C"]

# Zip zips arrays together
[1, 2, 3].zip([10, 20, 30])
# => [[1, 10], [2, 20], [3, 30]]

# Zip with a function applies the function to the results
[1, 2, 3].zip([10, 20, 30]) {|arr| arr[0] + arr[1]}
# => [11, 22, 33]

Simple Command Line Access

The command line is always readily available in Ruby

# Backticks invokes the command and returns the result as a string
`ls`
# => "Scripts\nStylesheets\nbin\nget_destinations.scpt\nlib\nnibs\n"
`ls`.split
# => ["Scripts", "Stylesheets", "bin", "get_destinations.scpt", "lib", "nibs"]

# System invokes the command, prints the result to stdout, and return a boolean
system('ls -l')
total 64
drwxr-xr-x  3 andersjanmyr  admin    102 Aug 29  2009 Scripts
drwxr-xr-x  3 andersjanmyr  admin    102 Aug 29  2009 Stylesheets
drwxr-xr-x  5 andersjanmyr  admin    170 Aug 29  2009 bin
-rw-r--r--  1 andersjanmyr  admin  12312 Aug 29  2009 get_destinations.scpt
drwxr-xr-x  3 andersjanmyr  admin    102 Aug 29  2009 lib
drwxr-xr-x  3 andersjanmyr  admin    102 Aug 29  2009 nibs
# => true

system('ls tapir')
ls: tapir: No such file or directory
# => false

Open Classes

Open classes mean that it is possible to add new methods to already existing classes. This is very powerful.


class String
  def vowels
    self.scan(/[aeiouy]/i)
  end
end

"A tapir is beautiful".vowels
# => ["A", "a", "i", "i", "e", "a", "u", "i", "u"]

There are two common reactions to this.

Wow! This is really cool, this will allow me to put everything in the right place.
What! That is too dangerous, our programmers will create an unmaintainable system.

I'm in the first camp!

Class Inheritance

In Ruby the classes are objects and follow the inheritance rules of objects. No statics are needed.

class Mammal
  # This is a class method, that returns all mammals of this kind
  def self.all
    mammals.select do |m|
      m.kind_of? self
    end
  end

  # Another class method that gives access to the class varibable
  def self.mammals
    # @@ is a class variable prefix
    @@all_mammals ||= []
  end

end

class Tapir < Mammal
end

# Add some mammals and a tapir
Mammal.mammals << Mammal.new << Mammal.new << Tapir.new

Tapir.all
# => [#<Tapir:0x000001010fc1a0>]

Mammal.all
# => [#<Mammal:0x000001010fc218>, #<Mammal:0x000001010fc1c8>, #<Tapir:0x000001010fc1a0>]

Meta Programming

When the classes are parsed, Ruby is already running and since it is possible to create methods and classes on the fly you can do some very powerful meta-programming with Ruby. Here is a simple example.

class Tapir

  [:sniff, :eat].each do |method_name|
    send :define_method, method_name do |thing|
      "I'm #{method_name}ing #{thing}!"
    end
  end

end


t = Tapir.new
t.eat 'bananas'
I'm eating bananas!

t.sniff 'glue'
I'm sniffing glue!

This type of meta-programming is what enables the simplicity of Rails' has_many, and belongs_to methods for declaring relationships.

Method Missing

The last feature of Ruby that I want to share is method_missing. method_missing is a feature that Ruby inherited from Smalltalk. If an object is invoked with a method that is not defined by its class or ancestors, method_missing(method, args) is called instead. By implementing this method you can effectively create all sorts of cool stuff such as proxies, prototypes, etc.

# Simple implementation of a Javascript-like structure without inheritance
class Prototype

  def props
    @props ||= {}
  end

  def method_missing(method, *args)
    m = method.to_s.sub('=', '') # Strip the trailing '=' to allow setters
    if args.empty?
      props[m]
    else
      props[m] = args[0]
    end
  end
end

prot = Prototype.new
prot.age
# => nil
prot.age 14
# => 14
prot.age
# => 14
prot.age= 16
# => 16
prot.age
# => 16

Inside method_missing it is of course also possible to define new methods and classes dynamically, but I'll leave that for another day.

Ruby is a supreme dynamic language that lets you do almost anything. Try it!

When you can do anything, you have to become more responsible. You own your fate. And, that, I think, is a situation which promotes responsibility. -- Michael Feathers, author of Working Effectively with Legacy Code

Javascript, the Interlingua of the Web, Karlskrona

2010-09-22T09:45:00.001+02:00

Links and Books from my Javascript presentation in Karlskrona.

The Y Combinator

Books

Tools

Webkit CSS Transitions and Transformations

2010-09-18T08:00:00.002+02:00

This blog is best viewed in Safari or Chrome since it uses Webkit-specific functionality.

Transitions

transition, the process or a period of changing from one state or condition to another

Transitions are usually performed using jQuery or some other Javascript framework to animate the changes to a property. When using Webkit (Safari, Chrome, iPad, iPhone) Javascript is not necessary anymore.

In Webkit the transitions are described using four different properties. It is worth noting that the properties only define how the transitions will work when the values on the element is changed. They don't actually do anything until the values are changed!

/* This sets up a transition property on all divs on opacity.*/
div {
  -webkit-transition-property: opacity;
  -webkit-transition-duration: 2s;
}
/* The fade-in class, changes the opacity when set on an element.
  And thus triggers the transition defined above if the element is a div. */
.fade-in {
  opacity: 1;
}

// Adds the class 'fade-in' on all divs, via jQuery.
$("div").each(function() {
  $(this).addClass('fade-in');
});

`-webkit-transition-property`

This is the property that will be affected by the transition, this can be any valid property, and it defaults to all. What this means is that a change to any attribute on an element will be affected by the transition and occur according to the specification.

/* only width should be affected */
-webkit-transition-property: width;

/* multiple comma-separated properties are supported */
-webkit-transition-property: opacity, width, height;

/* all properties should be affected, this is the default */
-webkit-transition-property: all;

`-webkit-transition-duration`

How long the transition should take? Valid units are s and ms.

/* Set the duration to 400 milliseconds */
-webkit-transition-duration: 400ms;

/* Multiple values are supported here too, in combination with
multiple -webkit-transition-property's */
-webkit-transition-property: opacity, width, height;
-webkit-transition-duration: 400ms, 1s, 3s;

`-webkit-transition-delay`

How long should we wait before the transition starts. Valid units are s and ms.

/* Set the delay to 1 second */
-webkit-transition-delay: 1s;

/* Multiple values are supported here too, in combination with
multiple -webkit-transition-property's */
-webkit-transition-property: width, height;
-webkit-transition-delay: 1s, 3s;

`-webkit-transition-timing-function`

The timing function is a function that changes the speed of the transition over its duration. It is described with a cubic-bezier curve, but it has some predefined values, ease (default), ease-in, ease-out, ease-in-out, and linear. In none of those values are to your liking you can define your own with the cubic-bezier function.

/* Transition with the default timing*/
-webkit-transition-timing-function: ease;
/* Multiple values are supported here too, in combination with
multiple -webkit-transition-property's */
-webkit-transition-property: opacity, -webkit-transform;
-webkit-transition-timing-function: ease-out, cubic-bezier(0.5, 0.2, 0.3, 1.0);

Thats it for transitions. But wait, what was that -webkit-transform in the end?

Transformations

transformation, a thorough or dramatic change in form or appearance

Webkit supports some really cool transformations, that allow you to animate the pages in remarkable ways.

`rotate()`

<div style = "width: 12em;
  margin-top: 5em;
  -webkit-transform: rotate(45deg)">
I am rotated!
</div>

I am rotated!

Rotate accepts the units deg, rad, and grad

It also comes in 3D forms, rotateX, rotateY, rotateZ, and rotate3d.

`scale()`

<div style = "width: 12em;
  margin: 2em 0 2em 18em;
  -webkit-transform: scale(4)">
I am Scaled!
</div>

I am Scaled!

Scale can take an additional parameter scale(x, y) If you want to scale un-proportionally. It also comes in 3D versions, scaleX, scaleY, scaleZ, and scale3d.

`translate()`

<div style = "width: 12em;
  margin: 2em;
  -webkit-transform: translate(4em, -2em)">
I am Translated!
</div>

I am Translated!

Translate takes one or two parameters and moves the element to the new location. It, too, comes in 3D forms, translateX, translateY, translateZ, and translate3d.

Additional Transforms

Apart from the translations above, Webkit also supports skew, perspective, and matrix. Read more in the Safari Reference Library

I'll finish up with a composite transform combined with transitions, best viewed with Safari. This transformations are initiated by changing the style from Javascript.

<div style="width: 4em; height: 4em;
  margin: 2em; background-color: blue;
  color: white;
  -webkit-transition-duration: 5s;
  -webkit-transition-function: ease-out;"
  onclick='style.webkitTransform="translate(16em, -16em) scale(6) rotate(720deg)"; style.backgroundColor="red";'> Click Me!
</div>

Click Me!

Using Git with Subversion

2010-09-02T09:42:00.000+02:00

I had the unfortunate experience of having to use Subversion again after using Git for a long time. It is amazing how fast I can forget. After renaming a directory at the prompt, and the agony that goes with it, I decided to switch back to Git.

$ mv requester sampler
# svn agony after renaming a directory
$ svn st
?       sampler
!       requester
!       requester.rb
A       sampler.rb

The tool to use when using Git with Subversion is, of course, git svn.

git svn works very well as long as you remember that Subversion is not Git. It does not handle merging well, and it will bite you if you don't respect that. So what does this actually mean? It means:

Always keep the Git master in sync with Subversion

To do this you have two commands you can use.

# Rebases the commits from the upstream Subversion server with your local master.
$ git svn rebase

You should only git svn rebase in your Git master, and you should ALWAYS do it before you git svn dcommit anything to the subversion repository. git svn rebase keeps the upstream subversion in sync with your local master by pulling down the changes, kind of like svn update.

# Commits the changes you have in your local master to the upstream Subversion server.
$ git svn dcommit

When you have changes ready to commit, you commit them to subversion with git svn dcommit. You should ALWAYS git svn rebase before you do the update, or it will fail.

That's it! As long as you follow these two simple rules, your life with git svn will be easy. If you forget to follow them, you will be bitten. When you get bitten, the cool thing about Git is that even if you screw up, it is always possible to sort it out.

It that was all there was to it, there would be no reason to use Git instead of Subversion. Git really shines when it comes to branching and merging. You may create as many local branches as you like with git branch branch_name or git checkout -b branch_name. You can hack around in these local branches as much as you want and merge them together. But, before you merge them into the master branch, you must rebase with master! Not merge, rebase! Rebase means replay the commits on top of the named branch. It creates new commits, the same content, but with a different SHA-1.

# Example session

(master)$ git svn rebase
(master)$ git checkout -b dev
hack, hack, hack, ...
(dev)$ git commit -am 'Commit the changes' 

(dev)$ git checkout master
(master)$ git checkout -b bugfix
hack, hack, hack, ..., done

(bugfix)$ git checkout master
(master)$ git svn rebase
(master)$ git checkout bugfix
$ git rebase master
(bugfix)$ git checkout master
(master)$ git merge --ff bugfix # --ff only fast-forwards, merges that don't need to merge. 
(master)$ git svn dcommit
(master)$ git branch -D bugfix # delete the branch it is not needed anymore

(master)$ git checkout dev
hack, hack, hack, ..., done

(dev)$ git checkout master
(master)$ git svn rebase
(master)$ git checkout dev
(dev)$ git rebase master
(dev)$ git checkout master
(master)$ git merge --ff dev # --ff only fast-forwards, merges that don't need to merge.
(master)$ git svn dcommit

Another thing to be aware of is that git svn dcommit creates an extra commit, so even if you haven't changed anything in the master you need to rebase the local branch with the master. This is only needed if you don't delete the branches after you are done with a commit.

In the example above, I ended with a git svn dcommit and I didn't remove the dev branch.

(master)$ git svn dcommit # from above
(master)$ git co dev
(dev)$ git rebase master # rebases the extra commit created by git svn dcommit

If you forget to rebase or something else happens that hinders a clean merge into the master. You can always back out of it with git reset --hard.

(master)$ git svn dcommit
... failed miserably, because I failed to git svn rebase, bollocks!
(aa..88dd|MERGING)$ git reset --hard
(master)$ git svn rebase
(master)$ git svn dcommit # Nice and clean commit

To get started you need to clone a subversion repository.

$ git svn clone http://svn.example.com/project/trunk
$ cd trunk
(master)$ git ...

Now, is a good time to start using Git. Get yourself anything by Scott Chacon, such as the book or the screencasts.

The Ruby Toolbox

2010-08-26T15:09:00.000+02:00

One of the great things about Ruby is the community. When something is not working out as well as it could, someone figures out a way to improve it.

It used to be difficult to select what gems, Ruby libraries, to use when solving a task that I was not familiar with. This is not the case anymore. Christoph Olszowka decided that he wanted to solve this problem and created The Ruby Toolbox.

The Ruby Toolbox, Know your options!

Ruby developers can choose from a variety of tools to get their job done.

The Ruby Toolbox gives you an overview of these tools, sorted in categories and rated by the amount of watchers and forks in the corresponding source code repository on GitHub so you can find out easily what options you have and which are the most common ones in the Ruby community.

This is surprisingly powerful. In my latest project I used the following libraries (among others).

Rails

Rails need no introduction, it is one of the best frameworks for creating dynamic web applications. With best, I mean most productive and fun to work with!

Haml

Haml (HTML Abstraction Markup Language) is a layer on top of XHTML or XML that's designed to express the structure of XHTML or XML documents in a non-repetitive, elegant, easy way!

Typhoeus

Typhoeus is a http client library, a thick wrapper around curl, just like curl it works like a charm.

Capistrano

Ahh, Capistrano, deployment scripting done right! The simplicity of the structure is worth showing since it should be copied by other lesser tools.

# Create the basic structure
$ cap deploy:setup
mkdir -p /var/rails/rsvc 
/var/rails/rsvc/releases 
/var/rails/rsvc/shared 
...

# Deploy from the repo into a timestamped release directory and symlink current
$ cap deploy
svn checkout -q  -r5454 https://svn.neo4j.org/qa/matrix-framework/rsvc/trunk/rsvc /var/rails/rsvc/releases/20100826122450 
echo 5454 > /var/rails/rsvc/releases/20100826122450/REVISION
....
rm -f /var/rails/rsvc/current && ln -s /var/rails/rsvc/releases/20100826122450 /var/rails/rsvc/current

It's so simple and powerful, that it makes me cry.

God

God, a wonderfully named, server monitoring tool written in Ruby. God aims to be the simplest, most powerful monitoring application available.

RSpec

RSpec, only code can do it justice.

describe Request do
  
  before do
    @sample = Sample.create(:name => 'my_sample', :method => "post")
  end
    
  context '#old' do
    it "should not return any" do
      Request.old.should be_empty
    end
    ...
  end

  context '#delete_old_non_failing' do
  
    it "should delete old non failing requests" do      
      create_old_request :response_status_code => 0
      create_old_request :response_status_code => 200
      expect {
        Request.delete_old_non_failed
      }.to change { Request.count }.by(-2)
    end
    ...
  end
  
  ...
end

Does your language have a Toolbox? Perhaps it is time to create one, or to switch to another language?

Good Practices for Rich Web Applications

2010-08-16T07:50:00.000+02:00

Use jQuery

jQuery is the best thing that has happened to Javascript since it got first class functions in version 1.2. The library is elegant, powerful and has exactly the right level of abstraction for working with the DOM. There is nothing more to say. Learn it and use it. Good resources are: the jQuery API, my view of the jQuery API

Learn Javascript

Javascript is the programming language of the web. Learn it! Javascript is different from most other programming languages. It is dynamic, it has prototypical inheritance, and works more like Scheme than any of the languages that you are probably used to. If you want to learn Javascript you should get the following books, The Little Schemer, The Seasoned Schemer, Javascript, the Good Parts, and possibly High Performance Javascript

Learn CSS

Many programmers think that CSS is the language of designers and not programmers. This is not the case at all. If you are lucky enough to have a designer on your team (most people don't), CSS is the language with which you communicate. It is the interface between designers and programmers and as a programmer you should know it better than the designers. By knowing CSS well you will reduce the misunderstandings between you and your designer.

Unfortunately, many designers don't care about how code looks, as long as the design looks good on the surface. It will be up to you to make sure that the CSS doesn't get out out of hand. It will also be up to you to keep the HTML clean, and a good way to do this is to use semantic HTML, combined with CSS. You have no idea what the designers can come up with.

<!-- 
  Old School rounded corners, invented by a GOOD designer. 
  All this code was actually needed to achieve the purpose.
  -->
<style>
.t {background: url(dot.gif) 0 0 repeat-x; width: 20em}
.b {background: url(dot.gif) 0 100% repeat-x}
.l {background: url(dot.gif) 0 0 repeat-y}
.r {background: url(dot.gif) 100% 0 repeat-y}
.bl {background: url(bl.gif) 0 100% no-repeat}
.br {background: url(br.gif) 100% 100% no-repeat}
.tl {background: url(tl.gif) 0 0 no-repeat}
.tr {background: url(tr.gif) 100% 0 no-repeat; padding:10px}
</style>
<div class="t">
  <div class="b">
    <div class="l">
      <div class="r">
        <div class="bl">
          <div class="br">
            <div class="tl">
              <div class="tr">
                Lorem ipsum dolor sit amet consectetur adipisicing elit
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

As an additional benefit you will become better at jQuery. Not only is CSS the query language of the browsers it is the query language of jQuery. Jariba!

Bulletproof Web Design is a good book web design, including CSS.

Decide how "Rich" your application should be

How rich should your application be? The scale varies from no Javascript to only Javascript, but you will probably want to land somewhere in between. Here are a few suggestions.

No Javascript, everything is server generated.
Slightly enhanced pages, simple validations, but no Ajax.
Ajax enhanced pages, but every page still reloads frequently.
Single page per area, entire area is handled by Javascript.
Only Javascript, Ajax interaction with the server
Only Javascript, no interaction with the server.

The important thing is to make a decision. If you don't make the decision, everyone will do different things on different parts of the application and you will loose consistency. In GUI, consistency is king. Make a decision and move on, you can always change your decision later.

Organize your code

Javascript

Make sure that all your Javascript code is namespaced properly. It is impolite to pollute the global namespace and it will bite you in the end. A simple variable declaration will do.

// Common namespace for your entire application
// This declaration lets you split your code of multiple files.
// If MyNamespace is defined use it, otherwise declare it.
MyNamespace = MyNamespace || {};

But, of course, it is also possible to get fancy and encapsulate the functions that you don't want to expose, if that is your cup of tea.

MyNamespace = MyNamespace || {};

MyNamespace.Tournament = function() {
 // Private stuff
 var tournamentCount = 0;
 function addTournament(tournament) {
  tournamentCount++;
 }
 
 return {
  //public stuff
  numberOfTournaments: function() {
   return tournamentCount;
  }
 }
}();

You should also separate your Javascript code into different files. The namespace idiom above helps to have the same namespace across multiple files. The same principles as with other type of code is valid with Javascript, organize the code by area, when it changes, where it is used, etc. Don't be afraid of the additional load time, splitting the files will give you. The files can easily be concatenated with tools like Rake, SCons, Ant or even a simple:

$ cat file1.js file2.js file3.js > all.js

They can also be compressed with JSMin or YUI Compressor.

Optimize your environment for development, not for production!

HTML

HTML is code! Divide your pages into partials by responsibilities. It allows you to keep your pages DRY and readable. The Single Responsibility Principle applies to HTML too.

Make sure you keep the Javascript with the code that it manipulates. If you, for example, have a calendar partial that uses jQuery DatePicker, you have to make sure that the partial includes all the necessary Javascript to configure the calendar. Don't keep Javascript code in the page away from the partial. Things that change together should be together.

CSS

Stylesheets are code too. They should also be split into areas that allow you to easily find and navigate them. Use Sass or SCCS to keep your CSS files DRY. Sass is good for designers to. It gives them the ability to use variable names, mixins, etc. and simplifies their usage of semantic names such as notice, and sidebar instead of yellow and left.

Optimize your environment for development, not for production!

Separate your Javascript from your HTML

All too often I see generated HTML pages with Javascript code in them. Don't do it. Keep the HTML free from Javascript.

<!-- DON'T DO THIS!  -->
<button id='update-button' onclick="MyNamespace.updateList();">Update List</button>

// In the Javascript file for the page.
MyNamespace.updateList = function() {...}


<!-- DO THIS! -->
<button id='update-button'>Update List</button>

// In the Javascript file for the page.
MyNamespace.updateList = function() {...}

$(function() {
  $("#update-button").click(function() {
    MyNamespace.updateList();
  });
});

It may seem like there is a lot more code in the good example, but notice the symmetry. The code that attaches the listener is in the same file as the code that uses the listener. This is good. Symmetry is good.

Use clone()

Separating the HTML and the Javascript goes both ways. Don't generate HTML code in Javascript. It doesn't matter that it is super simple to do it using jQuery.html(). Keep them separate, use jQuery.clone() instead.

// DON'T DO THIS
$("<li data-id='123'>My new item</li>").appendTo("ul");
// OR THIS
$("ul").append("<li data-id='123'>My new item</li>");

<!-- DO THIS -->
<ul>
<li id="list-template" class="template">All .template are hidden (display: none) in the CSS</li>
</ul>

// AND THIS
var $clone = $("#list-template").clone();
$clone.attr("data-id", "123").text("My new item").removeClass("template");
$("ul").append($clone);

The point of this is, again, to keep the HTML separate from the Javascript.

Decide how content flows between, the page, Javascript and the Server.

Once you have decided how rich your application should be, you have to decide on a method for moving the data between the HTML Page, Javascript and the Server. My preferred choice is to have every page that is served from the server include a context object with all the static data for the page and to have additional data that belongs to parts of the page be sent as data-attributes on the elements concerned.

The context object will contain all the data that is commonly needed in the page.

// Sample context object that is generated with the page.
MyNamespace = MyNamespace || {};
MyNamespace.Context = {
  user: {
    id: "28",
    name: "Anders Janmyr"
  },
  tournament: {
    id: "78344",
    name: "Fifa World Cup"
  }
};

I use the context object(s) to keep state on the client to. If it is important that the page looks exactly the same, even if the user reloads the page, I make sure that the state I stick into my context object is synched back to the server. This can easily be done, asynchronously, with Ajax and does not affect performance noticeable.

Elements specific data is sent along with the element it defines.

<!-- Element specific data attached to the elements with data-attributes -->
<ul id="tournament-menu">
  <li data-id="78344" data-participant-count="64">Fifa World Cup</li>
  <li data-id="666"  data-participant-count="44">Americas Cup</li>
  <li data-id="1464" data-participant-count="32">Rugby World Cup</li>
</ul>

The same argument as with the context object applies, as soon as I change an element in the GUI i need to feed that information back to the server. With elements I usually send the information to the server before updating the GUI, since element specific data is usually permanent data and not just session data.

An alternative solution to the context object above is to use the body element as the data-container, like this:

<body data-user-id="28" data-user-name="Anders Janmyr" data-tournament-id="78344" data-tournament-name="Fifa World Cup">

I tend to use the context object because I find it easier to add functionality to it than to the DOM element.

Only send the data that is needed to the client with the page. The rest of the data should be loaded on demand, with Ajax. Both JSON data and HTML templates can be loaded on demand. There is no need to deliver the entire page at once. Experiment and do what gives the best user experience.

Use file watchers to speed up feedback

If you compare the feedback cycle of Javascript and HTML development to Java and C# development, you are probably very happy with the short, tight feedback loop. This doesn't mean that you should be content. A feedback cycle cannot be too short.

xrefresh for Firefox and IE, and LiveReload for Safari and Chrome, are a couple of tools that will tighten your feedback loop even more.

Both tools are file watchers that listen to changes for files and refresh the browser when they change. If you combine this with two screens, you will have one screen for the browser, that updates continuously, while you edit your code on the other screen. Fabulous!

Conclusion

Rich web applications are very close to the traditional client-server model. We have to keep state on the client side to give the user a good experience. At the same time the application state, and indeed the entire application, can be swept away by a click of a button or a page reload.

This puts new demands on us as developers. We have to realize that we are responsible for the entire application, not just the business logic, but the HTML and CSS too. More that anything, we have to realize that Javascript is a first class programming language with its own programming techniques, which we need to master to be able to develop good web applications.

No Deadlines

2010-08-12T07:12:00.000+02:00

deadline, from New Oxford American Dictionary

the latest time or date by which something should be completed : the deadline for submissions is February 5th.

historical a line drawn around a prison beyond which prisoners were liable to be shot.

Deadlines in software development induce unnecessary stress, and transform otherwise enjoyable activities into chores.

We need to replace deadlines with livelines. A liveline is a deployed, working version of a product. It is after the fact, a milestone set when you are done.

How Deadlines Are Set

Deadlines are set in different ways in different projects.

In the classic Death March project, a deadline is set by someone outside the project with no clue whatsoever about how long it will take. Typically this kind of deadline is based on some external event, where the project will be demonstrated. The event may be a conference, or something similar.

In Scrum, the deadlines are set by the process itself. Most Scrum projects I have worked with use a two week deadline instead of the recommended 30 days. The team decides how many stories they can complete by that deadline, based on rough estimations done in the sprint planning.

Some projects set deadlines by a developer being asked (or asking himself) when the task will be done. The developer will say: "I'll have it done by 3 o'clock", a wild guess.

The problem with all these approaches is that the deadlines are, at best, based on guesses of the team doing the development. But guesses are guesses, not something you can set a deadline by.

And deadlines work both ways, a task is very rarely completed before it's deadline.

Kinds of Deadlines

Real Deadlines

If a task is not completed by a certain time someone will die. This kind of task is really common in novels and movies. A clock is ticking and if the task is not completed by the time the clock reaches zero, BOOM!

If have never seen this kind of deadline in software development, but they may exist.

Planning Deadlines

Some deadlines are set in order to simplify planning. In the large Death March projects, the plan is to show the project at a certain date. In integration projects, the deadlines are set to be able to synchronize with other teams working on the same product.

This kind of deadlines are actually understandable, but I am arguing that they actually slow down projects by introducing stress, which introduces bugs. The bugs introduce bugs in the dependent projects, and slows down many projects instead of just one.

Motivational Deadlines

Motivation deadlines are set to get people to work. The assumption is that if a deadline is not set people will procrastinate over different ideas, or just slack off since there is no date set for the release anyway.

This is, absolutely, the worst kind of deadline. If the team is not motivated enough to do a good job without a deadline, you need to get a new team, not set a deadline.

Why Deadlines Are Bad

Stress

Consider walking on a one foot wide plank. If it is on the ground it is easy. If it is two meters up it is difficult, but if you put it at twenty meters it is a whole different ballgame. That is what its like with deadlines, they put an unnecessary pressure on people.

Loss of Motivation

Work consist of whatever a body is obliged to do, and play consists of whatever one is not obliged to do. -- Mark Twain

In his book, Drive, Dan Pink differentiates between intrinsic motivation and extrinsic motivation.

Extrinsic motivation is motivation driven by carrots and sticks, rewards and punishment. It works well for simple, well-defined tasks.

Intrinsic motivation works well for creative, ill-defined tasks, like programming.

Intrinsic motivation is fragile, it disappears in the presence of carrots and sticks. A deadline is a stick, whether set by ourselves or someone else.

Livelines

A liveline is, quite simply, a task that is done. When a task is done, it is easy for others to use it, in their plans or in reality. If we want to build a stable house, we should build it on stable ground.

When using livelines the iteration length is the same length as a task. This is usually a lot shorter than the usual one to four weeks common in Scrum and other agile methods. It also leads to variations in the iteration length.

If you implement your task all the way into production, you get Continuous Deployment or Continuous Delivery. This puts a lot of pressure on your tooling and on automation but, I believe this is the way to go.

It is 2010, it is time to get rid of deadlines.

Ruby and Rails Summer Reading

2010-06-20T08:22:00.000+02:00

If you ask a Ruby programmer why he is using Ruby, you will probably get several answers like: It is dynamic. I allows me too keep my code DRY. I get results faster. Aside from all these statements, there is one statement that almost always comes up.

I use Ruby because it makes me happy!

If you search Amazon for Ruby you get 8,829 Results and if you search for Rails you get 14,113 Results. What books should you get?

Here are the books I recommend:

Ruby

Beginning Ruby is a superb book for people coming new to Ruby.

If you are not a complete novice, you should instead go for The Well Grounded Rybyist.

When you have finished the Well Grounded Rubyist, you are ready for Metaprogramming Ruby where you learn metaprogramming techniques to help you keep your code DRY and readable.

Rails

Since Rails is currently being updated to version 3, there are not that many books available. I can recommend The Rails3 Way, currently only available as a Rough-Cut.

The lack of updated books is not a problem. The The Ruby on Rails Guides are excellent. Make sure you select the edge version.

Even more information is available through the videos from Railsconf, subscribe to them in iTunes. The first videos where published, the day after they were filmed. Fantastic!

Have a happy Ruby-red summer.

Seamless Web Development

2010-06-08T21:34:00.000+02:00

Do you remember the time before 9/11 when you could arrive to the airport 10 minutes before the plane's departure and just walk on to the plane. Now, you have to arrive at least an hour before the plane departs and you have to strip to get through a security.

The security is just for show. I don't know how many times I have forgotten a bottle in my carry-on and I haven't noticed it until I unpacked the bag at home. There a number of ways to get a big bang for your bucks, that aren't checked at security

So what does this have to do with web development? Web development in a compiled language is like traveling after the terrorist attacks. You have to go through a lot of time consuming controls that are, mostly, for show. This dramatically slows down your development speed.

Since I did my comparison between Rails and .NET MVC I have done a lot of coding in both environments, and my conclusion is that Rails is a lot faster to develop with than .NET MVC.

The reason for this is that .NET MVC puts up a lot of seams that hinders development.

Seam #1, Runtime Development

When I'm using Rails for development I very rarely restart the server. When I make a change to the view, controller, model or database, the changes are visible on the next request. Immediate feedback!

In Visual Studio, there is something called "Edit and Continue", that you can set to allow you to make changes to the code while you are debugging. Mooah ha ha ha ha ha ha ha haaaaaaa! This sucks so bad that I cannot help bursting out in a crazy laughter. The flow disappears.

If I make changes to the view I can continue, unless I make changes to the I18N resources, because these are obviously not mutable. WTF!

Sometimes it is actually possible to make changes to the code, but when you try to save the changes "Edit and Continue"'s big brother appears and smacks you in the head, and the flow is gone again!

There is really no good reason for this. I'm sure there are a number of technical reasons having to do with type safety and class reloading, but I don't care. Just reload everything, do it fast and unnoticeable!

Seam #2, The Javascript Impedance Mismatch

Hopefully everyone has realized that Javascript is the hottest thing in web development! A language that has been spat on, and laughed at, has become the most wide spread programming language in the world. It's available in every browser and on every programming platform.

I have been talking about the greatness of Javascript for a few years now and it is great to see when someone finally get it.

"Its great, I just change this little thing, then click reload, and it works immediately!" -- A collegue, seeing the coolness of Javascript.

Javascript is Lisp in C-clothing and Lisp is the language that God used to create the world.

OK, Javascript is cool, and we all use it, but what is this impedance mumbo jumbo? Since a large part of a modern web application will be developed in Javascript, when we move over to the server side and everything is static and compiled, we are in for a lot of frustration.

Common Development Flow

Make a change to the GUI, in Javascript.
Reload
Make another change
Reload
Realizing that you need to change the controller for the next change.
Aaaaaaaaaarrrrrrrrrrrggggghhhhhhhh, I have to restart the application.
What was it that I was working on again? I'll go for a cup of coffee!

This is the Javascript impedance mismatch. If you are developing in a dynamic language you expect it to work like this all the time. You get used to it. It is better!

With Rails and Ruby, you get the same flow on the backend as you do on the client. The static-dynamic mismatch is gone. (There is still a language mismatch, between Javascript and Ruby, but it is minuscule by comparison.)

Seam #3, Cruft

Compare the following two partials

<!-- BookedCustomer.aspx partial in .NET MVC-->
<%@ Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<Bergqvists.Models.Booking>" %>

<div class="view">
    <div class="customer title"><%= Html.ActionLink(Model.Customer.Fullname, "Details", "Customers", new { Id = Model.CustomerID}, null) %></div >
    <div class="status"><%= Html.Encode(Model.BookingStatus.Name) %></div >
    <div class="booking-date"><%=ViewRes.BookingStrings.BookedOn %>
    <%= string.Format("{0:D}", Model.BookingDate) %></div>

</div>
<% Html.RenderPartial("BookingStatusLinks", Model);%>

<!-- _booked_customer.html.erb partial -->
<div class="view">
    <div class="customer title"><%= link_to customer.fullname, customer_path(customer)</div >
    <div class="status"><%= customer.booking_status.name %></div >
    <div class="booking-date"><%=t :booked_on %>
    <%= customer.booking_date, :format => :short %> %></div>

</div>
<%= render :booking_status_links, :locals => {:customer => customer} %>

As you can see the corresponding ERB is not much different, but at least everything in the code is meaningful. And what customer value does the first line bring?

<%@ Control Language="C#" Inherits="System.Web.Mvc.ViewUserControl<Bergqvists.Models.Booking>" %>

Nada! Zilch! It is only there to please the compiler. Cruft!

Now, most Ruby programmers I know use HAML instead of ERB and this cleans the view up considerably.

.view
  .customer.title= link_to customer.fullname, customer_path(customer)
  .status= customer.booking_status.name
  .booking-date
    = t :booked_on
    = customer.booking_date, :format => :short

= render :booking_status_links, :locals => {:customer => customer}

Beautiful!

Comparing Haml with ASP is not really fair, since it is possible to use NHaml instead of ASP in .NET MVC, but no one I know does that, maybe I hang with the wrong .Net people. NHAML is great, please switch it to the default for .NET MVC.

The example above is in the view layer. But the cruft spreads all over the stack.

Seam #4, Testing

Testing is always an interesting topic. The number one reason testing is not done is that it is to cumbersome. In .NET MVC I find it painful to test the controller and view layers, to the extent that I don't do it. With Rails on the other hand, you are setup for test from the start.

I always try to put as much code as possible into the model layer, but some code belongs in the controller. I want it to be easily tested too.

It is also worth noting that the Ruby community is driving the testing profession forward, with tools like RSpec, Capybara, Webkit and Cucumber.

Yes, I know that there are several implementation languages for Cucumber. It's possible to write your steps, in C# or Java or almost any other language. My advice is, don't do it! There is compilation, waiting, pain and suffering down that road. Use a dynamic language to test your code.

Conclusion

So what's my point? It's time to stop compiling and quit slacking off.

Rails should be the default choice for web development. You should have a very good reason for not choosing Rails on your next web project.

I know it may feel tough to have invested many years in something else, but as the turks say.

No matter how far down the wrong path you've gone, turn back! --- Turkish Proverb

Don't take my word for it. Check out this interview and this presentation

>Making Windows Keyboard Feel like OS X

2010-05-20T15:51:00.000+02:00

I have been unfortunate enough to have to start working with Windows on my Mac. I am using Bootcamp and VMWare Fusion. If I only need to make minor changes I can boot the Bootcamp partition via VMWare so I don’t have to reboot the computer every time I have to use Windows. VMWare allows me to customize my mappings to send the keys that I am used to using, so if this was the only way I was using Windows, that would be quite alright.

Since I need to develop in Visual Studio, using VMWare is only an option for very small projects, or the whole experience will become painfully slow.

Bootcamp does not support remapping the keys. Agony! Well, I found a solution.

Remapping Keys

I am using a US keyboard layout, your key mappings may be different.

SharpKeys is a tool that helps you perform some registry hacks, to allow me to remap the keys that I want to change.

I use it to remap the Alt and Windows keys, since they are reversed on the Mac. I also move the “`~” down to the bottom left corner where “|" usually are on Windows. “|" are in two different places on a US layout, I don’t know why.

Now that I have finished remapping the keys, I am done using SharpKeys. Now all I have to do is to get the Windows key to behave the way they do on the Mac on not like Windows keys.

Setting up the Shortcuts

The first thing to do is to disable the normal WindowKey functions. Here is a guide.

AutoHotKey is a tool the lets you remap the functions that are sent when a key is pressed. It can do a whole lot of other cool stuff too, so I encourage you to check it out even if you are not using a Mac. It is a great automation tool.

Here is my shortcuts.ahk file. I remap all my CMD-actions to the WinKey, and I also remap the Win- Left, Right, Up and Down to the corresponding Mac shortcuts.

#SingleInstance force
#a::Send ^a
#b::Send ^b
#c::Send ^c
#f::Send ^f
#n::Send ^n
#o::Send ^o
#p::Send ^p
#q::Send !{f4}
#r::Send ^r
#s::Send ^s
#t::Send ^t
#v::Send ^v
#w::Send ^{f4}
#x::Send ^x
#y::Send ^y
#z::Send ^z

#Left::Send {Home}
#Right::Send {End}
#Up::Send {PgUp}
#Down::Send {PgDn}

After this is done it is OK, almost, to start working. The sad part is that this configuration only works with an external keyboard. I couldn’t get the internal keys mapped properly on my Macbook Pro. But at least it is working.

The Tapir's Tale

Finding with Git

Where is my file?

In what files does word exist?

Who deleted my file?

When was a file added?

Who changed that line?

Who deleted that line?

What commits contain the string?

7 Reasons to go to Øredev 2011

Yehuda Katz

Felix Geisendörfer

Corey Haines

Ilya Grigorik

Trevor Burnham

Charles Nutter

Simon Peyton Jones

Summary

jQuery Changes From 1.4.2 to 1.6

Selected changes from 1.4.2+

delegate()

jQuery.now(), jQuery.type() and jQuery.parseXML()

Deferred()

Tips from Rails Anti-Patterns

delegate can take a :prefix argument

Transaction Scope

Association Methods

When to make a model active

Haml []

RESTful actions

Rake Tasks

Database Index

Conclusion

Ruby, an Exceptional Language

Tag Module

A Nested Exception Class

Throw – Catch

Wrap up

A Not Very Short Introduction To Node.js

History

Get Started

Networking

Modules

An example

Node Package Manager, npm

Noteworthy Modules

Templating Engines

Testing

Deployment

Joyent (no.de)

Nodester

Cloud Foundry

Tools

Reloaders

Debuggers

Idioms

Return on Callbacks

Exceptions in Callbacks

Parallel Execution

Sequential Execution

Library Support for Asynchronous Programming

Fibers

The async Library

Conclusion

Mastery

Instruction

Practice

Surrender

Intentionality

The Edge

Conclusion

What Is New in EcmaScript 5?

New Syntax

Library Additions

JSON

Date

Object

Array

String

Function

`delegate()`

`jQuery.now()`, `jQuery.type()` and `jQuery.parseXML()`

`Deferred()`

`delegate` can take a `:prefix` argument

The `async` Library