Thursday, August 26, 2010

The Ruby Toolbox

One of the great things about Ruby is the community. When something is not working out as well as it could, someone figures out a way to improve it.

It used to be difficult to select what gems, Ruby libraries, to use when solving a task that I was not familiar with. This is not the case anymore. Christoph Olszowka decided that he wanted to solve this problem and created The Ruby Toolbox.

The Ruby Toolbox, Know your options!

Ruby developers can choose from a variety of tools to get their job done.

The Ruby Toolbox gives you an overview of these tools, sorted in categories and rated by the amount of watchers and forks in the corresponding source code repository on GitHub so you can find out easily what options you have and which are the most common ones in the Ruby community.

This is surprisingly powerful. In my latest project I used the following libraries (among others).

Rails

Rails need no introduction, it is one of the best frameworks for creating dynamic web applications. With best, I mean most productive and fun to work with!

Haml

Haml (HTML Abstraction Markup Language) is a layer on top of XHTML or XML that's designed to express the structure of XHTML or XML documents in a non-repetitive, elegant, easy way!

Typhoeus

Typhoeus is a http client library, a thick wrapper around curl, just like curl it works like a charm.

Capistrano

Ahh, Capistrano, deployment scripting done right! The simplicity of the structure is worth showing since it should be copied by other lesser tools.

# Create the basic structure
$ cap deploy:setup
mkdir -p /var/rails/rsvc 
/var/rails/rsvc/releases 
/var/rails/rsvc/shared 
...

# Deploy from the repo into a timestamped release directory and symlink current
$ cap deploy
svn checkout -q  -r5454 https://svn.neo4j.org/qa/matrix-framework/rsvc/trunk/rsvc /var/rails/rsvc/releases/20100826122450 
echo 5454 > /var/rails/rsvc/releases/20100826122450/REVISION
....
rm -f /var/rails/rsvc/current && ln -s /var/rails/rsvc/releases/20100826122450 /var/rails/rsvc/current

It's so simple and powerful, that it makes me cry.

God

God, a wonderfully named, server monitoring tool written in Ruby. God aims to be the simplest, most powerful monitoring application available.

RSpec

RSpec, only code can do it justice.

describe Request do
  
  before do
    @sample = Sample.create(:name => 'my_sample', :method => "post")
  end
    
  context '#old' do
    it "should not return any" do
      Request.old.should be_empty
    end
    ...
  end

  context '#delete_old_non_failing' do
  
    it "should delete old non failing requests" do      
      create_old_request :response_status_code => 0
      create_old_request :response_status_code => 200
      expect {
        Request.delete_old_non_failed
      }.to change { Request.count }.by(-2)
    end
    ...
  end
  
  ...
end

Does your language have a Toolbox? Perhaps it is time to create one, or to switch to another language?

Monday, August 16, 2010

Good Practices for Rich Web Applications

Use jQuery

jQuery is the best thing that has happened to Javascript since it got first class functions in version 1.2. The library is elegant, powerful and has exactly the right level of abstraction for working with the DOM. There is nothing more to say. Learn it and use it. Good resources are: the jQuery API, my view of the jQuery API

Learn Javascript

Javascript is the programming language of the web. Learn it! Javascript is different from most other programming languages. It is dynamic, it has prototypical inheritance, and works more like Scheme than any of the languages that you are probably used to. If you want to learn Javascript you should get the following books, The Little Schemer, The Seasoned Schemer, Javascript, the Good Parts, and possibly High Performance Javascript

Learn CSS

Many programmers think that CSS is the language of designers and not programmers. This is not the case at all. If you are lucky enough to have a designer on your team (most people don't), CSS is the language with which you communicate. It is the interface between designers and programmers and as a programmer you should know it better than the designers. By knowing CSS well you will reduce the misunderstandings between you and your designer.

Unfortunately, many designers don't care about how code looks, as long as the design looks good on the surface. It will be up to you to make sure that the CSS doesn't get out out of hand. It will also be up to you to keep the HTML clean, and a good way to do this is to use semantic HTML, combined with CSS. You have no idea what the designers can come up with.

<!-- 
  Old School rounded corners, invented by a GOOD designer. 
  All this code was actually needed to achieve the purpose.
  -->
<style>
.t {background: url(dot.gif) 0 0 repeat-x; width: 20em}
.b {background: url(dot.gif) 0 100% repeat-x}
.l {background: url(dot.gif) 0 0 repeat-y}
.r {background: url(dot.gif) 100% 0 repeat-y}
.bl {background: url(bl.gif) 0 100% no-repeat}
.br {background: url(br.gif) 100% 100% no-repeat}
.tl {background: url(tl.gif) 0 0 no-repeat}
.tr {background: url(tr.gif) 100% 0 no-repeat; padding:10px}
</style>
<div class="t">
  <div class="b">
    <div class="l">
      <div class="r">
        <div class="bl">
          <div class="br">
            <div class="tl">
              <div class="tr">
                Lorem ipsum dolor sit amet consectetur adipisicing elit
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </div>
</div>

As an additional benefit you will become better at jQuery. Not only is CSS the query language of the browsers it is the query language of jQuery. Jariba!

Bulletproof Web Design is a good book web design, including CSS.

Decide how "Rich" your application should be

How rich should your application be? The scale varies from no Javascript to only Javascript, but you will probably want to land somewhere in between. Here are a few suggestions.

  • No Javascript, everything is server generated.
  • Slightly enhanced pages, simple validations, but no Ajax.
  • Ajax enhanced pages, but every page still reloads frequently.
  • Single page per area, entire area is handled by Javascript.
  • Only Javascript, Ajax interaction with the server
  • Only Javascript, no interaction with the server.

The important thing is to make a decision. If you don't make the decision, everyone will do different things on different parts of the application and you will loose consistency. In GUI, consistency is king. Make a decision and move on, you can always change your decision later.

Organize your code

Javascript

Make sure that all your Javascript code is namespaced properly. It is impolite to pollute the global namespace and it will bite you in the end. A simple variable declaration will do.

// Common namespace for your entire application
// This declaration lets you split your code of multiple files.
// If MyNamespace is defined use it, otherwise declare it.
MyNamespace = MyNamespace || {};

But, of course, it is also possible to get fancy and encapsulate the functions that you don't want to expose, if that is your cup of tea.

MyNamespace = MyNamespace || {};

MyNamespace.Tournament = function() {
 // Private stuff
 var tournamentCount = 0;
 function addTournament(tournament) {
  tournamentCount++;
 }
 
 return {
  //public stuff
  numberOfTournaments: function() {
   return tournamentCount;
  }
 }
}();

You should also separate your Javascript code into different files. The namespace idiom above helps to have the same namespace across multiple files. The same principles as with other type of code is valid with Javascript, organize the code by area, when it changes, where it is used, etc. Don't be afraid of the additional load time, splitting the files will give you. The files can easily be concatenated with tools like Rake, SCons, Ant or even a simple:

$ cat file1.js file2.js file3.js > all.js

They can also be compressed with JSMin or YUI Compressor.

Optimize your environment for development, not for production!

HTML

HTML is code! Divide your pages into partials by responsibilities. It allows you to keep your pages DRY and readable. The Single Responsibility Principle applies to HTML too.

Make sure you keep the Javascript with the code that it manipulates. If you, for example, have a calendar partial that uses jQuery DatePicker, you have to make sure that the partial includes all the necessary Javascript to configure the calendar. Don't keep Javascript code in the page away from the partial. Things that change together should be together.

CSS

Stylesheets are code too. They should also be split into areas that allow you to easily find and navigate them. Use Sass or SCCS to keep your CSS files DRY. Sass is good for designers to. It gives them the ability to use variable names, mixins, etc. and simplifies their usage of semantic names such as notice, and sidebar instead of yellow and left.

Optimize your environment for development, not for production!

Separate your Javascript from your HTML

All too often I see generated HTML pages with Javascript code in them. Don't do it. Keep the HTML free from Javascript.

<!-- DON'T DO THIS!  -->
<button id='update-button' onclick="MyNamespace.updateList();">Update List</button>

// In the Javascript file for the page.
MyNamespace.updateList = function() {...}


<!-- DO THIS! -->
<button id='update-button'>Update List</button>

// In the Javascript file for the page.
MyNamespace.updateList = function() {...}

$(function() {
  $("#update-button").click(function() {
    MyNamespace.updateList();
  });
});

It may seem like there is a lot more code in the good example, but notice the symmetry. The code that attaches the listener is in the same file as the code that uses the listener. This is good. Symmetry is good.

Use clone()

Separating the HTML and the Javascript goes both ways. Don't generate HTML code in Javascript. It doesn't matter that it is super simple to do it using jQuery.html(). Keep them separate, use jQuery.clone() instead.

// DON'T DO THIS
$("<li data-id='123'>My new item</li>").appendTo("ul");
// OR THIS
$("ul").append("<li data-id='123'>My new item</li>");


<!-- DO THIS -->
<ul>
<li id="list-template" class="template">All .template are hidden (display: none) in the CSS</li>
</ul>

// AND THIS
var $clone = $("#list-template").clone();
$clone.attr("data-id", "123").text("My new item").removeClass("template");
$("ul").append($clone);

The point of this is, again, to keep the HTML separate from the Javascript.

Decide how content flows between, the page, Javascript and the Server.

Once you have decided how rich your application should be, you have to decide on a method for moving the data between the HTML Page, Javascript and the Server. My preferred choice is to have every page that is served from the server include a context object with all the static data for the page and to have additional data that belongs to parts of the page be sent as data-attributes on the elements concerned.

The context object will contain all the data that is commonly needed in the page.

// Sample context object that is generated with the page.
MyNamespace = MyNamespace || {};
MyNamespace.Context = {
  user: {
    id: "28",
    name: "Anders Janmyr"
  },
  tournament: {
    id: "78344",
    name: "Fifa World Cup"
  }
};

I use the context object(s) to keep state on the client to. If it is important that the page looks exactly the same, even if the user reloads the page, I make sure that the state I stick into my context object is synched back to the server. This can easily be done, asynchronously, with Ajax and does not affect performance noticeable.

Elements specific data is sent along with the element it defines.

<!-- Element specific data attached to the elements with data-attributes -->
<ul id="tournament-menu">
  <li data-id="78344" data-participant-count="64">Fifa World Cup</li>
  <li data-id="666"  data-participant-count="44">Americas Cup</li>
  <li data-id="1464" data-participant-count="32">Rugby World Cup</li>
</ul>

The same argument as with the context object applies, as soon as I change an element in the GUI i need to feed that information back to the server. With elements I usually send the information to the server before updating the GUI, since element specific data is usually permanent data and not just session data.

An alternative solution to the context object above is to use the body element as the data-container, like this:

<body data-user-id="28" data-user-name="Anders Janmyr" data-tournament-id="78344" data-tournament-name="Fifa World Cup">

I tend to use the context object because I find it easier to add functionality to it than to the DOM element.

Only send the data that is needed to the client with the page. The rest of the data should be loaded on demand, with Ajax. Both JSON data and HTML templates can be loaded on demand. There is no need to deliver the entire page at once. Experiment and do what gives the best user experience.

Use file watchers to speed up feedback

If you compare the feedback cycle of Javascript and HTML development to Java and C# development, you are probably very happy with the short, tight feedback loop. This doesn't mean that you should be content. A feedback cycle cannot be too short.

xrefresh for Firefox and IE, and LiveReload for Safari and Chrome, are a couple of tools that will tighten your feedback loop even more.

Both tools are file watchers that listen to changes for files and refresh the browser when they change. If you combine this with two screens, you will have one screen for the browser, that updates continuously, while you edit your code on the other screen. Fabulous!

Conclusion

Rich web applications are very close to the traditional client-server model. We have to keep state on the client side to give the user a good experience. At the same time the application state, and indeed the entire application, can be swept away by a click of a button or a page reload.

This puts new demands on us as developers. We have to realize that we are responsible for the entire application, not just the business logic, but the HTML and CSS too. More that anything, we have to realize that Javascript is a first class programming language with its own programming techniques, which we need to master to be able to develop good web applications.

Thursday, August 12, 2010

No Deadlines

deadline, from New Oxford American Dictionary

  1. the latest time or date by which something should be completed : the deadline for submissions is February 5th.

  2. historical a line drawn around a prison beyond which prisoners were liable to be shot.

Deadlines in software development induce unnecessary stress, and transform otherwise enjoyable activities into chores.

We need to replace deadlines with livelines. A liveline is a deployed, working version of a product. It is after the fact, a milestone set when you are done.

How Deadlines Are Set

Deadlines are set in different ways in different projects.

In the classic Death March project, a deadline is set by someone outside the project with no clue whatsoever about how long it will take. Typically this kind of deadline is based on some external event, where the project will be demonstrated. The event may be a conference, or something similar.

In Scrum, the deadlines are set by the process itself. Most Scrum projects I have worked with use a two week deadline instead of the recommended 30 days. The team decides how many stories they can complete by that deadline, based on rough estimations done in the sprint planning.

Some projects set deadlines by a developer being asked (or asking himself) when the task will be done. The developer will say: "I'll have it done by 3 o'clock", a wild guess.

The problem with all these approaches is that the deadlines are, at best, based on guesses of the team doing the development. But guesses are guesses, not something you can set a deadline by.

And deadlines work both ways, a task is very rarely completed before it's deadline.

Kinds of Deadlines

Real Deadlines

If a task is not completed by a certain time someone will die. This kind of task is really common in novels and movies. A clock is ticking and if the task is not completed by the time the clock reaches zero, BOOM!

If have never seen this kind of deadline in software development, but they may exist.

Planning Deadlines

Some deadlines are set in order to simplify planning. In the large Death March projects, the plan is to show the project at a certain date. In integration projects, the deadlines are set to be able to synchronize with other teams working on the same product.

This kind of deadlines are actually understandable, but I am arguing that they actually slow down projects by introducing stress, which introduces bugs. The bugs introduce bugs in the dependent projects, and slows down many projects instead of just one.

Motivational Deadlines

Motivation deadlines are set to get people to work. The assumption is that if a deadline is not set people will procrastinate over different ideas, or just slack off since there is no date set for the release anyway.

This is, absolutely, the worst kind of deadline. If the team is not motivated enough to do a good job without a deadline, you need to get a new team, not set a deadline.

Why Deadlines Are Bad

Stress

Consider walking on a one foot wide plank. If it is on the ground it is easy. If it is two meters up it is difficult, but if you put it at twenty meters it is a whole different ballgame. That is what its like with deadlines, they put an unnecessary pressure on people.

Loss of Motivation

Work consist of whatever a body is obliged to do, and play consists of whatever one is not obliged to do. -- Mark Twain

In his book, Drive, Dan Pink differentiates between intrinsic motivation and extrinsic motivation.

Extrinsic motivation is motivation driven by carrots and sticks, rewards and punishment. It works well for simple, well-defined tasks.

Intrinsic motivation works well for creative, ill-defined tasks, like programming.

Intrinsic motivation is fragile, it disappears in the presence of carrots and sticks. A deadline is a stick, whether set by ourselves or someone else.

Livelines

A liveline is, quite simply, a task that is done. When a task is done, it is easy for others to use it, in their plans or in reality. If we want to build a stable house, we should build it on stable ground.

When using livelines the iteration length is the same length as a task. This is usually a lot shorter than the usual one to four weeks common in Scrum and other agile methods. It also leads to variations in the iteration length.

If you implement your task all the way into production, you get Continuous Deployment or Continuous Delivery. This puts a lot of pressure on your tooling and on automation but, I believe this is the way to go.

It is 2010, it is time to get rid of deadlines.