Running Scripts with npm

Most people are aware that is is possible to define scripts in package.json which can be run with npm start or npm test, but npm scripts can do a lot more than simply start servers and run tests.

Here is a typical package.json configuration.

// package.json
// Define start and test targets
{
  "name": "death-clock",
  "version": "1.0.0",
  "scripts": {
    "start": "node server.js",
    "test": "mocha --reporter spec test"
  },
  "devDependencies": {
    "mocha": "^1.17.1"
  }
}
// I am using comments in JSON files for clarity.
// Comments won't work in real JSON files.

start, actually defaults to node server.js, so the above declaration is redundant. In order for the test command to work with mocha, I also need to include it in the devDependencies section (it works in the dependencies section also, but since it is not needed in production it is better to declare it here).

The reason the above test command, mocha --reporter spec test, works is because npm looks for binaries inside node_modules/.bin and when mocha was installed it installed mocha into this directory.

The code that describes what will be installed into the bin directory is defined in mocha's package.json and it looks like this:

// Macha package.json
{
  "name": "mocha",
  ...
  "bin": {
    "mocha": "./bin/mocha",
    "_mocha": "./bin/_mocha"
  },
  ...
}

As we can see in the above declaration, mocha has two binaries, mocha and _mocha.

Many packages have a bin section, declaring scripts that can be called from npm similar to mocha. To find out what binaries we have in our project we can run ls node_modules/.bin

# Scripts availble in one of my projects
$ ls node_modules/.bin
_mocha      browserify  envify      jshint
jsx         lessc       lesswatcher mocha
nodemon     uglifyjs    watchify

Invoking Commands

Both start and test are special values and can be invoked directly.

# Run script declared by "start"
$ npm start
$ npm run start

# Run script declared by "test"
$ npm test
$ npm run test

All other values will have to be invoked by npm run. npm run is actually a shortcut of npm run-script.

{
  ...
  "scripts": {
    // watch-test starts a mocha watcher that listens for changes
    "watch-test": "mocha --watch --reporter spec test"
  },
}

The above code must be invoked with npm run watch-test, npm watch-test will fail.

Running Binaries Directly

All the above examples consists of running scripts that are declared in package.json but this is not required. Any of the commands in node_modules/.bin can be invoked with npm run. This means that I can invoke mocha by running npm run mocha directly instead of running it with mocha test.

Code Completion

With a lot of modules providing commands it can be difficult to remember what all of them are. Wouldn't it be nice if we could have some command completion to help us out? It turns out we can! npm follows the superb practice of providing its own command completion. By running the command npm completion we get a completion script that we can source to get completion for all the normal npm commands including completion for npm run. Awesome!

I usually put each of my completion script into their own file which I invoke from .bashrc.

# npm_completion.sh
. <(npm completion)

# Some output from one of my projects
$ npm run <tab>
nodemon                  browserify               build
build-js                 build-less               start
jshint                   test                     deploy
less                     uglify-js                express
mocha                    watch                    watch-js
watch-less               watch-server

Pretty cool!

Combining Commands

The above features gets us a long way but sometimes we want to do more than one thing at a time. It turns out that npm supports this too. npm runs the scripts by passing the line to sh. This allows us to combine commands just as we can do on the command line.

Piping

Lets say that I want to use browserify to pack my Javascript files into a bundle and then I want to minify the bundle with uglifyjs. I can do this by piping (|) the output from browserify into uglifyjs. Simple as pie!

  //package.json
  // Reactify tells browserify to handle facebooks extended React syntax
  "scripts": {
    "build-js": "browserify -t reactify app/js/main.js | uglifyjs -mc > static/bundle.js"
  },
  // Added the needed dependencies
  "devDependencies": {
    "browserify": "^3.14.0",
    "reactify": "^0.5.1",
    "uglify-js": "^2.4.8"
  }

Anding

Another use case for running commands is to run a command only if the previous command is successful. This is easily done with and (&&). Or (||), naturally, also works.

  "scripts": {
    // Run build-less if build-less succeeds
    "build": "npm run build-js && npm run build-less",
    ...
    "build-js": "browserify -t reactify app/js/main.js | uglifyjs -mc > static/bundle.js",
    "build-less": "lessc app/less/main.less static/main.css"
  }

Here I run two scripts declared in my package.json in combination with the command build. Running scripts from other scripts is different from running binaries, they have to prefixed with npm run.

Concurrent

Sometimes it is also nice to be able to run multiple commands at the concurrently. This is easily done by using & to run them as background jobs.

  "scripts": {
    // Run watch-js, watch-less and watch-server concurrently
    "watch": "npm run watch-js & npm run watch-less & npm run watch-server",
    "watch-js": "watchify app/js/main.js -t reactify -o static/bundle.js -dv",
    "watch-less": "nodemon --watch app/less/*.less --ext less --exec 'npm run build-less'",
    "watch-server": "nodemon --ignore app --ignore static server.js"
  },
  // Add required dependencies
  "devDependencies": {
    "watchify": "^0.6.2",
    "nodemon": "^1.0.15"
  }

The above scripts contain a few interesting things. First of all watch uses & to run three watch jobs concurrently. When the command is killed, by pressing Ctrl-C, all the jobs are killed, since they are all run with the same parent process.

watchify is a way to run browserify in watch mode. watch-server uses nodemon in the standard way and restarts the server whenever a relevant file has changed.

watch-less users nodemon in a less well-known way. It runs a script when any of the less-files changes and compiles them into CSS by running npm run build-less. Please note that the option --ext less is required for this to work. --exec is the option that allows nodemon to run external commands.

Complex Scripts

For more complex scripts I prefer to write them in Bash, but I usually include a declaration in package.json to run the command. Here, for example, is a small script that deploys the compiled assets to Heroku by adding them to a deploy branch and pushing that branch to Heroku.

#!/bin/bash

set -o errexit # Exit on error

git stash save -u 'Before deploy' # Stash all changes, including untracked files, before deploy
git checkout deploy
git merge master --no-edit # Merge in the master branch without prompting
npm run build # Generate the bundled Javascript and CSS
if $(git commit -am Deploy); then # Commit the changes, if any
  echo 'Changes Committed'
fi
git push heroku deploy:master # Deploy to Heroku
git checkout master # Checkout master again
git stash pop # And restore the changes

Add the script to package.json so that it can be run with npm run deploy.

  "scripts": {
    "deploy": "./bin/deploy.sh"
  },

Conclusion

npm is a lot more than a package manager for Node. By configuring it properly I can handle most of my scripting needs.

Configuring start and test also sets me up for integration with SaaS providers such as Heroku and TravisCI. Another good reason to do it.