Makefiles are really good at one thing, managing dependencies between files. In
make makes sure all files that depend on another file are
updated when that file changes.
make how to do this by declaring rules. A typical rule looks like
A Simple Makefile
# Makefile # Create bundle.js by concatenating jquery.js lib.js and main.js bundle.js: jquery.js lib.js main.js cat $^ > $@
There are three parts to this rule:
- The target,
bundle.js, before the colon (:).
- The prerequisites (what the target depends on),
jquery.js lib.js main.js, after the colon (:).
- The command, cat $^ > $@, on the next line after a leading tab, (\t).
There are two "automatic" variables in this command.
- $@ - filename representing the target, in this case
- $^ - filenames representing the list of the prerequisites (with duplicates removed).
"Automatic" means that the variables are automatically populated with relevant
filenames. This will
make more sense when we get into patterns later.
Here are some more variables that are useful.
- $< - filename representing the first prerequisite.
- $? - filenames representing the list of the prerequisites that are newer than the target.
$*- filename representing the stem of the target, in the above case
The Make Manual contains the full list of automatic variables
make with the above Makefile results in the following execution.
$ make cat jquery.js lib.js main.js > bundle.js $ make make: 'bundle.js' is up to date.
make runs the first target it finds in the file if none is given on the
command line. In this case it is the only target.
The second run didn't do anything since
bundle.js is up to date. To be up to
date means that the
last-modified time of
bundle.js is newer than any of
last-modified times. Simple but powerful! When we create
new targets all we have to worry about is making sure that our targets know
what files it depends on, and what files they depend on, and so on.
It is possible to enter many targets on the left of the colon (:).
treat them as separate rules and the automatic variable will make sure that the
correct files are built.
make treats the rules as separate rules, it will only build the
first of them, the default target.
# Makefile bundle.js bundle2.js: jquery.js lib.js main.js cat $^ > $@
$ make make: 'bundle.js' is up to date.
If we want to build
bundle2.js, we can do it by explicitly telling
make to do
it by giving the target as command line parameter.
$ make bundle2.js cat jquery.js lib.js main.js > bundle2.js
make to build both targets at once, we need to add a new,
# Makefile .PHONY: bundles bundles: bundle.js bundle2.js bundle.js bundle2.js: jquery.js lib.js main.js cat $^ > $@
make now results in (after removing bundle*)
$ make cat jquery.js lib.js main.js > bundle.js cat jquery.js lib.js main.js > bundle2.js
.PHONY: target is a target without a corresponding file for
make to check
last modified time on. This means the target will always be run, forcing
to check if all the target's prerequisites needs to be built. The
label is not strictly necessary. If it is left out,
make will check to see if
there is a file called
bundles and since there isn't one it will build it
Here's an illustration:
# Makefile build: echo 'Running build'
# Makefile2 .PHONY build: echo 'Running build'
$ touch build $ make make: 'build' is up to date. $ make -f Makefile2 echo 'Running build'
Marking a target that doesn't represent a file as
.PHONY: is easy to do and
avoids annoying problems once your
Makefile contains a
clean target to remove all the
artifacts that are built. In the above case it would contain something like:
# Makefile .PHONY: clean clean: rm -f bundle*.js
make clean will now clean out all files created by the Makefile.
make usually needs a bit of special treatment. Let's say we want
the bundles above to end up in a
build directory. The following
illustrates a problem.
# Makefile bundles: build/bundle.js build/bundle2.js build/bundle.js build/bundle2.js: jquery.js lib.js main.js cat $^ > $@
make illustrates the problem:
cat jquery.js lib.js main.js > build/bundle.js /bin/sh: build/bundle.js: No such file or directory make: *** [build/bundle.js] Error 1
The directory is not automatically created by
cat. There are three ways to
solve this and one is better than the others.
mkdir -pto all rules creating files in the directory.
- Add a prerequisite to create the directory on the
- Add an ordering prerequisite (|) to the rules creating the files in the directory.
1. is not good because the directory will be created more than once, one for
each bundle (this is why the
-p is needed).
2. is not good because the
build directory is not a prerequisite target
3. is good because the
build directory is clearly a prerequisite of the rule
that creates the bundles in this directory.
The reason we have to use an ordering prerequisite instead of a normal
prerequisite is that
cat would fail otherwise. Here's the resulting good
# Makefile bundles: build build/bundle.js build/bundle2.js build: mkdir build build/bundle.js build/bundle2.js: jquery.js lib.js main.js | build cat $^ > $@ clean: rm -rf build
Now, we know the basics of Makefiles. We can create rules with targets, prerequisites and commands that are run when needed. But, we have been working with named files all this time. This works fine for small examples like above, but when we have hundreds of files this quickly gets out of hand. Patterns to the rescue.
Let's say that we have a bunch of images that we would like to optimize by
running them through an optimizer. The images are in the
and the optimized images are built into
build/images. The naive (and not
working) way to do this is shown below. (I'm faking optimize with a simple
% sign is glob matched with the part of the filename that is
# Makefile (NOT WORKING) optimize: build/images/* build/images/%: images/% | build/images cp $< $@ build/images: mkdir -p $@
To see why this is not a viable Makefile, we try to run it with
$ make mkdir -p build/images cp images/a.png build/images/*. $ tree build build/ └── images └── *
What is going on here? Why is only one image copied and why is it copied as
build/images/*? The problem is that the target files don't exist yet and
* is interpreted literally. If we copy the files into the build directory
and touch the source files, it works the way we want.
# Copy the image directory into build $ cp -r images build # Touch the orignal images $ touch images/* # Build works since build/image/* evaluates to the list of images $ make cp images/a.png build/images/a.png cp images/b.png build/images/b.png
Here is the main rule to know about patterns. The target file list has to
be created from the available source files. To do this we have help of a
number of functions, including
shell will allow us
to call anything that we can call from the shell This is very powerful!
How do we solve the above problem? We can do this by getting a list of source images and transforming this list into a list of target images. This is easily done.
# Makefile # 1. Get the souce list of images images := $(wildcard images/*.png) # 2. Tranform the source list into the target list target_images := $(images:%=build/%) # 3. Our default target, optimize, depends on all the target_images optimize: $(target_images) # 4. Build the targets from the sources, make sure build/images exist $(target_images): build/% : % | build/images cp $< $@ build/images: mkdir -p $@
The first line introduces both variables and functions.
Variables can be declared in a number of
ways, but the
:=-declaration is the simplest. It evaluates the value on the right and sets
the value on the left to the result, like variables in most programming
Functions are called with the
$() construct, and
wildcard is a function
that evaluates a shell filename pattern and returns a list of filenames.
The full line above populates
images with the
.png files from the
The second line converts the source images into the target images. Variables
are evaluated the same way as function calls, with the
$() construct. By
adding a colon-equals expression, a variable substitution reference, after the
variable name we can substitute a pattern for another. Example
files := "src/a.java src/b.java" # Pattern replaces the files into "target/a.class target/b.class" class_files := $(files:src/%.java:target/%.class)
The third line tells
make that our
optimize target depends on all the
targets existing. This makes sure that all the targets are built.
The fourth line sets up the targets `$(target_images) and its prerequisites with a static pattern rule. The pattern does the opposite of the variable substitution reference above, it deconstructs a single target into the source it depends on. The final part of the of this line is an order prerequisite on the rule to create the directory.
A Recipe for Creating Makefiles
- Create a list of targets that you want to create from the sources. You have
the full power of
awk, etc. at your disposal.
- Create a static pattern rule to convert a single target into the source it can be created from.
- Add order prerequisites to make sure directories are automatically created.
- Add a callable target that depends on all the target files that you want to create.
Here's a more exotic example of what you can use
make for. We have a directory
lib. The corresponding test files are in
test. There may be multiple directories below both
testfiles are named like the source files with an added
.spec after the stem
of the filename.
We want to use a makefile to help us run only the tests that are relevant based
on the files that are changed. To keep track of what tests have been run we're
going to use marker files and
touch them every time a test is run.
# Create the list of test files by using the shell function and find test_files := $(shell find test -name '*.spec.js' -print) # Convert the test files into marker files with variable substitution # A marker files looks like this tmp/model/person_test.marker marker_files := $(test_files:%.js=tmp/%.marker) # Do the same thing for the test directories test_dirs := $(shell find test -type d -print) # The marker directories have their normal names, no special ending marker_dirs := $(test_dirs:%=tmp/%) # test is the default target .PHONY: test test: $(marker_files) # The marker files order depend on the marker directories $(marker_files): | $(marker_dirs) # Marker files depend on the source files # Deconstruct a marker file into a source file $(marker_files): tmp/test/%.spec.marker : lib/%.js # Marker files depend on the test files # Deconstruct the marker file into a test file # When any prerequisite changes, run the tests and then touch the marker file $(marker_files): tmp/%.marker : %.js mocha $< @touch $@ # Create the marker dirs $(marker_dirs): @mkdir -p $@ # Clean the project by removing the entire tmp directory .PHONY: clean clean: rm -rf tmp
Now whenever you run
make, it will run only the relevant tests.
# Modify a test file $ touch test/models/passbook.spec.js $ make mocha test/models/passbook.spec.js passbook ✓ generate pass strips image names ✓ doesnt crash with no store number 2 passing (21ms) # Modify a source file $ touch lib/models/passbook.js $ make mocha test/models/passbook.spec.js passbook ✓ generates pass strip image names ✓ doesnt crash with no store number 2 passing (22ms)
Makefiles are really good at one thing: building only stale files. If that
is our problem, we should give
make a try.