Waf: a pleasant build system

A good software project must have a good build system. Unless you have a small code base consisting entirely of dynamic, scripted languages, you probably need to “build” your code before you can use it. Until around an year ago, the only build tool that I used and was familiar with was GNU Make. Make and the autotools family of tools have served the developer community well the past few decades.

But the Make model is rife with problems. Here are a few of them:

  • Make requires the use of its own domain specific language — this is, in general, not a good idea. Have you looked at any sizable project’s Makefile lately? Its hard to understand, and harder to modify.
  • In the same vein, autoconf/automake are notoriously hard to use. Bear in mind that these tools are supposed to make your life easier.
  • Makefile are so hard to write and extend that several popular build systems today are essentially Makefile generators. A good example is CMake.
  • Make relies heavily on file timestamps to detect changes.
  • Make is slow.
  • Makefile are not modular. Recursive Make is especially evil.

I recently began work on a new pet project. As is usually the case, I spent a lot more time figuring out what tools and libraries I would use for my project, than in actually writing any code for the project :) Part of the investigation was to survey the state of the art in build systems. At work, we started using SCons for most of our build, which was already a huge improvement over Make. But SCons has its own set of issues.

One of the nicest features in SCons is that build files are regular Python files. This provides enormous flexibility and immediate familiarity. Unfortunately, the SCons documentation leaves much to be desired. I still don’t quite understand the execution model of SCons very well. For instance, I know how to extend SCons to support cross-compilation for multiple platforms. However, I don’t really understand why those modifications work — there’s quite a bit of black magic that goes on behind the scenes. As a concrete example, there are several magic variables such _LIBDIRFLAGS that have strange powers.

Waf
Waf

After some more looking around, I discovered Waf. And now that I’ve played around with it a little bit, I’m happy to say that it is the most pleasant build system I’ve ever used. Things I really like about Waf:

  • The execution model just makes sense to me. You typically build a project in phases: there’s a configure phase, to sort out dependencies, tools etc; there’s the actual build phase; and then there’s the install phase. It is not uncommon to have a ‘dist’ step as well, to prepare the source for distribution. Waf understands these operations as first class entities. There is a very strong notion of workflow built into Waf.
  • Comprehensive documentation. Check out the Waf book and the wiki.
  • Waf has a very strong task model. There is a much stronger notion of dependencies (powered by content hashes, not timestamps). Waf also enforces that all generated code lands up in a separate “build” directory, so your source tree always remains clean.
  • Using waf is a breeze — there are no big dependencies, no packages to install, no bloated software to include with your code. Just a single 80kb script.
  • Progress indication and colored output is built in, not an after thought. Like SCons, Waf build files are regular Python files.
  • Waf is fast. Faster than SCons.

Of course, Waf is not perfect. Coming from a Make/SCons world, I sorely miss the ability to build specific targets. Yes there are ways to achieve this in Waf, but they are all clumsy. The API documentation (and the source itself) are a bit hard to parse.

4 comments

  1. Henk

    Waf is way too difficult in practise. This actually makes it slower when you account for the time required to start using it. Scons is very easy to use and it’s pretty quick as is. Waf is actually so incredibly weird that it’s easier to use plain old make!

    Sure it doesn’t have coloured output but I don’t care, I’d rather be able to log the build process without weird ANSI control codes in it.

    • Diwaker Gupta

      Henk: To each his own. I’ve used Scons and find it very hard to extend, enhance and maintain. Makefiles are great to start off simple projects. In all fairness, I haven’t yet used waf for a large production project, but I’m looking forward to it. Could you clarify what you mean when you say “waf is so incredibly weird”?

  2. drahnr

    I recently stumbled upon waf, and so far it is one of the most pleasant build systems I’ve used so far. (scons, cmake, make, qmake, autotools,..)
    The documentation is actually worth reading it.

    • Johnny

      I had to use waf as one of our vendors requires and depends on it. After a lot of work put into it I can say it proudly: I’d rather write my own build system than use waf. Frankly – writing makefiles by hand is more plesant experience!

Leave a Reply