Vladimir Vukićević

I Make Fun Stuff

Faster Firefox (re-)Build Speeds

One of the common complains about Firefox development is that build speed is slow, and that the normal “edit, compile, test” cycle is painful. Over time, developers learn tricks (“If I change file X, I have to rebuild in foo/, bar/, toolkit/library/”), but this is error prone. New developers also don’t have any of that knowledge, and they’re struggling with a lot of new things that having the build system slow down their learning hurts significantly.

Ehsan and BenWa had an idea for how to make this better. The core idea was to just annotate the relevant build rules to spit out specific rules for each target and their inputs, thereby generating an alternate, faster, non-recursive build system as part of the first build. Others have had this idea before, but usually only for testing or gathering some data. BenWa did a first prototype, generating makefile fragments that could be included together. I did some hacking to create a little python script for the fragment generation, which let us start experimenting with generating ninja files instead of using make – for what we were doing, ninja is perfect, since its purpose is to execute “generated build systems”.

BenWa is working on extending it to generate project files for some IDEs. Since we have the full dependency graph, we can also do parallel builds with 100% cpu utilization, and can significantly distribute builds as well with little overhead.

Here’s the main result that you’re probably wondering about, all on my Windows 7, quad-core 4th Gen Core i7 (8 threads), 7200 RPM hard drive, 16GB memory desktop:

  • Clean tree
    • top level pymake (-s -j8)
      • 490 seconds (8 minutes, 10 seconds)
    • top level ninja
      • 1.46 seconds
  • Touch a deep inner source (gfxGDIShaper.cpp – requires libxul relink)
    • top level pymake (-s -j8)
      • 544 seconds (9 minutes, 4 seconds)
    • magic knowledge pymake (pymake -s -j8 -C gfx/thebes && pymake -s -j8 -C toolkit/library)
      • 66 seconds
    • top level ninja
      • 61 seconds (3 total commands executed)
  • Touch some core headers (nsTArray.h, nsTSubstring.h)
    • top level pymake (-s -j8)
      • 33 minutes, 14 seconds
    • top level ninja
      • 21 minutes, 48 seconds

I was going to make a chart, but it would have needed a logarithmic axis.

These commands of course don’t do the same thing; running make will redo the export step as well as recurse into a bunch of other targets that are never seen by hacky.mk. But the result is the same for most developers, and the time savings is significant. These numbers are also a good target for other more comprehensive build system work; the results above are purely bound my compiler and linker speed (and IO; I need to retry this on a SSD – and also wrapper scripts for python, executing bash, etc. There’s a bit of room for improvement!). It would be very difficult to end up with faster builds than the above, given equivalent parallelization.

How to use it

  1. Clone hacky.mk from github: http://github.com/bgirard/hacky.mk. Then run ./hackify ../path/to/mozilla-central. It’ll patch rules.mk and copy some files into place (alternatively, you can patch by hand symlink the hackymake dir from the repo as build/tools/hackymake and js/src/build/tools/hackymake).
  2. Do a normal build. You’ll want to do a full build into a brand new objdir.
  3. While that’s going, clone http://github.com/martine/ninja.git and build it. Copy the binary somewhere in your PATH.
  4. When the Firefox build is done, run $srcdir/tools/hackymake/reninja
  5. To build, run ninja

What it can do:

  • Rebuild objects from C/C++ sources (for JS, libxul, and friends)
  • Rebuild libraries
  • On Windows, copy exported include files into your objdir’s dist/include dir if the original changes (since we can’t use symlinks on Windows) ** caveat: not JS, yet; JS has its own export rules that aren’t hooked up yet

What it can’t do:

Basically, anything not listed above.

It doesn’t do anything outside of (mostly) libxul or JS deps. It doesn’t know anything about any of the external build systems (e.g. nspr).

It doesn’t know how to build binaries (e.g. firefox.exe, the js shell). It also doesn’t know how to regenerate headers from IDL files, or IPC stubs from ipdl files. None of that would be very hard to add, it just wasn’t a top priority.

It can’t handle additions or removals of source files, at least not automatically. If you just run a regular “make” in the directory where the source file was added and you rerun reninja, that should be enough. Removals are harder; manually remove the relevant file from $objdir/.hacky and run reninja.

It can’t handle changes to the build configuration. Build configuration changes are unlikely to be ever handled; they’ll require a rebuild using the regular build system first.

Next steps, how can you help?

  • Use it, tell others about it
  • Provide feedback; Github issues work fine for this
  • We’ll work to get it into the tree
  • Add support for additional rules (binaries, idl/ipdl generators, etc!)