[Aptitude-devel] Towards a better build system

[Daniel Burrows]

  aptitude currently uses autoconf/automake as the basis of its build
system, with various extra stuff hacked on as needed for
internationalization, documentation, etc.  This is nice since it follows
the de facto free software standards automatically.  It's not so nice in
a few other ways:

  1) Builds can't be performed entirely in parallel, because make only
     builds one directory at a time.  This is actually what got me
     looking into this problem; I bought myself a shiny new desktop
     with eight cores, and was disappointed to realize my aptitude
     builds were bottlenecked by ranlib.

  2) The whole make/automake/autoconf/m4 toolset doesn't offer very good
     facilities for abstraction.  You can see this, for instance, in the
     pile of copy-and-paste in doc/, where each language seems to copy
     another language's Makefile and hack it up.  We should be able to
     just say "build the docs for XX".

  3) Because the autotools are built in languages that don't lend
     themselves to open-ended programming and have a lot of weird quirks
     (sh, m4, make), writing simple "configure" tests tends to be a major
     task, to the point that I avoid it if at all possible.

  4) "make distcheck" fails because it tries to remove something in
     "/var".  (unless, perhaps, you're root.  I don't want to know what
     it does then)

  5) POTFILES.in.in is built automatically from the source tree.  But
     it doesn't know the exact list of files (even though automake
     does), so it ends up with references to files that only exist on
     my computer, and the Debian patch ends up cluttered with lines that
     remove them.

  6) autoconf/automake don't provide support for canned build options.
     Most aptitude builds use one of two sets of compile flags, but this
     isn't known to the build system, so I have to tell new contributors
     to enter "CXXFLAGS=-g\ -O0\ -fno-inline ./configure" before
     building if they want to debug.

  6) If I want to switch between configurations, I have to "make clean"
     and reconfigure, then recompile.  This means I lose all my build
     output whenever I switch back and forth; ccache helps a bit, but
     not enough.  It also means that if I need both builds, I can't get
     them in parallel.  This isn't just a theoretical question: try
     building aptitude's .deb in parallel and it'll fail because
     it's trying to configure the same directory tree twice (in fact,
     configure itself has race conditions that usually cause failure).



  Now, all the above *could* be solved in an automake-based system.
However, by my estimate the amount of work involved is on the same scale
as the work required to switch to another build system -- except that
if I switch to one that's more inherently flexible, the amount of work
*next time* I run into problems with the build system will be greatly
decreased.

  So, somewhat arbitrarily, I've decided to start experimenting with a
collection of SCons build rules for aptitude.  SCons has some nice
features:

  1) Its build scripts are written in a general-purpose programming
     language that's widely known and supported (Python).  This
     eliminates several of my problems above immediately; for instance,
     Python has very well-developed facilities for modularization,
     encapsulation, and abstraction.

  2) I read its documentation back in 2003, when it wasn't ready for
     actual use yet, so I was vaguely familiar with it.

  3) It seems to be sufficiently expressive to do what we need.  In
     particular, the trickiest stuff above is supported, like having
     multiple build directories for different configurations.



  The one downside of SCons is that it's not really a complete build
system, more of a language for writing build systems.  This means it's
not as well standardized, since everyone ends up with a bunch of custom
code, and that there's more work to do to get a fully functional build
system put together.  On the other hand, that means it can fit better
to exactly what aptitude needs -- and I don't think the coding will be
that onerous, since I have a real language at my disposal to do it in.

  If you're curious, check out the Mercurial repository, where I have
a bunch of poorly factored and broken SCons stuff checked into the
tree.  It won't show up in distributed tarballs since I haven't
registered it with automake yet.

  Daniel