[Aptitude-devel] Towards a better build system
dburrows at debian.org
Thu Apr 22 14:56:12 UTC 2010
aptitude currently uses autoconf/automake as the basis of its build
system, with various extra stuff hacked on as needed for
internationalization, documentation, etc. This is nice since it follows
the de facto free software standards automatically. It's not so nice in
a few other ways:
1) Builds can't be performed entirely in parallel, because make only
builds one directory at a time. This is actually what got me
looking into this problem; I bought myself a shiny new desktop
with eight cores, and was disappointed to realize my aptitude
builds were bottlenecked by ranlib.
2) The whole make/automake/autoconf/m4 toolset doesn't offer very good
facilities for abstraction. You can see this, for instance, in the
pile of copy-and-paste in doc/, where each language seems to copy
another language's Makefile and hack it up. We should be able to
just say "build the docs for XX".
3) Because the autotools are built in languages that don't lend
themselves to open-ended programming and have a lot of weird quirks
(sh, m4, make), writing simple "configure" tests tends to be a major
task, to the point that I avoid it if at all possible.
4) "make distcheck" fails because it tries to remove something in
"/var". (unless, perhaps, you're root. I don't want to know what
it does then)
5) POTFILES.in.in is built automatically from the source tree. But
it doesn't know the exact list of files (even though automake
does), so it ends up with references to files that only exist on
my computer, and the Debian patch ends up cluttered with lines that
6) autoconf/automake don't provide support for canned build options.
Most aptitude builds use one of two sets of compile flags, but this
isn't known to the build system, so I have to tell new contributors
to enter "CXXFLAGS=-g\ -O0\ -fno-inline ./configure" before
building if they want to debug.
6) If I want to switch between configurations, I have to "make clean"
and reconfigure, then recompile. This means I lose all my build
output whenever I switch back and forth; ccache helps a bit, but
not enough. It also means that if I need both builds, I can't get
them in parallel. This isn't just a theoretical question: try
building aptitude's .deb in parallel and it'll fail because
it's trying to configure the same directory tree twice (in fact,
configure itself has race conditions that usually cause failure).
Now, all the above *could* be solved in an automake-based system.
However, by my estimate the amount of work involved is on the same scale
as the work required to switch to another build system -- except that
if I switch to one that's more inherently flexible, the amount of work
*next time* I run into problems with the build system will be greatly
So, somewhat arbitrarily, I've decided to start experimenting with a
collection of SCons build rules for aptitude. SCons has some nice
1) Its build scripts are written in a general-purpose programming
language that's widely known and supported (Python). This
eliminates several of my problems above immediately; for instance,
Python has very well-developed facilities for modularization,
encapsulation, and abstraction.
2) I read its documentation back in 2003, when it wasn't ready for
actual use yet, so I was vaguely familiar with it.
3) It seems to be sufficiently expressive to do what we need. In
particular, the trickiest stuff above is supported, like having
multiple build directories for different configurations.
The one downside of SCons is that it's not really a complete build
system, more of a language for writing build systems. This means it's
not as well standardized, since everyone ends up with a bunch of custom
code, and that there's more work to do to get a fully functional build
system put together. On the other hand, that means it can fit better
to exactly what aptitude needs -- and I don't think the coding will be
that onerous, since I have a real language at my disposal to do it in.
If you're curious, check out the Mercurial repository, where I have
a bunch of poorly factored and broken SCons stuff checked into the
tree. It won't show up in distributed tarballs since I haven't
registered it with automake yet.
More information about the Aptitude-devel