Dealing with autotools

[James Westby]

On Sun, 2009-05-10 at 09:05 +0200, martin f krafft wrote:
> also sprach Manoj Srivastava <srivasta at acm.org> [2009.05.10.0030 +0200]:
> > > If I may sum it up in my own words and mix it with my own opinion,
> > > then distributions like Debian aren't really necessarily the target
> > > audience of the autotools output and it might make a lot more sense
> > > for us to build from VCS. This means that autotools would become
> > > part of the build process *and* that we rid ourselves of the "dogma"
> > > of distributing pristine tar sources.
> > 
> >         Huh? These do not logically follow. I always autoreconf during
> >  the build process for my Debian packages, and yet, I also ship pristine
> >  sources. I see no contradiction.
> 
> The files generated by autotools are not in VCS, so when you build
> from VCS, they don't exist and need to be created as part of the
> build process. No tarball involved anymore.

They aren't exclusive though.

With current practices there is effectively a "Debian upstream" branch
from the VCS, which contains thes history of the tarballs that made up
the source packages.

Even when building from a VCS it is possible, and I would say 
encouraged, to maintain this branch. That branch can contain the 
autotools files.

The process of packaging a new upstream snapshot is then to export a
tarball from the desired revision, add autotools files to that tarball,
and then import that back to the "Debian upstream" branch, add
pristine-tar information, and then merge to the packaging branch (or
do whatever machinations are required to get the final tree to build
the source package from).

  (I actually think autotools at build time isn't a bad thing, so you
   could just import a plain export tarball instead and do that, it
   obviously depends on your rules file though)

The above ensures that you get repeatability with the pristine-tar
information in case you need a -2 upload, and correctly represents
what happened in the case where there was some transformation done
to the VCS tree to get the tarball, as this transformation will be
in the diff between the imported tarball revision and the revision
that it was taken from.

I am aiming to make this process very easy in bzr-builddeb, as I
feel it should be *the* way to pull in new upstream changes. (Upstream
has released a tarball from a tag? Use that tarball rather than
a generated one. Upstream has no public VCS? Just import the tarball
without the extra parent.) The above, with the exception of a 
transformation of the tarball, is all a single command in the latest
versions.

I'm interested in adding the transformation step, but I haven't decided
what the best way is yet, suggestions welcome. Some ideas:

  1. Have an option to the command (and possibly a config option)
     that specifies the transformation (and allow $SHELL for those
     that don't value repeatability). The difficulty with this is that
     sometimes I want the command to actually produce a tarball
     (setup.py sdist is crucial for one project that I package), so
     there has to be some way to pick up the resulting tarball.

  2. Do something similar to get-orig-source (though I would like
     that to be replaced with something a little more useful anyway).
     This is basically the same as above, but encodes the process in
     to the source package, so doesn't require the use of the particular
     tool to get a good result. It's possible that an improved 
     get-orig-source would allow this to easily work.

As I said, suggestions welcome, I think this could be one useful
thing for the project to come up with without having to come up
with a complete solution for packaging in a VCS.


Thanks,

James