Dealing with autotools

[James Westby]

On Mon, 2009-05-11 at 10:02 +0200, martin f krafft wrote:
> also sprach James Westby <jw+debian at jameswestby.net> [2009.05.10.1523 +0200]:
> > With current practices there is effectively a "Debian upstream" branch
> > from the VCS, which contains thes history of the tarballs that made up
> > the source packages.
> > 
> > Even when building from a VCS it is possible, and I would say 
> > encouraged, to maintain this branch.
> 
> I've never done so, and I've never understood the need, but in the
> light of the autotools discussion, it's starting to dawn on me.
> 
> Is this a separate branch with one commit per unpacked tarball, or
> is this a branch which gets upstream merged and then the tarball
> imported, such that the diff of that commit would effectively add
> all the changes and files introduced in the act of 'make dist' or
> 'make tarball'?

The way I maintain it is a branch with a revision per tarball used,
as you suggest. The first parent of each revision is the revision
of the previous tarball. If the public VCS is known, then a link
to the revision this tarball corresponds to is a second parent.

Also, the revisions upstream releases from may not make a clean
ancestry. If they branch per-release, and stabilise it, and then
release from that branch, then the above method will thread these
revisions together in the history of the packaging branch, which to
me is conceptually what is happening, and gives the diffs you
would expect. (packaging->tarball is the packaging changes,
tarball->previous tarball is the things that changed upstream, including
generated file changes, tarball->upstream is the addition of 
auto-generated things)

> > The above ensures that you get repeatability with the pristine-tar
> > information in case you need a -2 upload, and correctly represents
> > what happened in the case where there was some transformation done
> > to the VCS tree to get the tarball, as this transformation will be
> > in the diff between the imported tarball revision and the revision
> > that it was taken from.
> 
> Good point, but I am not sure it's 100% necessary. pristine-tar is
> necessary, at least in the Debian context, but whether we have its
> xdiffs, or real diffs, I am not sure which to prefer.

My scheme has both. The xdiff which is invisible to the user, and the
diff that they can query in the normal fashion with their VCS if
they wish.

>  We're dealing
> with auto-generated files after all. pristine-tar abuses Git for
> storing the information necessary to recreate the auto-generated
> files, but it's doing so in a way that is not intrusive to the user.
> Asking the user to maintain a branch with auto-generated files does
> not have a clear-cut benefit for me.

Who says they have to maintain it. It is possible for the tool to
maintain it for them by bundling a few VCS operations in to one command,
with the underlying VCS operations being possible to do manually if
required. This is much the same as what pristine-tar does, only it
uses VCS operations that aren't obvious from its function for git,
and tools already automate its use.

> Am I just not seeing it?
> 
> > I am aiming to make this process very easy in bzr-builddeb, as
> > I feel it should be *the* way to pull in new upstream changes.
> > (Upstream has released a tarball from a tag? Use that tarball
> > rather than a generated one. Upstream has no public VCS? Just
> > import the tarball without the extra parent.) The above, with the
> > exception of a transformation of the tarball, is all a single
> > command in the latest versions.
> 
> I see your argument and the reasons for limiting the interface
> between distro and upstream to release tarballs, but doesn't that
> get in the way of cooperating on code and sharing patches, a benefit
> of situations where upstream and packagers use the same (D)VCS?

This I don't understand. I don't think I outlined a scheme where
co-operation was limited to tarballs, and I'm surprised you think
I would do such a thing.

All my scheme proposes is that there is always an extra "branch"
between upstream and the distro that contains a representation of
a tarball.

You can still branch upstream to make changes, and propose them for
upstream inclusion while still merging them in to the distro branch.
Yes, you have an issue if you say modify configure.ac that requires
some way of getting the changed configure script, however that's the
same issue you have now.

If your wish is to remove that question, and always regenerate autotools
at build time, then that's no problem, you can still do that. If you
never want to see generated autotools files before they are generated at
build time by the distro then I think you are going to struggle.

Thanks,

James