[Pkg-salt-team] Ubuntu packaging and repository location

[Russ Allbery]

I'm not entirely sure on the context of this, but in lieu of a more
complete writeup, I can offer a bit of general advice.  This is based on
my experience as both upstream and a Debian package maintainer, including
Debian packaging for my own packages.

A few background assumptions:

* All of this assumes you're using Git and that you want to make full use
  of Git for the packaging (as opposed to having other reasons for doing
  something based on tarballs).

* As upstream, you're still interested in making regular tarball releases
  for other users (not just tagging a Git repository and calling it good).

As a matter of personal preference, I like to base packaging on the
tarball release, not just on the corresponding tag in the Git repository.
I do this even for my own software.  My rationale is that the tarball is
the public release artifact of the project, and the packaging of a
released version should have a clear relationship with it.  This is an
old-school position, and others (legitimately) disagree.  If this isn't
important to you, you can simplify my workflow somewhat.

However, note that at some point you will have to generate a tarball of
upstream source as part of the Debian source package format, and that
tarball must be identical across Debian patch releases (-1, -2, -3 of the
same upstream version).  So you want to have some way of recovering that
tarball to use as the basis of preparing new versions.  You can always
pull it back out of the archive, but I prefer to have the ability to
recreate it reliably from only the contents of the Git repository.

My workflow is basically the git-buildpackage workflow.  What I'm going to
describe is what I do when I'm both upstream and Debian package
maintainer.

First, I would separate the Debian packaging on a separate branch in the
repository rather than having a debian directory as part of the upstream
source (master branch).  *Most* of the time, you can do this either way,
but there are three big reasons why I think this is conceptually cleaner:

1. If the Debian packaging is a separate repository branch, you can also
   apply Debian-specific changes to the source on that branch when it's
   appropriate (such as backports of specific fixes from the current line
   of development that haven't appeared in a released version yet).  The
   Debian package is really a separate line of development that may have
   changes outside the packaging files.  If you keep your Debian packaging
   files on the trunk, you end up having to branch the trunk when you have
   to do this, and those branches are harder to track than just having a
   persistent debian branch.

2. If the Debian packaging is on your trunk, you'll tag it as part of an
   upstream release.  Even if you don't include it in the upstream release
   tarball, this doesn't necessarily conceptually make sense.  The Debian
   packaging isn't really part of the upstream release: it may (and often
   does) change for Debian-specific reasons after the upstream release
   without warranting another upstream release, it may be targetted at a
   version of the source with some temporary Debian-specific changes, it
   may be maintained by someone else with a different development cycle,
   timing-wise, than the upstream releases, etc.

3. It provides a cleaner point for people who need to create varient
   packages to branch from, such as for new Debian derivatives, or for the
   Ubuntu packaging.  Similarly, it lets you merge back and forth between
   a Debian branch and an Ubuntu branch, while the trunk can only (easily)
   hold one and only one set of packaging files.

It is a good idea to have the debian branch be branched off of your master
branch so that you can merge releases into it.  Generally, I merge a
release in by tag.  You can do this as a direct relationship (merge the
release tag directly into the debian branch).  However, doing this loses
the association with the contents of the tarball, which will frequently
not match anything in Git due to generated files included in the release
(such as a configure script).

Therefore, I continue to merge the master branch into the debian branch
via an intermediate upstream branch, as if I were importing tarballs from
an entirely separate project.  However, I do this via git-import-orig with
the --upstream-vcs-tag flag.  What this does is create an upstream branch
which contains a series of commits, each of which being a *merge* of the
contents of the tarball and the release tag in Git.  Then you merge the
upstream branch into the debian branch.  (git-import-orig does all this
for you.)  This means the debian branch has a well-defined descendent
relationship from your master branch (so you can merge and cherry-pick),
but is based on the contents of the release tarballs.

Again, as mentioned above, this is a personal preference; if you don't
care about the relationship with the release tarball, you can drop this
intermediate branch.

Finally, I use pristine-tar to store the necessary instructions for
rebuilding the release tarball from the upstream branch.  Again,
git-import-orig does this automatically.  This way, anyone with a copy of
the repository can use pristine-tar to reconstruct the byte-for-byte
identical copy of the tarball on which the packaging is based.  You will
want to do this regardless of whether you use the release tarball as the
basis for packaging, since you will want to be able to reconstruct the
tarball that you used for packaging when creating new Debian -2, -3, etc.
releases.

So, the overall workflow looks like this:

1. Upstream release is made and tagged in Git from the master branch (or
   whatever other upstream branch structure) and contains no Debian
   packaging files.

2. Debian packager obtains the release tarball and imports it into the
   repository as a merge with the release tag using git-import-orig with
   the --upstream-vcs-tag flag.  This updates the upstream branch, creates
   an upstream/* tag used by the Debian packaging, and merges the contents
   of the new tarball into the debian branch.

3. Update the Debian packaging as needed for the release and commit those
   changes.

4. Build with git-buildpackage --git-debian-branch=debian (you probably
   want to put that in .gbp.conf so that you don't have to keep specifying
   it), test, etc.

5. Add the URL of the repository to Vcs-Git with the -b debian flag so
   that anyone using debcheckout knows that you're using the debian branch
   for the packaging.

6. When you have something you're happy with, tag the debian branch with a
   debian/<version> tag.  (Keeping it out of the namespace of any other
   project tags.)

7. Further Debian revisions are done on the debian branch, tagged with
   debian/* tags, etc., independent of master development, although you
   can cherry-pick commits and so forth as needed.

8. When the next release happens, git-import-orig with --upstream-vcs-tag
   again.

Note that, once you follow this model, it really makes no difference
whether the Debian packaging is done in a separate repository or the same
repository as upstream.  It's all the same to Git since it's all separate
related branches; the answer can even be "both" by just pushing the
repository to multiple places.  I personally don't feel any particular
need to mirror my packaging repositories on git.debian.org when I'm
upstream, so I just use the upstream Git repository.  But it's easy to do
either as you prefer.

One other wrinkle is that if you use the 3.0 (quilt) Debian source package
format, which in general I would recommend, you have to figure out how to
represent the changes to the upstream source outside of the debian
packaging (from Debian-specific short-term changes, cherry-picked changes
from the current development, etc.).  There are various complex things you
can do to maintain separated patches in debian/patches, but I think
they're all pointless complexity if the packaging is being done in close
cooperation with upstream.  What I do is create
debian/source/local-options with "single-debian-patch" and then a
debian/source/local-patch-header file with a Debian patch header that just
says "these are all the changes relative to upstream; see Git for the
separated commits."  This lets you use 3.0 (quilt) for its other features
(xz support, less ambiguity, etc.) while still just building one simple
patch file with all the changes relative to upstream and not spending a
lot of time trying to maintain separate patches.  Naming those files with
local- means that they won't be included in the package as uploaded, so
other people's changes (NMUs, etc.) will get separate patch files per
upload, which will make it easier to incorporate those changes later.

-- 
Russ Allbery (rra at debian.org)               <http://www.eyrie.org/~eagle/>