[Debian-ha-maintainers] shared Git repository layout

[Ferenc Wagner]

Richard B Winters <rik at mmogp.com> writes:

> On Tue, 2015-04-14 at 13:01 +0200, Ferenc Wagner wrote:
>
>> And I don't think gbp patch-queue branches should usually
>> be published at all, as they are rebased frequently.
>
> No probably not, but please allow me to explain:
>
> - The upstream branch was created as follows:
>
> 1. git checkout --orphan upstream
> 2. git pull upstream/master

Calling the upstream origin "upstream" makes sense, but also creates
some opportunity of confusion with the upstream branch.

> 3. git reset --hard 3f7de65 (resets to latest release commit based on
> release tarballs commit hash)
>
> So as you can see upstream is a fork of the real upstream with all of
> it's history.

Yes, but this is not what I meant.  In my interpretation of DEP-14,
the master branch tracks the upstream master (with full history), and
the upstream branch registers the released tarball contents.  In
general, the released tarball at a specific commit does not (only)
contain the tree from that commit: it contains other files, too
(autotools products, .tarball-version, etc.)  On the one hand, having
the tarball contents easily accessible on a branch can be useful, and on
the other, pristine-tar works best when given a tree very close to the
one contained in the tarball.

> The next step is to create the main debian branch...
>
> 1. git checkout --orphan debian

The packaging branch shouldn't be an orphan, it's more logical to branch
it off the upstream branch (and merge later upstream imports into it).

> 2. copied debian directory into branch after deleting all files staged
> in commit.

And the packaging branch should contain the full unpacked Debian source
package, not its debian directory only.

> you can't make a branch with a '/' in its name...

I can: git checkout -b debian/sid.  Works without a debian branch.
(Reminder: DEP-14 suggests git symbolic-ref HEAD refs/heads/debian/sid.)

> so you make a main 'debian' branch..then branch the other branches
> when necessary (i.e.  debian/sid etc). I simply haven't needed to do
> that yet.  I can branch from the current debian into debian/sid in a
> matter of seconds when we want - anyone of us can.

Sure.  It usually goes like this: you release version A from debian/sid,
and create the tag debian/A.  Eventually the produced packages reach a
stable release (say wheezy).  Some time later wheezy is frozen.  Then
you release version B from debian/sid (tagging debian/B).  Then you have
to fix a security issue in wheezy, so you branch off debian/wheezy at
tag debian/A, commit your fix there and release A+deb6u1.  Then you
decide to backport version B to wheezy, so you branch off
debian/wheezy-backports at tag debian/B, do your backporting there and
release B~bpo70+1.  And so on...

> The next step was using 'gbp-pq' to import the patches into separate
> branches, importing your patches into branches let's us now easily
> offer up those patches for cherry-picking to upstream.

Upstream can even pull from the patch-queue branches if we merge the
upstream master into our packaging branches (through the upstream
branch).  See for example the following documentation
http://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.html#GBP.IMPORT.UPSTREAM.GIT.TARBALL
(where the packaging is done on the master branch, this is pre-DEP-14
work).

> And the last step, is to prep the master branch for packaging work:
>
> 1. git checkout upstream
> 2. git checkout --orphan master
> 3. git add .
> 4. git commit -m 'Initial upstream import'
> 5. git read-tree --prefix=debian/ debian  (this gets the debian branch
> read into the debian directory in master)
>
> Now we can work in master until the package is ready, then we create a
> tag for master.

This is something I'd rather not do, see above.

> Another thing I should have done ahead of time, was to create a v0.17.1
> tag for the upstream branch commit used for the release (which is the
> commit used for the upstream release tarball). This I will do.

This is something git-import-orig does automatically:

$ git-import-orig --debian-branch=debian/sid --upstream-vcs-tag=v0.17.1 --pristine-tar /tmp/libqb-0.17.1.tar.gz

> All the guides I've read suggest master has debian/<whichever package>
> merged into a sub-directory called debian. It is also how importing the
> tarball sets up the branches.

Merged into a sub-directory?  I don't follow you.  What guides are you
reading?

>> We want to base our packages on upstream tarball releases, though, so
>> we need an upstream branch and import the releases there.
>
> I don't like this, if we're forking upstream I don't want to do
> this....this seems extremely pointless. I believe we should choose 1 or
> the other....work off of the tarballs only, or work off of the
> repository and let the git tools recreate the tarball from the
> repository alone.

If you recreate the tarball from the git repository, then you obviously
use the tarballs.  This is not a choice, we do both: use the upstream
master and the upstream tarballs.

> Honestly this will fix our issue with needing the tarbal-version file
> for properly building on debian...since we have the .git directory now
> during the build.

The build daemons won't have the .git directory during the build.

>> We also want to be able to regenerate the upstream tarballs from our
>> repository alone, thus we need a pristine-tar branch as well.
>
> I think using the repository, and the specific commits used for the
> tarballs is enough, especially if we'll also leverage this pristine-tar
> branch (which should be automatically created by the git tools no?)

To commit on the pristine-tar branch, you need the pristine tarballs
from upstreams.  The upstream tarballs are certainly linked to specific
upstream commits, and we preserve this information by merging upstream
master into our upstream branch (which contains the tarball contents),
but they are not the same: the tarball contains generated files and
.tarball-version for example.

>> The packaging work should be done on the debian/sid branch (not
>> simply debian, to gain uniformity when we'll later have to do
>> backports, stable updates or experimental uploads).  Thus the debian
>> directory will only be present in the debian/* branches.  Tools like
>> git-buildpackage can be configured for this convention.
>
> debian/sid is a child of debian...you can't have a '/' character in a
> branch name...

I could.  What error message did you get?

> hence create debian first -> as this is a new repository package and a
> fresh start...once its ready for unstable, I'll create the sid branch
> off of the debian branch.  It will happen...just not there yet.

There's little point in using two branches for this.  See DEP-14, which
mentions debian/master, but is not convinced either.

>> To be honest, I've got very limited experience with this layout, but I
>> can see how this would be better than the more conventional layouts I've
>> got more experience with.  That's why I recommend the above.  
>
> I agree, I like the layout as well, but I don't like doing all that
> extra work (using upstream clone _and_ importing release tarballs).

See above.  I don't think you can reasonably help this.
-- 
Regards,
Feri.