[Pkg-xen-devel] debian-xen git workflow

Mon Apr 30 23:59:32 BST 2018

Hi Hans,

Thanks for the detailed message, while I haven’t actually done any work on Xen in Debian (yet, well, aside from that small UEFI stuff), it does make a lot of sense to me to use the style currently used in Salsa, the old(er) system doesn’t track well with upstream indeed, and since Debian uses the same structure for the kernel packages I don’t see why continuing with the older style helps anyone. 
From my perspective: good job!

John

> On 30 Apr 2018, at 21:36, Hans van Kranenburg <hans at knorrie.org> wrote:
> 
> Hi Ian, team,
> 
>    In the week of Feb 23 (time flies!:), we had a very productive day
> \o/ working on the stretch security/stable package and some irc
> discussions in oftc #debian-xen which ended with an open end.
> 
> Ian, you were a bit surprised that 1) my debian-xen repo started with an
> import of a 4.8 package 2) it was not a git-pbuilder style repo with
> upstream source included, and asked me to at least make a conscious
> decision about why not doing those two things instead.
> 
> To start with the conclusion: I indeed want to continue in the same way
> as the current repository on salsa is set up, since I believe it fits
> the type of package and work (and workflow for contributors) quite well.
> 
> Summarizing that's:
> * packaging-only repo
> * debian changelog version is an unambiguous reference to an upstream
> version/commit
> * generated files (4.x version dependent) are in the git ignore
> * branching done like explained at
> https://salsa.debian.org/xen-team/debian-xen/wikis/home
> 
> I'd appreciate if you just would let me do this for a while and see what
> track record we achieve while rolling forward.
> 
> The rest of this wall of text is a collection of thoughts and things
> that happened in the last months. Feel free to read or ignore (but not
> ask questions which are answered here already) or let me know how I
> could have provided the same information using less words.
> 
> -------- >8 --------
> 
> For Xen packaging, there are two different types of work running in
> parallel:
> * Following the latest greatest releases and putting them in Debian
> unstable (yes, there's actually an audience for that, and for
> stable-backports)
> * Provide fixes for security and stability issues for Debian stable.
> 
> Also, for Xen packaging, there are two peculiarities which make it look
> a bit different from many other packages in Debian:
> * Instead of simple a linear development model, upstream supports
> multiple releases in parallel (e.g. 4.8, 4.9, 4.10). There is no path
> forward from a stable branch to the next release (e.g 4.10-stable ->
> 4.11) in upstream version control.
> * There are quite some binary packages, and a bunch of them have a
> version number in them, which changes when moving to a new 4.x release.
> 
>    In Nov/Dec 2017, the first thing I tried was searching for an
> existing packaging git repo.
> 
> I could find an old repository on alioth, which had some packaging from
> 2014 and 2015, but nothing recent. Since the way of working in there was
> very different again (git-dpm I guess), and about 75% of the full commit
> history is the same set of patches over and over and over and over and
> over again, and since I could find only exacty one commit in there that
> has a bit more explanation than a single meaningless line of ("Update
> XYZ") which describes why the change is done and what it solves
> (092785a0d8), I respectfully ignored this and later archived it at
> https://salsa.debian.org/xen-team/xen-2014-2015
> 
>    Since I couldn't find anything else, I assumed the current packages
> were just "based on changing the previous version" and then uploading
> that. Ubuntu already had a Xen 4.9 package, and based on that work, the
> first thing I tried was to create a git-pbuilder style repo with
> upstream + debian packaging.
> 
> But, I quickly ran into a set of problems and inconveniences:
> * git-pbuilder can't handle the git-attributes replacement magic when
> creating an orig tar, so it always ends up with a failing build because
> of unexpected upstream changes. I couldn't figure out a way to work
> around this.
> * Having the upstream source attached to the packaging is not a good
> match for the parallel-development style repo that Xen upstream has.
> There is no path forward from e.g 4.10-stable to 4.11. Merge commits of
> upstream source make wip-rebasing, branching and merging of packaging
> changes between versions very awkward, noisy and highly meaningless.
> * The ubuntu package obviously had been constructed by modifying
> generated content (all the generated files and files with the 4.x
> version in their name) in place, and having these files around meant I
> had to rename them from 4.9 to 4.10, resulting in noisy error-prone
> non-optimal git commits with no default history for the new files.
> 
>    At that moment I discovered these version-dependent files could be
> generated, and when taking a closer look, I also discovered that the
> packaging looked much alike the packaging repo of the debian linux
> kernel team. Since I've been following and using their packaging for
> quite a bunch of years, I was pleasantly surprised about this. BOOM!
> 
> What I did directly after this is weeding out all generated content, and
> that's where the repo started.
> 
> I documented a few things in messages to the team mailing list:
> 
> https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2017-December/007118.html
> https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2018-January/007153.html
> 
> Those messages got zero feedback from anyone(!!), so I just continued
> working in that way.
> 
> It was only in March 2018 when I learned about the dgit system and
> discovered the part of the packaging for the current Stretch release.
> However, after playing around for a few hours with dgit, I wasn't able
> to construct a local repository which made much sense. What I ended up
> with was multiple initial commits and unrelated branches based without
> finding out how to combine them into a meaningful history.
> 
> There's undoubtedly a lot more to say, but this message is already too
> long :)
> 
> Let useful discussions continue, but aside from that, I would really
> like to end up with an upload of Xen 4.10 to Debian Unstable soon.
> 
> Hans
> 
> _______________________________________________
> Pkg-xen-devel mailing list
> Pkg-xen-devel at alioth-lists.debian.net
> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-xen-devel