[PKG-Openstack-devel] Improving the infra CI/CD for Deb packaging

Thomas Goirand zigo at debian.org
Wed Feb 22 17:37:17 UTC 2017

On 02/22/2017 01:23 PM, Ondrej Novy wrote:
>     >  3. it's always not possible to have one commit per change (i need to
>     >     git commit --amend after merge)
>     Could you expand on this? I don't understand why there's a problem, and
>     why one would need to do a --amend after merge.
> ehm, https://wiki.openstack.org/wiki/DEB-packaging#Releasing_a_new_upstream_release_package
> You MUST use --amend after git merge.
> Cite: Note that the fix of debian/changelog and debian/control should be
> done in the same single commit as the merge commit, otherwise the
> package will fail to build.
> So in 1 commit I need:
>   * merge upstream
>   * change d/changelog
>   * change d/control for deps
>   * add d/patches which fixies build
> This is crazy. I want to do as small commits as possible, at least one
> commit = one changelog entry.

In which way does this differ from what we had on Alioth + Jenkins build
server? (This is retorical, let me answer) Well, before, we could push a
merge, but then there was a chance the package couldn't build. Now, we
make sure than 100% of the time, the package *can* build. So IMO it's a
very good thing that now, the merge commit must contain fixes so the
package continues to build. On top of this, we have a consistent git
history, were we always can build.

>     >  4. it's hard to manage deb-auto-backports. It took me a day to fix
>     >     auto-backports just to upload new Swift version
>     True. Though it has the huge advantage that it keeps backports
>     up-to-date, which was never the case with the hack system of doing it by
>     hand. Also, you never had to take care of it, because *I* was doing it
>     by hand, sometimes cheating and just ignoring build problems (shame on
>     me). It was even more painful by hand...
> but that's theory. I waited ~ week for fix and then i fixed it myself.

I don't remember what that was. Can you refresh my memory?

> We can't depend on single man to do critical job. We need solution which
> anyone can fix :). Current solution is hackish. Example: If you need to
> add new package for auto backports, every other package is checked for
> update and if there is update, it gets updated.

This is a very good thing, so we stay current. Meaning that if there's a
security issue fixed in Sid, we get it too. The original plan was to
rebuild all packages as a periodic job. It is my opinion that such a
periodic job should be setup, because otherwise we may miss some
security patches and bugfixes.

Yes, it's inconvenient, and sometimes even painful. Though I don't think
we have a choice here.

> Next: We should
> prefer depedency packages from official jessie backports if they are
> newer and (auto-)remove old one from our repo.

There's 2 options for us to use: --download-from-jessie-backports (which
I used for a few packages which we couldn't build), and -d
jessie-backports. If you and everyone else believe it's the best option
for us, then I don't have any objection to use extensively the -d
jessie-backports (which I prefer over --download-from-jessie-backports
so we check if the package can be built). The reason why I think it's a
good idea, is because this way, we take advantage of the maintenance
work done within Debian directly, which is a very good thing.

We could even upload 3rd party libs to official jessie-backports first,
and *then* use the -d jessie-backports if we believe it's a good
workflow, however, this would mean waiting for a dak run and mirror
sync, which could take up to a day. I'd say this last option is for the
cases were we really need specific stable-backports patch.

Please do answer here and voice your opinion.

>     >  5. it doesn't build packages for unstable - which is important,
>     because
>     >     we are uploading to unstable and instead it builds/checks Jessie
>     >     backports (which is less important)
>     We could add that feature. Also, it is my view that one should attempt
>     an sbuild build on Sid before sending a CR. With the old system, we also
>     had the issue.
> yep i can say anyone should do sbuild on Jessie backports before doing
> CR too. So why we have gating for Jessie bpo?

stable-backports has always been what Debian users are consuming. At
least, it was my view that it was how it should have been. Also because
Sid *does* break from times to times (think Python 3.5 transition, for
example...), so it's harder to keep that gating stable.

> Because gates are here for
> checking if people are doing correct thing. And because we are uploading
> to Sid, we must be sure it builds on sid (with deps package only from sid).

I also agree we should be checking with Sid, probably as non-voting
first, and see how it goes.

> so:
> 8. add repo for Ocata

Hopefully, Alison will be able to corner someone from infra this week,
and make them create the ocata package repos.


Thomas Goirand (zigo)

More information about the Openstack-devel mailing list