[PKG-Openstack-devel] Migration of packaging development workflow to OpenStack project

[Thomas Goirand]

On 09/05/2016 11:24 AM, James Page wrote:
>     > package build and publication processes in the OpenStack project
>     > infrastructure
> 
>     No. This isn't about publication. The intend *has never* and *never will
>     be* to have OpenStack infra as the place to publish packages. The only
>     reason why there is a Debian repository is because we have
>     interconnected build-depenencies. But The intend really is to continue
>     to publish within Debian.
> 
> 
> OK -  I guess that helps unblock interconnected depends without having
> to upload to Debian first; just need to be good at communicating that
> people should not deploy from these repositories in the real world I guess!

IMO, this can be done by the correct commits in the install-guide.

>     Now, if what is bothering you is that the packaging is hosted by
>     OpenStack infra, then I would very much welcome producing the same kind
>     of Gerrit / Zuul / Nodepool setup that OpenStack infra has for Debian as
>     a whole. I've been thinking about this for more than a year already. And
>     I discussed it during my talk in Debconf. Some other DDs are also
>     interested by the idea.
> 
> 
> Actually the only thing that is bothering me is I don't feel that this
> change has been +1'ed explicitly by the pkg-openstack team.

All of the contributors that I know of, except the Canonical server
team, are ok with the move. It'd be awesome if you joined. I'd
immediately make yourself and Corey as core (which is needed for things
like merge commit and tag push). If you think David should have core
rights, up to you (but he contributed less than you and Corey).

>     So far, you haven't given any reason but "it feels like it's done
>     outside of Debian/Ubuntu", which doesn't feel right to me. The feeling I
>     have is that this move doesn't fit in Canonical's agenda. Of course I
>     have no way to know, and this is just a double-guess gratuitous
>     sentence.
> 
> It is.  These views are my own as someone who has worked across Ubuntu
> and Debian (and OpenStack) for the last 6 years - don't read any
> commercial agenda into this.

If I was wrong here, then great. I'm sorry if I had the wrong feeling. I
just couldn't hold this for myself, and I had to put this on the table.
As we've been working together for so long, you know I'm a very direct
person, I very quickly feel uncomfortable when I keep things for myself.
I'm happy we've cleared that out.

> I'm just super keen that we keep the right
> communities engaged in packaging of OpenStack (and its large dependency
> set) across both distributions, leveraging the facilities that both
> distributions have with regards to review/CI etc...

The only thing that bothers me if moving to OpenStack upstream CI, is
loosing the support for non-amd64 arch. Previously, I had a Jenkins that
was building for arm64, with an instance provided by Linaro. I'm really
not sure how to keep this.

Canonical has PPC64 and arm64 clouds, right? Do you think it'd be
possible to leverage these? Otherwise, I could ask Linaro to provide us
with a small pool of VMs which we'd put under nodepool. Though last
time, it looked like that's not what they are used to provide, and I'm
not sure if they'd accept. And we still wouldn't have PPC64. The other
option is to do a receive hook in git.openstack.org, so that it would
trigger the builds in jenkins. This would have the added benefits that
it'd be easy to control for me, but I'd still be the only root admin
with Ivan, so that's not ideal wither. Suggestions here are welcome.

> For OpenStack in Ubuntu, we use the development distribution in a
> different way to how you do in Debian; its very much the focus of
> development for packaging, with CI between proposed and release pockets,
> in much the same fashion as for Debian unstable->testing, albeit we also
> hook autopkgtests into that process, gaining function regression testing
> when dependencies of a OpenStack package change as well.  As a result we
> CI the entire distro, not just individual changes to a single package.

Ondrej proposed to run autopkgtest after a successful build. It was even
implemented in our Jenkins at some point. Maybe we could enable this in
OpenStack infra as well. That, and piuparts too. There's in fact a long
list of things which I would like to add as checks in the infra build
CI, as voting (or not) checks.

> TBH whether pkg-openstack moves its dev process under git/gerrit or not
> does not impact Ubuntu hugely anyway; the project was setup to be
> loosely couple with Debian, so delta could be maintained where it was
> felt it was important still;  I'm just concerned that we create a third
> source of packaging truth, this may confuse contributors and users more,
> even if it enables them to contribute more easily. There is no
> guarantee that a change accepted into the OpenStack packaging
> repositories will make it directly into either Debian or Ubuntu, as both
> distro's may have to flex around other distro-centric concerns.

How is this different from a git commit in Alioth? IMO, that's the exact
same problem: there's no guarantee that a commit on Alioth is going to
be uploaded. It's just Canonical's team and other PKG OpenStack
contributors that are doing the work.

At present moment, the packaging on OpenStack infra is what we've done
until Newton b2. There's no new commits, except in openstack-pkg-tools,
but that is contained in the build-tools/pkgos-infra-* scripts which are
used *only* for the setup of the build system in infra. I do intend to
upload to Debian all changes I'll do in Gerrit. We have to establish a
workflow for it. In fact, we haven't defined a workflow for what we've
done so far in Alioth, so it'd be good if we did, explicitly, this time,
rather than just instinctively.

Another thing is that by uploading to Debian, we also receive
archive-wide tests, like full package rebuilds, and FTBFS reports. So
that's really worth it too.

> I've made my views clear; however I'm also just one person with the
> team, and if the pkg-openstack team as a whole feel that this is the
> right move to make, I'm not going to be the fly in the ointment; we are
> a team within Debian , not a dictatorship after all!
> 
> I'm just concerned that there is a ot of implicit consent being assumed
> right now, and that does not feel right.

Let's consider who contributed (irc nicks in brackets):
- Ondrej (onovy)
- Gustavo (gfa)
- Ivan Udovichenko (tlbr)
- Igor Yozhikov (igorYozhikov)
- Daniil (not sure what's his Freenode/OFTC nick...)
- Corey (coreycb)
- David (ddellav)
- You (jamespage)
- Me (zigo)

Let's consider everyone, one at a time, then.

- Gustavo isn't contributing much these days, only to openstackclient.
Even if I'm sorry about it, I don't think we should put the focus on his
opinion. Gustavo, feel free to express yourself anyway.

- Ondrej already expressed that he's happy about the move.

- Ivan, Daniil and myself are *very much* for the move.

So the only remaining people would be Canonical server team. So far, you
were the only person that expressed an opinion. I'd really like reading
David and Corey about this.

Then, there's all other contributors, interested in a single project,
like this person today on IRC, about Magnum, the Melanox guys, etc. All
of them were either happy about such a move, or didn't express any opinion.

> AFAICT this is the first time the move has been discussed directly on
> pkg-openstack; I appreciate that we've has some conversations at the
> OpenStack summits and on the openstack-dev ML, but it feels like the
> proposed processes should have been formally proposed on *this* ML as
> well so that as a team we can explicitly decide on the way we want to
> take development forward.
> 
> I hope that clarifies my position and the intent of my initial email.

Thanks for opening the topic if you thought it was needed. I didn't
think it was, probably because I was too excited about the idea.

Cheers,

Thomas Goirand (zigo)