[PKG-Openstack-devel] Migration of packaging development workflow to OpenStack project

Tue Sep 6 13:16:27 UTC 2016

On Mon, Sep 5, 2016 at 8:48 AM, Thomas Goirand <zigo at debian.org> wrote:

> On 09/05/2016 11:24 AM, James Page wrote:
> >     > package build and publication processes in the OpenStack project
> >     > infrastructure
> >
> >     No. This isn't about publication. The intend *has never* and *never
> will
> >     be* to have OpenStack infra as the place to publish packages. The
> only
> >     reason why there is a Debian repository is because we have
> >     interconnected build-depenencies. But The intend really is to
> continue
> >     to publish within Debian.
> >
> >
> > OK -  I guess that helps unblock interconnected depends without having
> > to upload to Debian first; just need to be good at communicating that
> > people should not deploy from these repositories in the real world I
> guess!
>
> IMO, this can be done by the correct commits in the install-guide.
>
> >     Now, if what is bothering you is that the packaging is hosted by
> >     OpenStack infra, then I would very much welcome producing the same
> kind
> >     of Gerrit / Zuul / Nodepool setup that OpenStack infra has for
> Debian as
> >     a whole. I've been thinking about this for more than a year already.
> And
> >     I discussed it during my talk in Debconf. Some other DDs are also
> >     interested by the idea.
> >
> >
> > Actually the only thing that is bothering me is I don't feel that this
> > change has been +1'ed explicitly by the pkg-openstack team.
>
> All of the contributors that I know of, except the Canonical server
> team, are ok with the move. It'd be awesome if you joined. I'd
> immediately make yourself and Corey as core (which is needed for things
> like merge commit and tag push). If you think David should have core
> rights, up to you (but he contributed less than you and Corey).
>
> >     So far, you haven't given any reason but "it feels like it's done
> >     outside of Debian/Ubuntu", which doesn't feel right to me. The
> feeling I
> >     have is that this move doesn't fit in Canonical's agenda. Of course I
> >     have no way to know, and this is just a double-guess gratuitous
> >     sentence.
> >
> > It is.  These views are my own as someone who has worked across Ubuntu
> > and Debian (and OpenStack) for the last 6 years - don't read any
> > commercial agenda into this.
>
> If I was wrong here, then great. I'm sorry if I had the wrong feeling. I
> just couldn't hold this for myself, and I had to put this on the table.
> As we've been working together for so long, you know I'm a very direct
> person, I very quickly feel uncomfortable when I keep things for myself.
> I'm happy we've cleared that out.
>
> > I'm just super keen that we keep the right
> > communities engaged in packaging of OpenStack (and its large dependency
> > set) across both distributions, leveraging the facilities that both
> > distributions have with regards to review/CI etc...
>
> The only thing that bothers me if moving to OpenStack upstream CI, is
> loosing the support for non-amd64 arch. Previously, I had a Jenkins that
> was building for arm64, with an instance provided by Linaro. I'm really
> not sure how to keep this.
>
> Canonical has PPC64 and arm64 clouds, right? Do you think it'd be
> possible to leverage these? Otherwise, I could ask Linaro to provide us
> with a small pool of VMs which we'd put under nodepool. Though last
> time, it looked like that's not what they are used to provide, and I'm
> not sure if they'd accept. And we still wouldn't have PPC64. The other
> option is to do a receive hook in git.openstack.org, so that it would
> trigger the builds in jenkins. This would have the added benefits that
> it'd be easy to control for me, but I'd still be the only root admin
> with Ivan, so that's not ideal wither. Suggestions here are welcome.
>
> > For OpenStack in Ubuntu, we use the development distribution in a
> > different way to how you do in Debian; its very much the focus of
> > development for packaging, with CI between proposed and release pockets,
> > in much the same fashion as for Debian unstable->testing, albeit we also
> > hook autopkgtests into that process, gaining function regression testing
> > when dependencies of a OpenStack package change as well.  As a result we
> > CI the entire distro, not just individual changes to a single package.
>
> Ondrej proposed to run autopkgtest after a successful build. It was even
> implemented in our Jenkins at some point. Maybe we could enable this in
> OpenStack infra as well. That, and piuparts too. There's in fact a long
> list of things which I would like to add as checks in the infra build
> CI, as voting (or not) checks.
>
> > TBH whether pkg-openstack moves its dev process under git/gerrit or not
> > does not impact Ubuntu hugely anyway; the project was setup to be
> > loosely couple with Debian, so delta could be maintained where it was
> > felt it was important still;  I'm just concerned that we create a third
> > source of packaging truth, this may confuse contributors and users more,
> > even if it enables them to contribute more easily. There is no
> > guarantee that a change accepted into the OpenStack packaging
> > repositories will make it directly into either Debian or Ubuntu, as both
> > distro's may have to flex around other distro-centric concerns.
>
> How is this different from a git commit in Alioth? IMO, that's the exact
> same problem: there's no guarantee that a commit on Alioth is going to
> be uploaded. It's just Canonical's team and other PKG OpenStack
> contributors that are doing the work.
>
> At present moment, the packaging on OpenStack infra is what we've done
> until Newton b2. There's no new commits, except in openstack-pkg-tools,
> but that is contained in the build-tools/pkgos-infra-* scripts which are
> used *only* for the setup of the build system in infra. I do intend to
> upload to Debian all changes I'll do in Gerrit. We have to establish a
> workflow for it. In fact, we haven't defined a workflow for what we've
> done so far in Alioth, so it'd be good if we did, explicitly, this time,
> rather than just instinctively.
>
> Another thing is that by uploading to Debian, we also receive
> archive-wide tests, like full package rebuilds, and FTBFS reports. So
> that's really worth it too.
>
> > I've made my views clear; however I'm also just one person with the
> > team, and if the pkg-openstack team as a whole feel that this is the
> > right move to make, I'm not going to be the fly in the ointment; we are
> > a team within Debian , not a dictatorship after all!
> >
> > I'm just concerned that there is a ot of implicit consent being assumed
> > right now, and that does not feel right.
>
> Let's consider who contributed (irc nicks in brackets):
> - Ondrej (onovy)
> - Gustavo (gfa)
> - Ivan Udovichenko (tlbr)
> - Igor Yozhikov (igorYozhikov)
> - Daniil (not sure what's his Freenode/OFTC nick...)
> - Corey (coreycb)
> - David (ddellav)
> - You (jamespage)
> - Me (zigo)
>
> Let's consider everyone, one at a time, then.
>
> - Gustavo isn't contributing much these days, only to openstackclient.
> Even if I'm sorry about it, I don't think we should put the focus on his
> opinion. Gustavo, feel free to express yourself anyway.
>
> - Ondrej already expressed that he's happy about the move.
>
> - Ivan, Daniil and myself are *very much* for the move.
>
> So the only remaining people would be Canonical server team. So far, you
> were the only person that expressed an opinion. I'd really like reading
> David and Corey about this.
>

For my work personally, this is a step backwards.  If I have to get a +2 on
every package change I make, it's going to slow down my workflow
significantly.  For example there are many times during the cycle where we
sprint on openstack dependencies, and it's normal to get 10-20 packages
pushed/uploaded in a day and then move on to the next task.  If I have to
wait on +2's for all those deps now, then I'm moving backward and this is
slowing down my workflow.

My workflow goes from:

1) clone repo
2) make change
3) push repo
4) upload

To:

1) clone repo
2) make change
3) initiate review
4) ping core dev for +2
5) wait for merge
6) possibly repeat steps 4+5
7) upload

>
> Then, there's all other contributors, interested in a single project,
> like this person today on IRC, about Magnum, the Melanox guys, etc. All
> of them were either happy about such a move, or didn't express any opinion.
>
> > AFAICT this is the first time the move has been discussed directly on
> > pkg-openstack; I appreciate that we've has some conversations at the
> > OpenStack summits and on the openstack-dev ML, but it feels like the
> > proposed processes should have been formally proposed on *this* ML as
> > well so that as a team we can explicitly decide on the way we want to
> > take development forward.
> >
> > I hope that clarifies my position and the intent of my initial email.
>
> Thanks for opening the topic if you thought it was needed. I didn't
> think it was, probably because I was too excited about the idea.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
>
> _______________________________________________
> Openstack-devel mailing list
> Openstack-devel at lists.alioth.debian.org
> http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/openstack-devel
>

-- 
Regards,
Corey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/openstack-devel/attachments/20160906/70e394c8/attachment-0001.html>