Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 05/27/2015 10:14 AM, Thomas Goirand wrote:
> Hi all,
> 
> tl;dr: - We'd like to push distribution packaging of OpenStack on
> upstream gerrit with reviews. - The intention is to better share
> the workload, and improve the overall QA for packaging *and*
> upstream. - The goal is *not* to publish packages upstream -
> There's an ongoing discussion about using stackforge or openstack. 
> This isn't, IMO, that important, what's important is to get
> started. - There's an ongoing discussion about using a distribution
> specific namespace, my own opinion here is that using
> /openstack-pkg-{deb,rpm} or /stackforge-pkg-{deb,rpm} would be the
> most convenient because of a number of technical reasons like the
> amount of Git repository. - Finally, let's not discuss for too long
> and let's do it!!! :)
> 
> Longer version:
> 
> Before I start: some stuff below is just my own opinion, others are
> just given facts. I'm sure the reader is smart enough to guess
> which is what, and we welcome anyone involved in the project to
> voice an opinion if he/she differs.
> 
> During the Vancouver summit, operation, Canonical, Fedora and
> Debian people gathered and collectively expressed the will to
> maintain packaging artifacts within upstream OpenStack Gerrit
> infrastructure. We haven't decided all the details of the
> implementation, but spent the Friday morning together with members
> of the infra team (hi Paul!) trying to figure out what and how.
> 
> A number of topics have been raised, which needs to be shared.
> 
> First, we've been told that such a topic deserved a message to the
> dev list, in order to let groups who were not present at the
> summit. Yes, there was a consensus among distributions that this
> should happen, but still, it's always nice to let everyone know.
> 
> So here it is. Suse people (and other distributions), you're
> welcome to join the effort.
> 
> - Why doing this ================ It's been clear to both
> Canonical/Ubuntu teams, and Debian (ie: myself) that we'd be a way
> more effective if we worked better together, on a collaborative
> fashion using a review process like on upstream Gerrit. But also,
> we'd like to welcome anyone, and especially the operation folks, to
> contribute and give feedback. Using Gerrit is the obvious way to
> give everyone a say on what we're implementing.

I was impressed to know that Ubuntu and Debian do not share packaging.
You guys obviously should do it! A fork is always better than a new
wheel. Kudos for the initiative.

> 
> As OpenStack is welcoming every day more and more projects, it's
> making even more sense to spread the workload.
> 
> This is becoming easier for Ubuntu guys as Launchpad now understand
> not only BZR, but also Git.
> 
> We'd start by merging all of our packages that aren't core
> packages (like all the non-OpenStack maintained dependencies, then
> the Oslo libs, then the clients). Then we'll see how we can try
> merging core packages.
> 
> Another reason is that we believe working with the infra of
> OpenStack upstream will improve the overall quality of the
> packages. We want to be able to run a set of tests at build time,
> which we already do on each distribution, but now we want this on
> every proposed patch. Later on, when we have everything implemented
> and working, we may explore doing a package based CI on every
> upstream patch (though, we're far from doing this, so let's not
> discuss this right now please, this is a very long term goal only,
> and we will have a huge improvement already *before* this is
> implemented).

So Delorean (aka RDO master, but now also RDO/kilo) already maintains
its packages on github [1] and has a gerrit review in place (f.e. see
[2]) and even CI (f.e. see [3]). It would be so great to see those
packages moved under the official tent and have CI running inside our
own infra.

I even thought that the idea was to start on github but then move into
stackforge/, so your initiative would be a good trigger for RDO
magicians to start working on moving under the tent.

I hope that other distributions can reuse all that is achieved by RDO
project in this regard so that tooling and workflow is shared.

For me personally, moving Delorean packages to official infra would be
a time saver since I wouldn't be forced into using different gerrit
and CI installations and other dev scripts around git to contribute to
upstream and packaging.

[1]: https://github.com/openstack-packages
[2]: https://review.gerrithub.io/#/q/project:openstack-packages/ceilomet
er
[3]:  https://prod-rdojenkins.rhcloud.com/job/delorean-ci/335/

[...skip...]

> - Let's keep efficiency in mind =============================== 
> Over the last few years, I've been able to maintain all of
> OpenStack in Debian with little to no external contribution. Let's
> hope that the Gerrit workflow will not slow down too much the
> packaging work, even if there's an unavoidable overhead. Hopefully,
> we can implement some liberal ACL policies for the core reviewers
> so that the Gerrit workflow don't slow down anyone too much. For
> example we may be able to create new repositories very fast, and it
> may be possible to self-approve some of the most trivial patches
> (for things like typo in a package description, adding new debconf
> translations, and such obvious fixes, we shouldn't waste our
> time).

I can talk here for RPM packages (RDO, Delorean). For neutron packages
that I care about, we have multiple engineers that also work on
neutron in upstream that also care about the packaging, providing
reviews and patches for the latter. I guess other teams may be
understaffed, as Debian one, but it all depends on the distro and the
package. I think current distro maintainers should decide what
governance model applies to their situation better.

> 
> There's a middle ground between the current system (ie: only write 
> access ACLs for git.debian.org with no other check what so ever)
> and a too restrictive fully protected gerrit workflow that may slow
> down everyone too much. Currently, there's a small amount of people
> involved in the packaging. While we can expect a lot of operators
> will be interested to work on core packages like Nova, Neutron and
> so on, I am at the same time expecting that nobody will care about
> a given indirect python module dependency (and I may still continue
> to be the only one working on these...). We really don't want to be
> stuck in situations with nobody reviewing these.
> 
> - /stackforge or /openstack namespace 
> ===================================== There's been this question
> floating around. Should we try joining directly as part of
> OpenStack, as many suggested, or should we go through a first round
> (during one dev cycle, until Liberty is released?) through
> stackforge.
> 
> I have to admit that I'm not strongly opinionated about this
> myself. Sure, being an official OpenStack big-tent project would be
> nice, and it would feel worm, but also there's the issue that we
> don't have any commits within the project yet, and therefore
> finding who's core and who's the PTL could be an issue if we want
> to follow the official guidelines.
> 
> However, without even discussing the topic between us, it's obvious
> that people like James Page (plus others from his team) and myself
> would be core approvers for the Debian packaging. Fedora guys can
> decide for themselves (I must write names I have in mind for the
> RPM based-OS side of things: M. Runge, Haikel Gemmar and Alan
> Pevec, for example. Please add the missing ones here...).
> 
> Another thing is that going for stackforge now means that one day,
> we'll have to suffer from a migration to the openstack namespace,
> which may be 1/ a lot of work and 2/ disruptive. Avoiding such a
> migration would be good.
> 
> All together, the packaging team will happily accept whatever is
> decided by the TC. What counts for us is that a decision is taken
> quickly, so that we can start implementing it and share our
> repositories. So, dear TC, please let us know ASAP after your next
> meeting.

I would vote for the stable namespace, whatever it is, and wouldn't
bother staying in stackforge/ indefinitely. If people will eventually
try to get into openstack/ for whatever reason, then let's just start
with that namespace to avoid the switch hassle later.

> 
> Anyhow, please feel free to discuss the issue within the
> governance patch which I started: 
> https://review.openstack.org/#/c/185187/
> 
> - Specific packaging namespace ============================== Since
> distribution packaging through Git implies running a lot of git 
> repository (one per package), it's been discussed that we may want
> to use a specific namespace for distributions. For example, 
> /stackforge-packaging or /openstack-packaging.
> 
> Let's keep in mind that, for OpenStack packaging by the
> distributions, we're talking about more than 200 packages. 235 so
> far on the OpenStack packaging group in Debian [1], and it's
> increasing all the time...
> 
> Personally, I believe it'd be nice to have a specifc namespace for 
> distributions, for multiple reasons. The first one is that we
> already have a lot of projects inside /stackforge or /openstack,
> and that searching through them isn't easy. Now, consider that
> we'll have duplicates for each and every project. Also, we don't
> have a complete match between OpenStack package names. For example,
> in Debian/Ubuntu, we have "python-" as prefix for each and every
> python libraries (including for example all oslo packages, but not
> only). Plus we'd be forced to use a prefix, for example "pkg-nova",
> to avoid collision. However, it'd be nice to keep the same
> repository names as per the name of the source package in the
> distributions, otherwise we'd have to fix our tooling which
> obviously will become slightly more complex.

I also think that a separate namespace would be better for indexing
and searching reasons.

> 
> - Distribution specific git repositories or specific branches 
> ============================================================= 
> There's 2 issues if we keep the same repository names for multiple 
> distribution.
> 
> First, we'd like to have a different set of core reviewers for a
> given distribution family. This implies that, if we use the same
> repository for RPM and DEB, then we'd have to set ACLs on
> per-branch basis, which may be harder to maintain (compared to
> giving access to the full set of Git repositories for a given
> distro family). Yes, we could give access to both for core
> reviewers, and trust them to behave. This trust model has been
> proved as successful within the Debian community, but still, adding
> and removing ACLs for new core reviewers would be harder to
> maintain.
> 
> Then, as written above, it is desirable to have a matching pair
> between distribution source package names and the name of the
> underlying git repository (so that for building X, you git clone X,
> not prefix-X). However, distributions don't have the same names for
> packages. For example, RPM supports upper case, while dpkg systems
> don't. Also, Red Hat uses a prefix "openstack-" for all core
> projects (like "openstack-nova"), while Debian based distribution
> don't, and so on...

I wouldn't mind to store neutron rpm packages under 'neutron' repo as
long as its artifacts are named with openstack- prefix. If that would
help to get common space to collaborate cross-distro, I'm fine with that
.

> 
> - Finally ========= Finally, well, please don't discuss the color
> of the bike-shed for too long. We don't really care the color as
> long as we have a shed to share the hosting of our packages Git.

I have all kinds of color suggestions, but I will save them for later.

Ihar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJVZcdrAAoJEC5aWaUY1u57+8UH/2x5n1LSWpBVWn1twfhHxJfb
ejHNrL0YjaZjsQyH2t4sNfqkvI5BzY1DysOjEXJIpnSMgVua3Q8o097rZbRVKcGD
5SnU/SDI9dXy8jyDUPRH89Gnxq30x3wlC5SUr1PBglCq5RibqDDwl+nvdFyQodCc
mtUZ91hNG2QZ/Ittex954x1/4qGlp650YdS236HPrt6psNyYOZf1nyRx6z7CYRBc
QJX5JNs42aOvKVHw92dCKmYOXmqf4q3RFvGr+XuF1VvIdJZF+B+weKQ6lYSZfXB4
QyuRstWaULXAQuuYlFxaxN2d9cRw25bySmZSEL0s99LTo66fOv4E4oc9q47P/OQ=
=JMux
-----END PGP SIGNATURE-----