Open Stack

On 03/19/2016 11:10 AM, Monty Taylor wrote:
> The patch looks good, but it conflicts with the
> move-nova-jobs-to-db-macro and the "add debian jessie support for bindep
> fallback" change. Rather than fighting the rebase fight, let's put a
> brief hold on this (sorry, I know) and land it as soon as those land.

Ok.

It'd be nice if someone could ping me when I can resume my work on it,
if you guys are following closely the other 2 patches (if not, I'll try
to remember to check for them).

> We'll need to work to make sure that we're using zuul-cloner to get the
> right things checked out

Is this a macro the package build needs to use?

> Luckily, we've got and apt-repository infrastructure already set up and
> ready in infra thanks to the mirror work we did this last cycle. It's
> using reprepro fwiw.

Outch ! I had multiple very bad experiences with reprepro. For example,
if we rebuild the same package with the same version, reprepro will
*not* pick-up the package. And we do need to do this, because the
debian/changelog needs to match what is uploaded to the Debian archive,
and it cannot increment.

The thing is, maintaining a Debian repository can be done with a very
small shell script like this one:

http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/build-tools/pkgos-scan-repo

Hopefully, we can switch to something like this that allows more control
than reprepro allows.

> I believe we should add jessie to the list of things we mirror in it,
> and then also add a volume to hold things we publish ourselves.

These are typically "one off" backports which we don't need to care care
much of, but which are needed for other OpenStack to build or run.
Probably something based on a yaml file listing all the packages could
be enough.

> We'll also need to move from having an unsigned reprepro to a signed
> reprepro if we're going to publish our own packages. We've not been
> signing the repo so far because we've sort of wanted to discourage use
> of our mirror outside of the gate - but it turns  out our mirror is
> AMAZING - so I think it's time we change that.

Ok. The script I listed above signs the repo, it's not hard to do, as
you probably know already.

>> Finally, we'll need a way to build backports from Sid and also publish
>> them.
> 
> Hrm. We might want to mirror sid then too. I'd like to talk about the
> backport building process - hopefully a process that does not need to
> require us making a repo in gerrit for each package we want to backport
> and include in our repo.

Exactly!

The list of packages, you may find it here:
http://mitaka-jessie.pkgs.mirantis.com/debian/pool/jessie-mitaka-backports-nochange/

Or, more easily, in the Sources file here:
ftp://mitaka-jessie.pkgs.mirantis.com/debian/dists/jessie-mitaka-backports-nochange/main/source/

The list of packages could be maintained in a .yaml file for example,
then parsed to regularly maintain the backports, and sending an alert if
one package fails to build.

> It would also be good to tie off with the security team about this. One
> of the reasons we stopped publishing debs years ago is that it made us a
> de-facto derivative distro. People were using our packages in
> production, including backports we'd built in support of those packages,
> but our backports were not receiving security/CVE attention, so we were
> concerned that we were causing people to be exposed to issues.

I'd like to also take care of these packages, and upload them to Sid. I
already packaged some of them, though I was stopped because of the lack
of support for statsd >= 3.x (which is in Sid), which has since been
added. I could resume that work once Mitaka is done (in the mean while,
I'm *very* busy with it).

> We'll also want to make sure we're building packages for trusty and xenial.

All I write works for both. It is fully my intention to support Trusty
and Xenial as well as Jessie and hopefully Sid (with probably Sid as
non-voting, as it will break too often).

> Yay for movement!

+1 !!!
If you're really going to be the PTL *and* help this to happen, that's
fantastic. Thanks for your comments already.

Cheers,

Thomas Goirand (zigo)

P.S: ACK for Fungi's reply about security.