Open Stack

2011/8/25 Thomas Goirand <thomas at goirand.fr>:
> On 08/25/2011 04:11 AM, Soren Hansen wrote:
>> This is false. The packages are perfectly capable of building and
>> running on Debian (barring currently unknown bugs).
> No. After Cactus was released, it didn't build because of issues in
> python-eventlet related to the OpenSSL transition.

Yes. *Because of issues in python-eventlet*. Forking *the packaging*
because of lack of Debian support when the real problem is with the
state of OpenStack's dependencies isn't going to help.

> Then it built without a single error, so I uploaded. Then a few months
> later it didn't build, and I got warned about it by:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=634406

Because of python-webob being out of date.

>> Debian releases are rare and unpredictable.
> Actually, they are predictable. We have now scheduled *freeze* time
> (each 2 years). Yes, you wont know when it will be released, but you
> can use testing a bit after it's frozen. To my experience, after 1 or
> 2 months of frozen stage, it's pretty stable. Reasonably, it would be
> released about 6 months of time after the freeze (see the release
> history, that's about the time-line nearly all the time).

6 months freeze time makes it fail the other requirement: Having
up-to-date packages. If we could not use software that had been written
withtin the last 6 months, we wouldn't be able to progress anywhere near
as fast as we do now.

Again, don't get me wrong: I think what Debian is going is fine. In
fact, I think it's great. Debian is a better distro because of it. It's
just not a very convenient home for a project like OpenStack that
evolves so incredibly quickly.

>> Alternatively, you have to maintain a set of backports, but that's a
>> major pain (and it only gets worse over time).
> Actually, backports are fairly easy to work with, and I don't think they
> are a pain. I think they are even more easy than anything else. That's
> always what I suggest to anyone willing to go in production using Debian.

Using backports (especially locally maintained ones) means that you're
all alone in dealing with security or any sort of other issues. That's
the sort of stuff that makes it hard for me to sleep. Clearly, I care
too much.

>> They're doing it because there'll supposedly be differences in the
>> behaviour Ubuntu (or any other distro for that matter) will want and
>> what Rackspace will want from the packages.
> As much as I heard, they do backports because all their infrastructure
> is running Debian stable, they are happy with it, and want to continue
> this way.

Again, this is not about backports. It's about the packaging of
OpenStack (Swift, specifically) itself.

>>> - PPAs are Async. This makes integration testing harder.
>> PPAs were chosen because
>> a) they are almost exactly identical to the builders for Ubuntu
>> proper, so the on-commit-to-trunk builds serve as way to know
>> immediately if we introduce a change that will break when uploaded to
>> Ubuntu proper. So, they're the closest we'll get to integration tests
>> for real Ubuntu builds,
>> b) we don't have to maintain them, and
>> c) adding a PPA as a source to an Ubuntu system is incredibly
>> straight-forward and a very well-supported operation.
> a) and b) are very valid point, c) isn't (frankly, how hard is it to
> just add a line in your sources.list compared to the complexity of
> setting-up OpenStack?)

You also need to import the keys, but no, it's not very complicated. But
if you stack enough "not very complicated" things, you end up with
something that's pretty darned complicated.

>>> If it's building them locally, and that's what's being tested, why
>>> wouldn't that be what's published?
>> Because of c) above and because PPAs are trustworthy.
> I don't think that c) is valid (see above), and I don't think PPAs are
> more trustworthy than a private repository with a well maintained GPG
> key and a ${whatever}-archive-keyring package published.  As long as
> the release file is signed, you have the same security, IMHO. Please
> let me know if/why I'm wrong here.

It's not just about gatekeeping and signing of Packages and Release
files. It's also about trusting its reproducability. I'm not saying
anyone has ill intentions. It's just that even with ill intentions, you
can't compromise the integrity of the chroots your stuff gets built in
in PPAs. Even by accident. You also can't reach the Internet, so if a
part of build speaks to the internet (which isn't always obvious) that's
an unknown factor that compromises the reproducability of the build.

> By the way, there are ongoing talks inside Debian to have PPAs as
> well, because we'd be happy to use the buildd machines to have ports
> for many different arch. I really hope this will happen soon.

That would be great!

>>> - Direct injection to Ubuntu not via Debian. The publically stated
>>> best-practice for things entering Ubuntu is to be added to/uploaded to
>>> Debian and then synced, unless there is some compelling reason not to do
>>> this. In this case, we had Ubuntu devs and no Debian devs in the core,
>>> so we keep improving packages and adding packages to Ubuntu but are not
>>> doing the same to Debian.
>> For the umpteenth time, this is not about OpenStack. It's about the
>> dependencies. Getting OpenStack itself into Debian was never the hard
>> part. Getting all the dependencies into Debian was. Not just libraries
>> that weren't in Debian at all (like gflags, which I got uploaded to
>> Debian and synced into Ubuntu afterwards), but existing libraries in
>> Debian that we needed either a newer version of or a patch applied to.
>> In Ubuntu we have the power to upload these newer versions or patches
>> or whatever, by implicitly accepting responsibility for it if it
>> should happen to cause problems. We have no way to even begin to
>> ensure that we can do this in Debian. Debian packages have
>> maintainers. The maintainers decide whether to accept a patch or not.
>> If they're not comfortable with a patch, rejecting it is entirely
>> their prerogative (and for good reason! They're the ones who
>> ultimately have to support it).
> This is a pretty bad picture that you are painting of Debian.

I'm sorry. That was not my intention at all.

> Theoretically, you are right, at the exception that if there is a
> strong technical disagreement with the maintainer of a given package,
> then you can ask the technical committee to decide. But in reality,
> most package maintainers will be very happy to have patches sent
> against their package, and will accept them.

I'm not saying whether any given maintainer will or will not accept
whatever random patches we throw at them. I'm just saying they *can*,
and I truly believe that they absolutely should be free to do so. I
wouldn't dream of complaining to the tech board because a maintainer
doesn't want to accept maintenance of a patch I wrote. This just makes
it really difficult to rely on being able to get a patch accepted.

> Also, more and more, we are seeing people
> forming packaging teams and work collaboratively. Just have a look in
> Alioth how many packages are in collab-maint! Having teams for packaging
> really is the preferred way, unlike few years ago.

And that's great! I'm part of the Debian Python Module Team as well.
These efforts are excellent and I applaud them, but just like I can't
rely on any given maintainer to accept my patches, I also can't rely on
all our dependencies (current and future) to be maintained in such an
open way.

>> In every Ubuntu release since we started work on OpenStack, I've
>> added at least one patch to libvirt to support something we needed
>> for OpenStack. I've submitted them upstream (to libvirt) as well, but
>> we can't stop everything while waiting for libvirt to make a release,
>> the Debian maintainer to pick it up, and for Ubuntu to sync it.
> Your example with libvirt is especially wrong. It's team maintained in
> Debian, see the maintainer field:
>
> Debian Libvirt Maintainers <pkg-libvirt-maintainers at lists.alioth.debian.org>
>
> Have you tried joining the Alioth project? This group has 8 members
> right now.

I understand. libvirt was simply an example of a package I knew I had
had to patch on several occasions. Eventlet is another one I've had to
patch, but which also *happens to* be maintained by people I know (Monty
and Stefano Rivera) so that's also not a current, actual problem.
"Happens to" are the keywords here, though.

>> Debian support is a massive undertaking and I'm thrilled to see it
>> happen
> I think that's more a mater of communication. With some maintainers,
> it's going to be easy. With others, it will be harder.

There's no guarantee it's possible.

Let's take the Eventlet example. Had it been maintained by someone else,
I'm far from convinced the most recent patch we added to it would be
accepted, even if the maintainer was a very reasonable person. Why?
Because upstream has expressed rather lukewarm feelings about the patch.
In such a situation, it's common (and perfectly reasonable!) for Debian
maintainers to say "no, thanks".

-- 
Soren Hansen
Ubuntu Developer    http://www.ubuntu.com/
OpenStack Developer http://www.openstack.org/