Open Stack

Excerpts from Ruslan Kamaldinov's message of 2014-04-09 10:24:48 -0700:
> On Tue, Apr 8, 2014 at 8:42 PM, Sean Dague <sean at dague.net> wrote:
> > I think it's important to understand what we mean by "stable" in the
> > gate. It means that the end point is 99.9999% available. And that it's
> > up or down status is largely under our control.
> >
> > Things that are not stable by this definition which we've moved away
> > from for the gate:
> >  * github.com - one of the reasons for git.openstack.org
> >  * pypi.python.org - one of the reasons for our own pypi mirror
> >  * upstream distro mirrors (we use cloud specific mirrors, which even
> > then do fail some times, more than we'd like)
> >
> > Fedora.org is not stable by this measure either. Downloading an iso from
> > fedora.org fails 5% of the time in the gate.
> >
> > I'm sure the Hortonworks folks are good folks, but by our standards of
> > reliability, no one stacks up. And an outage on their behalf means that
> > any project which gates on it will be blocked from merging any code
> > until it's addressed. If Ceilometer wants to take that risk in their
> > check queue (and be potentially blocked) that might be one thing, and we
> > could talk about that. But we definitely can't co-gate and block all of
> > openstack because of a hortonworks outage (which will happen, especially
> > if we download packages from them 600 - 1000 times a day).
> 
> A natural solution for this would be a local to infra package mirror for
> HBase, Ceilometer, Mongo and all the dependencies not present in upstream
> Ubuntu. It seems straightforward from the technical point of view. It'll help
> to keep the Gate invulnerable to any outages in 3-rd party mirrors. Of course,
> someone has to signup to create scripts for that mirror and support it in the
> future.
> 
> But, other concerns were expressed in the past. Let me quote Jeremy Stanley
> (from https://review.openstack.org/#/c/66884/):
> > This will need to be maintained in Ubuntu (and backported to 12.04 in Ubuntu
> > Cloud Archive or if necessary a PPA managed by the same package maintenance
> > team taking care of it in later Ubuntu releases). We don't install test
> > requirements system-wide on our long-running test slaves unless we can be
> > assured of security support from the Linux distribution vendor.
> 
> There is no easy workaround here. Traditionally this kind of software is
> installed from vendor-supported mirrors and distributions. And they're the
> ones who maintain and provide security updates from Hadoop/HBase packages.
> In case of Ceilometer, I think that importance of having real tests on real
> databases is more important than the requirement for the packages to have
> security support from a Linux distribution.

This is a huge philosophical question for OpenStack in general. Do we
want to recommend things that we won't, ourselves, use in our
infrastructure?

I think for the most part we've taken a middle of the road approach
where we make sure the default backends and drivers are things that
_are_ supported in distros, and are things we're able to use. We also
let in the crazy-sauce backend drivers for those who are willing to run
3rd-party testing for them.

So I think what is needed here is for MagnetoDB to have at least one
backend that _is_ supported in distro, and legally friendly to OpenStack
users. Unfortunately:

* HBase - not in any distro I could find (removed from Debian actually)
* Cassandra - not in any distro
* MongoDB - let's not have that license discussion again

Now, this is no simple matter. When I attempted to package Cassandra for
Ubuntu 3 years ago, there was zero interest upstream in supporting it
without embedding certain java libraries. The attempts at extracting the
embedded libraries resulted in failed tests, patches, and an endless
series of "why are you doing this?" type questions from upstream.
Basically the support model of the distro isn't compatible with the
support model of these databases.

What I think would have to happen, would be infra would need to be
willing to reach out and have a direct relationship with upstream for
Cassandra and HBase, and we would need to be willing to ask OpenStack
users to do the same. Otherwise, I don't think MagnetoDB could ever be
integrated with either of them as the default driver.