[openstack-dev] [OpenStack-Infra][Ceilometer][MagnetoDB] HBase database in devstack

Ruslan Kamaldinov rkamaldinov at mirantis.com
Wed Apr 9 17:24:48 UTC 2014


On Tue, Apr 8, 2014 at 8:42 PM, Sean Dague <sean at dague.net> wrote:
> I think it's important to understand what we mean by "stable" in the
> gate. It means that the end point is 99.9999% available. And that it's
> up or down status is largely under our control.
>
> Things that are not stable by this definition which we've moved away
> from for the gate:
>  * github.com - one of the reasons for git.openstack.org
>  * pypi.python.org - one of the reasons for our own pypi mirror
>  * upstream distro mirrors (we use cloud specific mirrors, which even
> then do fail some times, more than we'd like)
>
> Fedora.org is not stable by this measure either. Downloading an iso from
> fedora.org fails 5% of the time in the gate.
>
> I'm sure the Hortonworks folks are good folks, but by our standards of
> reliability, no one stacks up. And an outage on their behalf means that
> any project which gates on it will be blocked from merging any code
> until it's addressed. If Ceilometer wants to take that risk in their
> check queue (and be potentially blocked) that might be one thing, and we
> could talk about that. But we definitely can't co-gate and block all of
> openstack because of a hortonworks outage (which will happen, especially
> if we download packages from them 600 - 1000 times a day).

A natural solution for this would be a local to infra package mirror for
HBase, Ceilometer, Mongo and all the dependencies not present in upstream
Ubuntu. It seems straightforward from the technical point of view. It'll help
to keep the Gate invulnerable to any outages in 3-rd party mirrors. Of course,
someone has to signup to create scripts for that mirror and support it in the
future.

But, other concerns were expressed in the past. Let me quote Jeremy Stanley
(from https://review.openstack.org/#/c/66884/):
> This will need to be maintained in Ubuntu (and backported to 12.04 in Ubuntu
> Cloud Archive or if necessary a PPA managed by the same package maintenance
> team taking care of it in later Ubuntu releases). We don't install test
> requirements system-wide on our long-running test slaves unless we can be
> assured of security support from the Linux distribution vendor.

There is no easy workaround here. Traditionally this kind of software is
installed from vendor-supported mirrors and distributions. And they're the
ones who maintain and provide security updates from Hadoop/HBase packages.
In case of Ceilometer, I think that importance of having real tests on real
databases is more important than the requirement for the packages to have
security support from a Linux distribution.

Thanks,
Ruslan



More information about the OpenStack-dev mailing list