Open Stack

Excerpts from Robert Collins's message of 2014-08-18 23:41:20 -0700:
> On 18 August 2014 09:32, Clint Byrum <clint at fewbar.com> wrote:
> 
> I can see your perspective but I don't think its internally consistent...
> 
> > Here's why folk are questioning Ceilometer:
> >
> > Nova is a set of tools to abstract virtualization implementations.
> 
> With a big chunk of local things - local image storage (now in
> glance), scheduling, rebalancing, ACLs and quotas. Other
> implementations that abstract over VM's at various layers already
> existed when Nova started - some bad ( some very bad!) and others
> actually quite ok.
> 

The fact that we have local implementations of domain specific things is
irrelevant to the difference I'm trying to point out. Glance needs to
work with the same authentication semantics and share a common access
catalog to work well with Nova. It's unlikely there's a generic image
catalog that would ever fit this bill. In many ways glance is just an
abstraction of file storage backends and a database to track a certain
domain of files (images, and soon, templates and other such things).

The point of mentioning Nova is, we didn't write libvirt, or xen, we
wrote an abstraction so that users could consume them via a REST API
that shares these useful automated backends like glance.

> > Neutron is a set of tools to abstract SDN/NFV implementations.
> 
> And implements a DHCP service, DNS service, overlay networking : its
> much more than an abstraction-over-other-implementations.
> 

Native DHCP and overlay? Last I checked Neutron used dnsmasq and
openvswitch, but it has been a few months, and I know that is an eon in
OpenStack time.

> > Cinder is a set of tools to abstract block-device implementations.
> > Trove is a set of tools to simplify consumption of existing databases.
> > Sahara is a set of tools to simplify Hadoop consumption.
> > Swift is a feature-complete implementation of object storage, none of
> > which existed when it was started.
> 
> Swift was started in 2009; Eucalyptus goes back to 2007, with Walrus
> part of that - I haven't checked precise dates, but I'm pretty sure
> that it existed and was usable by the start of 2009. There may well be
> other object storage implementations too - I simply haven't checked.
> 

Indeed, and MogileFS was sort of like Swift but not HTTP based. Perhaps
Walrus was evaluated and inadequate for the CloudFiles product
requirements? I don't know. But there weren't de-facto object stores
at the time because object stores were just becoming popular.

> > Keystone supports all of the above, unifying their auth.
> 
> And implementing an IdP (which I know they want to stop doing ;)). And
> in fact lots of OpenStack projects, for various reasons support *not*
> using Keystone (something that bugs me, but thats a different
> discussion).
> 

My point was it is justified to have a whole implementation and not
just abstraction because it is meant to enable the ecosystem, not _be_
the ecosystem. I actually think Keystone is problematic too, and I often
wonder why we haven't just do OAuth, but I'm not trying to throw every
project under the bus. I'm trying to state that we accept Keystone because
it has grown organically to support the needs of all the other pieces.

> > Horizon supports all of the above, unifying their GUI.
> >
> > Ceilometer is a complete implementation of data collection and alerting.
> > There is no shortage of implementations that exist already.
> >
> > I'm also core on two projects that are getting some push back these
> > days:
> >
> > Heat is a complete implementation of orchestration. There are at least a
> > few of these already in existence, though not as many as their are data
> > collection and alerting systems.
> >
> > TripleO is an attempt to deploy OpenStack using tools that OpenStack
> > provides. There are already quite a few other tools that _can_ deploy
> > OpenStack, so it stands to reason that people will question why we
> > don't just use those. It is my hope we'll push more into the "unifying
> > the implementations" space and withdraw a bit from the "implementing
> > stuff" space.
> >
> > So, you see, people are happy to unify around a single abstraction, but
> > not so much around a brand new implementation of things that already
> > exist.
> 
> If the other examples we had were a lot purer, this explanation would
> make sense. I think there's more to it than that though :).
> 

If purity is required to show a difference, then I don't think I know
how to demonstrate what I think is obvious to most of us: Ceilometer
is an end to end implementation of things that exist in many battle
tested implementations. I struggle to think of another component of
OpenStack that has this distinction.

> What exactly, I don't know, but its just too easy an answer, and one
> that doesn't stand up to non-trivial examination :(.
> 
> I'd like to see more unification of implementations in TripleO - but I
> still believe our basic principle of using OpenStack technologies that
> already exist in preference to third party ones is still sound, and
> offers substantial dogfood and virtuous circle benefits.
> 

Right, agree 100%. What I'm trying to say is, because we are using the TC
approved "winners", the unknowns and the "not developed in our ecosystem"
tools are all framed as losers. Given that, it should come as no surprise
that we have no other tools participating themselves.

In fact, we ourselves have just barely toyed with things like Ansible
replacing bits and pieces, and it has been met with immediate suspicion
by our community (sorry Steve Baker, I'm struggling to come up with
better words to characterize your well intentioned questions).