Open Stack

Caution: words below may cause discomfort. I ask that folks read *all* 
of my message before reacting to any piece of it. Thanks!

On 08/19/2014 02:41 AM, Robert Collins wrote:
> On 18 August 2014 09:32, Clint Byrum <clint at fewbar.com> wrote:
>
> I can see your perspective but I don't think its internally consistent...
>
>> Here's why folk are questioning Ceilometer:
>>
>> Nova is a set of tools to abstract virtualization implementations.
>
> With a big chunk of local things - local image storage (now in
> glance), scheduling, rebalancing, ACLs and quotas. Other
> implementations that abstract over VM's at various layers already
> existed when Nova started - some bad ( some very bad!) and others
> actually quite ok.
>
>> Neutron is a set of tools to abstract SDN/NFV implementations.
>
> And implements a DHCP service, DNS service, overlay networking : its
> much more than an abstraction-over-other-implementations.
>
>> Cinder is a set of tools to abstract block-device implementations.
>> Trove is a set of tools to simplify consumption of existing databases.
>> Sahara is a set of tools to simplify Hadoop consumption.
>> Swift is a feature-complete implementation of object storage, none of
>> which existed when it was started.
>
> Swift was started in 2009; Eucalyptus goes back to 2007, with Walrus
> part of that - I haven't checked precise dates, but I'm pretty sure
> that it existed and was usable by the start of 2009. There may well be
> other object storage implementations too - I simply haven't checked.
>
>> Keystone supports all of the above, unifying their auth.
>
> And implementing an IdP (which I know they want to stop doing ;)). And
> in fact lots of OpenStack projects, for various reasons support *not*
> using Keystone (something that bugs me, but thats a different
> discussion).
>
>> Horizon supports all of the above, unifying their GUI.
>>
>> Ceilometer is a complete implementation of data collection and alerting.
>> There is no shortage of implementations that exist already.
>>
>> I'm also core on two projects that are getting some push back these
>> days:
>>
>> Heat is a complete implementation of orchestration. There are at least a
>> few of these already in existence, though not as many as their are data
>> collection and alerting systems.
>>
>> TripleO is an attempt to deploy OpenStack using tools that OpenStack
>> provides. There are already quite a few other tools that _can_ deploy
>> OpenStack, so it stands to reason that people will question why we
>> don't just use those. It is my hope we'll push more into the "unifying
>> the implementations" space and withdraw a bit from the "implementing
>> stuff" space.
>>
>> So, you see, people are happy to unify around a single abstraction, but
>> not so much around a brand new implementation of things that already
>> exist.
>
> If the other examples we had were a lot purer, this explanation would
> make sense. I think there's more to it than that though :).
>
> What exactly, I don't know, but its just too easy an answer, and one
> that doesn't stand up to non-trivial examination :(.

I actually agree with Robert about this; that Clint may have 
oversimplified whether or not certain OpenStack projects may have 
reimplemented something that previously existed. Everything is a grey 
area, after all. I'm sure each project can go back in time and point to 
some existing piece of software -- good, bad or Java -- and truthfully 
say that there was prior art that could have been used.

The issue that I think needs to be addressed more directly in this 
thread and the ongoing conversation on the TC is this:

By graduating an incubated project into the integrated release, the 
Technical Committee is blessing the project as "the OpenStack way" to do 
some thing. If there are projects that are developed *in the OpenStack 
ecosystem* that are actively being developed to serve the purpose that 
an integrated project serves, then I think it is the responsibility of 
the Technical Committee to take another look at the integrated project 
and answer the following questions definitively:

  a) Is the Thing that the project addresses something that the 
Technical Committee believes the OpenStack ecosystem benefits from by 
the TC making a judgement on what is "the "OpenStack way" of addressing 
that Thing.

and IFF the decision of the TC on a) is YES, then:

  b) Is the Vision and Implementation of the currently integrated 
project the one that the Technical Committee wishes to continue to 
"bless" as the "the OpenStack way" of addressing the Thing the project does.

If either of the above answers is NO, then I believe the Technical 
Committee should recommend that the integrated project be removed from 
the integrated release.

HOWEVER, I *also* believe that the previously-integrated project should 
not just be cast away back to Stackforge. I think the project should 
remain in its designated Program and should remain in the openstack/ 
code namespace. Furthermore, active, competing visions and 
implementations of projects that address the Thing the 
previously-integrated project addressed should be able to apply to join 
the same Program, and *also* live in the openstack/ namespace.

All of these projects should be able to live in the Program, in the 
openstack/ code namespace, for as long as the project is actively 
developed, and let the contributor communities in these competing 
projects *naturally* work to do any of the following:

  * Pick a best-of-breed implementation from the projects that address 
the same Thing
  * Combine code and efforts to merge the good bits of multiple projects 
into one
  * Let multiple valid choices of implementation live in the same 
Program with none of them being "blessed" by the TC to be part of the 
integrated release

The process of incubation and graduation to integration would be 
essentially unchanged, with the exception that an additional condition 
would be added to the graduation requirements: no other 
actively-developed projects are in the same Program that address the 
same Thing as the project proposing for incubation.

The integrated release would continue to be the projects from each 
Program that are validated via the continuous integration gates to work 
properly with each other, and Infra/CI resources would be prioritized of 
course to the integrated and incubated projects.

In this way, we would:

  1) Recognize the openstack/ code namespace for the value it brings, 
and free projects to live in the openstack/ code namespace without being 
incubated, integrated, or any of that stuff. We need to be aware that 
the openstack/ code namespace brings with it a sense of legitimacy, and 
this sense of legitimacy does wonders in getting both momentum and 
$resources behind a project. As a community, I believe we should be 
doing as much as we can to bring more projects into the OpenStack fold, 
and having *actively-developed* projects under the openstack/ code 
namespace is a way to do that.

  2) Get the TC out of the business of picking winners and losers. 
Instead, the developer market would be able to essentially incubate 
competing ideas within their own Program, naturally letting the best 
ideas float to the top, or merging good stuff together, or even choosing 
to not make a choice and letting multiple projects live harmoniously in 
the same Program.

  3) Get the TC out of the business of deciding what is or isn't in 
scope for "What is OpenStack". Instead, the TC would welcome new Program 
additions where the scope of the Program addressed some Thing that is 
relevant to cloud computing. And the TC would welcome 
*actively-developed* projects that fit into these Programs. The TC would 
be play an advisory role in architecture and design issues, but leave 
the final decisions on project development to the contributors in that 
program. And the TC would focus its decisions/judgements solely on 
incubation and graduation reviews.

> I'd like to see more unification of implementations in TripleO - but I
> still believe our basic principle of using OpenStack technologies that
> already exist in preference to third party ones is still sound, and
> offers substantial dogfood and virtuous circle benefits.

No doubt Triple-O serves a valuable dogfood and virtuous cycle purpose. 
However, I would move that the Deployment Program should welcome the 
many projects currently in the stackforge/ code namespace that do 
deployment of OpenStack using traditional configuration management tools 
like Chef, Puppet, and Ansible. It cannot be argued that these 
configuration management systems are the de-facto way that OpenStack is 
deployed outside of HP, and they belong in the Deployment Program, IMO.

As a TC member, I would welcome someone from the Chef community 
proposing the Chef cookbooks for inclusion in the Deployment program, to 
live under the openstack/ code namespace. Same for the Puppet modules.

Just my 2 cents,
-jay