Open Stack

John Dickinson wrote:
>> [...]
>> In the same vein, if we consider lower-level projects (which often
>> require such native optimization) as part of "OpenStack", rather
>> than as external open source projects that should be integrated by
>> "OpenStack", then we need a language like golang in our toolbelt.
>> There is basically no point in saying no to golang in OpenStack if
>> we need lower-level native optimization in OpenStack: we'll have to
>> accept the community cost that comes with such a community scope.
>
> Defining "lower-level" is very hard. Since the Nova API[1] is
> listening to a public network interface and coordinating with various
> services in a cluster, is it low-level enough to need to consider
> optimizations? Does the Nova API require optimization to handle a very
> large number of connections using all of the hardware available on a
> single server? If Nova is going to eek out every drop of performance
> possible from a server, it probably does need to consider all kinds of
> "low-level" optimizations.[2]

That is fair. There is no clear line between lower-level and 
higher-level (although one might argue there is a line between projects 
requiring Go and projects that don't require it). It's more of a gradient.

> [...]
>> So the real question we need to answer is... where does OpenStack
>> stop, and where does the wider open source community start ? If
>> OpenStack is purely an "integration engine", glue code for other
>> lower-level technologies like hypervisors, databases, or distributed
>> block storage, then the scope is limited, Python should be plenty
>> enough, and we don't need to fragment our community. If OpenStack is
>> "whatever it takes to reach our mission", then yes we need to add one
>> language to cover lower-level/native optimization, because we'll
>> need that... and we need to accept the community cost as a
>> consequence of that scope choice. Those are the only two options on
>> the table.
>>
>> I'm actually not sure what is the best answer. But I'm convinced we,
>> as a community, need to have a clear answer to that. We've been
>> avoiding that clear answer until now, creating tension between the
>> advocates of an ASF-like collection of tools and the advocates of a
>> tighter-integrated "openstack" product. We have created silos and
>> specialized areas as we got into the business of developing time-
>> series databases or SDNs. As a result, it's not "one community"
>> anymore. Should we further encourage that, or should we focus on
>> what the core of our mission is, what we have in common, this
>> integration engine that binds all those other open source projects
>> into one programmable infrastructure solution ?
>
> You said the answer in your question. OpenStack isn't defined as an
> integration engine[3]. The definition of OpenStack is whatever it
> takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> I mean that we've already gone to the effort of defining OpenStack. It's
> our mission statement. We're all about building a cloud platform upon
> which people can run their apps ("cloud-native" or otherwise), so we
> write the software needed to do that.
>
> So where does OpenStack stop and the wider community start? OpenStack
> includes the projects needed to fulfill its mission.

I'd totally agree with you if OpenStack was developed in a vacuum. But 
there is a large number of open source projects and libraries that 
OpenStack needs to fulfill its mission that are not in "OpenStack": they 
are external open source projects we depend on. Python, MySQL, libvirt, 
KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should 
be included in OpenStack, and we are not NIHing replacements for those 
in OpenStack either.

So it is not as clear-cut as you present it, and you can approach this 
dependency question from two directions.

One is community-centric: "anything produced by our community is 
OpenStack". If we are missing a lower-level piece to achieve our mission 
and are developing it ourselves as a result, then it is OpenStack, even 
if it ends up being a message queue or a database.

The other approach is product-centric: "lower-level pieces are OpenStack 
dependencies, rather than OpenStack itself". If we are missing a 
lower-level piece to achieve our mission and are developing it as a 
result, it could be developed on OpenStack infrastructure by members of 
the OpenStack community but it is not "OpenStack the product", it's an 
OpenStack *dependency*. It is not governed by the TC, it can use any 
language and tool deemed necessary.

On this second approach, there is the obvious question of where 
"lower-level" starts, which as you explained above is not really 
clear-cut. A good litmus test for it could be whenever Python is not 
enough. If you can't develop it effectively with the language that is 
currently sufficient for the rest of OpenStack, then developing it as an 
OpenStack dependency in whatever language is appropriate might be the 
solution...

That is what I mean by 'scope': where does "OpenStack" stop, and where 
do "OpenStack dependencies" start ? It is a lot easier and a lot less 
community-costly to allow additional languages in OpenStack dependencies 
(we already have plenty there).

-- 
Thierry Carrez (ttx)