Open Stack

John Dickinson wrote:
> summary:
>   * Defining the scope of OpenStack projects DOES NOT define the
>     languages needed to implement them. The considerations are
>     orthogonal.
>   * We've already defined OpenStack--it's whatever it takes to
>     fulfill its mission statement.
>
> On 19 May 2016, at 6:19, Thierry Carrez wrote:
>
>> Hi everyone,
>>
>> The discussion on the addition of golang focuses on estimating
>> community costs vs. technical benefits, so that the TC can make the
>> right call for "OpenStack". From that discussion, we established
>> that the costs for cross-project teams (Infra...) seem to be
>> workable. There is still significant community fragmentation
>> effects as we add another language, creating language-expert silos,
>> duplicating efforts, and losing some productivity as some
>> needlessly rewrite things in golang. But we seem generally ready to
>> accept that cost because we are missing a tool in our toolbelt: a
>> language that lets us do such native optimization.
>>
>> We have a number of projects in OpenStack that are more low-level
>> than others and which require such optimization, and for those
>> projects (or subprojects) our current mix of language is not
>> satisfactory. The Swift team in particular has spent a lot of time
>> trying to get the I/O behavior they require with hard disk access
>> using Python, and they certainly did not jump on the golang
>> bandwagon lightly.
>>
>> I believe the programming languages you need in OpenStack official
>> projects are a function of the scope you define for OpenStack
>> official projects. We wouldn't need to have JavaScript in the mix if
>> we considered that web interfaces that purely consume OpenStack
>> APIs are projects that consume OpenStack, rather than part of
>> OpenStack itself.
>>
>> In the same vein, if we consider lower-level projects (which often
>> require such native optimization) as part of "OpenStack", rather
>> than as external open source projects that should be integrated by
>> "OpenStack", then we need a language like golang in our toolbelt.
>> There is basically no point in saying no to golang in OpenStack if
>> we need lower-level native optimization in OpenStack: we'll have to
>> accept the community cost that comes with such a community scope.
>
> Defining "lower-level" is very hard. Since the Nova API[1] is
> listening to a public network interface and coordinating with various
> services in a cluster, is it low-level enough to need to consider
> optimizations? Does the Nova API require optimization to handle a very
> large number of connections using all of the hardware available on a
> single server? If Nova is going to eek out every drop of performance
> possible from a server, it probably does need to consider all kinds of
> "low-level" optimizations.[2]
>
> Because deployers of any OpenStack project do want efficient software
> that is performant, all parts of OpenStack need to consider what it
> takes to meet the performance demanded. Most of the time this will not
> require rewriting code in a different language (that's almost never
> the right answer), but my point is that I believe you're wrong that
> defining the scope of OpenStack projects also defines the languages
> needed to implement them. The considerations are orthogonal.
>
> [1] substitute your favorite OpenStack project
>
> [2] http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
>
>
>> So the real question we need to answer is... where does OpenStack
>> stop, and where does the wider open source community start ? If
>> OpenStack is purely an "integration engine", glue code for other
>> lower-level technologies like hypervisors, databases, or distributed
>> block storage, then the scope is limited, Python should be plenty
>> enough, and we don't need to fragment our community. If OpenStack is
>> "whatever it takes to reach our mission", then yes we need to add one
>> language to cover lower-level/native optimization, because we'll
>> need that... and we need to accept the community cost as a
>> consequence of that scope choice. Those are the only two options on
>> the table.
>>
>> I'm actually not sure what is the best answer. But I'm convinced we,
>> as a community, need to have a clear answer to that. We've been
>> avoiding that clear answer until now, creating tension between the
>> advocates of an ASF-like collection of tools and the advocates of a
>> tighter-integrated "openstack" product. We have created silos and
>> specialized areas as we got into the business of developing time-
>> series databases or SDNs. As a result, it's not "one community"
>> anymore. Should we further encourage that, or should we focus on
>> what the core of our mission is, what we have in common, this
>> integration engine that binds all those other open source projects
>> into one programmable infrastructure solution ?
>
> You said the answer in your question. OpenStack isn't defined as an
> integration engine[3]. The definition of OpenStack is whatever it
> takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> I mean that we've already gone to the effort of defining OpenStack. It's
> our mission statement. We're all about building a cloud platform upon
> which people can run their apps ("cloud-native" or otherwise), so we
> write the software needed to do that.
>
> So where does OpenStack stop and the wider community start? OpenStack
> includes the projects needed to fulfill its mission.
>
> [3] There are foundation staff who have used that phrase to help
> describe some of the work we do (mostly to explain how OpenStack still
> matters in a docker/kubernetes/container world). While that may be good for
> explaining and bringing people in, it's not the definition of
> OpenStack.

And to expand this. IMHO its not even a good way of explaining 'how 
OpenStack still matters in a docker/kubernetes/container world' because 
that world doesn't fully exist yet (and likely won't for years) and 
already saying that the way for OpenStack to work here is via an 
integration engine is sad.

This is especially wrong because those docker/kubernetes are still 
maturing projects and people (from both communities) are working on 
connecting the two different worlds together in a way that makes sense 
(and that work is not frozen in time) 
[seehttps://groups.google.com/forum/#!forum/kubernetes-sig-openstack for 
one such variant]. One take on that work progressing is that kubernetes 
could be just another project that uses keystone, cinder, neutron ... 
(in a similar vein to how nova uses those same projects).

>
> [4] "To produce a ubiquitous Open Source Cloud Computing platform that
> is easy to use, simple to implement, interoperable between
> deployments, works well at all scales, and meets the needs of users
> and operators of both public and private clouds."
>
> [5] From openstack.org: "OpenStack software controls large pools of
> compute, storage, and networking resources throughout a datacenter,
> managed through a dashboard or via the OpenStack API."
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev