Open Stack

On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> John Dickinson wrote:
> > [...]
> >> So the real question we need to answer is... where does OpenStack
> >> stop, and where does the wider open source community start ? If
> >> OpenStack is purely an "integration engine", glue code for other
> >> lower-level technologies like hypervisors, databases, or distributed
> >> block storage, then the scope is limited, Python should be plenty
> >> enough, and we don't need to fragment our community. If OpenStack is
> >> "whatever it takes to reach our mission", then yes we need to add one
> >> language to cover lower-level/native optimization, because we'll
> >> need that... and we need to accept the community cost as a
> >> consequence of that scope choice. Those are the only two options on
> >> the table.
> >>
> >> I'm actually not sure what is the best answer. But I'm convinced we,
> >> as a community, need to have a clear answer to that. We've been
> >> avoiding that clear answer until now, creating tension between the
> >> advocates of an ASF-like collection of tools and the advocates of a
> >> tighter-integrated "openstack" product. We have created silos and
> >> specialized areas as we got into the business of developing time-
> >> series databases or SDNs. As a result, it's not "one community"
> >> anymore. Should we further encourage that, or should we focus on
> >> what the core of our mission is, what we have in common, this
> >> integration engine that binds all those other open source projects
> >> into one programmable infrastructure solution ?
> >
> > You said the answer in your question. OpenStack isn't defined as an
> > integration engine[3]. The definition of OpenStack is whatever it
> > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > I mean that we've already gone to the effort of defining OpenStack. It's
> > our mission statement. We're all about building a cloud platform upon
> > which people can run their apps ("cloud-native" or otherwise), so we
> > write the software needed to do that.
> >
> > So where does OpenStack stop and the wider community start? OpenStack
> > includes the projects needed to fulfill its mission.
> 
> I'd totally agree with you if OpenStack was developed in a vacuum. But 
> there is a large number of open source projects and libraries that 
> OpenStack needs to fulfill its mission that are not in "OpenStack": they 
> are external open source projects we depend on. Python, MySQL, libvirt, 
> KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should 
> be included in OpenStack, and we are not NIHing replacements for those 
> in OpenStack either.
> 
> So it is not as clear-cut as you present it, and you can approach this 
> dependency question from two directions.
> 
> One is community-centric: "anything produced by our community is 
> OpenStack". If we are missing a lower-level piece to achieve our mission 
> and are developing it ourselves as a result, then it is OpenStack, even 
> if it ends up being a message queue or a database.
> 
> The other approach is product-centric: "lower-level pieces are OpenStack 
> dependencies, rather than OpenStack itself". If we are missing a 
> lower-level piece to achieve our mission and are developing it as a 
> result, it could be developed on OpenStack infrastructure by members of 
> the OpenStack community but it is not "OpenStack the product", it's an 
> OpenStack *dependency*. It is not governed by the TC, it can use any 
> language and tool deemed necessary.
> 
> On this second approach, there is the obvious question of where 
> "lower-level" starts, which as you explained above is not really 
> clear-cut. A good litmus test for it could be whenever Python is not 
> enough. If you can't develop it effectively with the language that is 
> currently sufficient for the rest of OpenStack, then developing it as an 
> OpenStack dependency in whatever language is appropriate might be the 
> solution...
> 
> That is what I mean by 'scope': where does "OpenStack" stop, and where 
> do "OpenStack dependencies" start ? It is a lot easier and a lot less 
> community-costly to allow additional languages in OpenStack dependencies 
> (we already have plenty there).
> 

I strongly agree with both of the points about what OpenStack is defined
as. We are  a set of projects attempting to fulfill our mission. In
doing so, we try to use outside dependencies to help us as much as
possible. Sometimes we cannot find an outside dependency to satisfy a
need whether due to optimization needs, licensing issues, usability
problems, or simply because an outside project doesn't exist. That is
when things become less clear-cut and we might need to develop software
not purely/directly related to fulfilling our mission.

In the product-centric approach I worry that we are going to be paying
many of the costs which the existing TC resolutions hoped to prevent. We
still will have to maintain and debug these dependencies and if the
OpenStack community is the only ones doing so then the language
fragmentation costs will come in to play. Our goal should be to only pay
these costs when necessary, and I am not sure this type of distinction
helps us do that.

Additionally, I am not sure how this would solve the swift case. The
object server is the component that has been rewritten so either it
would need to live outside of OpenStack with the rest of Swift being
part of OpenStack, or all of Swift would need to move outside. Neither
of these seems ideal.

This makes me wonder - does this decision need to be an all-or-nothing
thing? Can we say that the swift object server is a special case, let it
be written in another language, and leave it at that? My thinking is -
there is only value in us deciding on and allowing an additional
language for solving these issues generally if there are other places
that require this new language. It has been mentioned that Gnocchi could
potentially be a case in the future but until that problem actually
comes up there isn't much benefit to us deciding on another language
anyone can use and there are plenty of downsides to doing so.

Cheers,
Greg