Open Stack

On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes
> <greg at greghaynes.net> wrote:
>> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
>> > John Dickinson wrote:
>> > > [...]
>> > >> So the real question we need to answer is... where does
>> > >> OpenStack
>> > >> stop, and where does the wider open source community start ? If
>> > >> OpenStack is purely an "integration engine", glue code for other
>> > >> lower-level technologies like hypervisors, databases, or
>> > >> distributed
>> > >> block storage, then the scope is limited, Python should be
>> > >> plenty
>> > >> enough, and we don't need to fragment our community. If
>> > >> OpenStack is
>> > >> "whatever it takes to reach our mission", then yes we need to
>> > >> add one
>> > >> language to cover lower-level/native optimization, because we'll
>> > >> need that... and we need to accept the community cost as a
>> > >> consequence of that scope choice. Those are the only two options
>> > >> on
>> > >> the table.
>> > >>
>> > >> I'm actually not sure what is the best answer. But I'm convinced
>> > >> we,
>> > >> as a community, need to have a clear answer to that. We've been
>> > >> avoiding that clear answer until now, creating tension between
>> > >> the
>> > >> advocates of an ASF-like collection of tools and the advocates
>> > >> of a
>> > >> tighter-integrated "openstack" product. We have created silos
>> > >> and
>> > >> specialized areas as we got into the business of developing time-
>> > >> series databases or SDNs. As a result, it's not "one community"
>> > >> anymore. Should we further encourage that, or should we focus on
>> > >> what the core of our mission is, what we have in common, this
>> > >> integration engine that binds all those other open source
>> > >> projects
>> > >> into one programmable infrastructure solution ?
>> > >
>> > > You said the answer in your question. OpenStack isn't defined
>> > > as an
>> > > integration engine[3]. The definition of OpenStack is whatever it
>> > > takes to fulfill our mission[4][5]. I don't mean that as a
>> > > tautology.
>> > > I mean that we've already gone to the effort of defining
>> > > OpenStack. It's
>> > > our mission statement. We're all about building a cloud platform
>> > > upon
>> > > which people can run their apps ("cloud-native" or otherwise),
>> > > so we
>> > > write the software needed to do that.
>> > >
>> > > So where does OpenStack stop and the wider community start?
>> > > OpenStack
>> > > includes the projects needed to fulfill its mission.
>> >
>> > I'd totally agree with you if OpenStack was developed in a
>> > vacuum. But
>> > there is a large number of open source projects and libraries that
>> > OpenStack needs to fulfill its mission that are not in
>> > "OpenStack": they
>> > are external open source projects we depend on. Python, MySQL,
>> > libvirt,
>> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those
>> > should
>> > be included in OpenStack, and we are not NIHing replacements for
>> > those
>> > in OpenStack either.
>> >
>> > So it is not as clear-cut as you present it, and you can
>> > approach this
>> > dependency question from two directions.
>> >
>> > One is community-centric: "anything produced by our community is
>> > OpenStack". If we are missing a lower-level piece to achieve our
>> > mission
>> > and are developing it ourselves as a result, then it is
>> > OpenStack, even
>> > if it ends up being a message queue or a database.
>> >
>> > The other approach is product-centric: "lower-level pieces are
>> > OpenStack
>> > dependencies, rather than OpenStack itself". If we are missing a
>> > lower-level piece to achieve our mission and are developing it as a
>> > result, it could be developed on OpenStack infrastructure by
>> > members of
>> > the OpenStack community but it is not "OpenStack the product",
>> > it's an
>> > OpenStack *dependency*. It is not governed by the TC, it can
>> > use any
>> > language and tool deemed necessary.
>> >
>> > On this second approach, there is the obvious question of where
>> > "lower-level" starts, which as you explained above is not really
>> > clear-cut. A good litmus test for it could be whenever Python
>> > is not
>> > enough. If you can't develop it effectively with the language
>> > that is
>> > currently sufficient for the rest of OpenStack, then developing it
>> > as an
>> > OpenStack dependency in whatever language is appropriate might
>> > be the
>> > solution...
>> >
>> > That is what I mean by 'scope': where does "OpenStack" stop, and
>> > where
>> > do "OpenStack dependencies" start ? It is a lot easier and a
>> > lot less
>> > community-costly to allow additional languages in OpenStack
>> > dependencies
>> > (we already have plenty there).
>> >
>>
>> I strongly agree with both of the points about what OpenStack is
>> defined
>> as. We are  a set of projects attempting to fulfill our mission. In
>> doing so, we try to use outside dependencies to help us as much as
>> possible. Sometimes we cannot find an outside dependency to satisfy a
>> need whether due to optimization needs, licensing issues, usability
>> problems, or simply because an outside project doesn't exist. That is
>> when things become less clear-cut and we might need to develop
>> software
>> not purely/directly related to fulfilling our mission.
>>
>> In the product-centric approach I worry that we are going to
>> be paying
>> many of the costs which the existing TC resolutions hoped to
>> prevent. We
>> still will have to maintain and debug these dependencies and if the
>> OpenStack community is the only ones doing so then the language
>> fragmentation costs will come in to play. Our goal should be to
>> only pay
>> these costs when necessary, and I am not sure this type of
>> distinction
>> helps us do that.
>>
>> Additionally, I am not sure how this would solve the swift case. The
>> object server is the component that has been rewritten so either it
>> would need to live outside of OpenStack with the rest of Swift being
>> part of OpenStack, or all of Swift would need to move outside.
>> Neither
>> of these seems ideal.
>>
>> This makes me wonder - does this decision need to be an all-or-
>> nothing
>> thing? Can we say that the swift object server is a special
>> case, let it
>> be written in another language, and leave it at that? My
>> thinking is -
>> there is only value in us deciding on and allowing an additional
>> language for solving these issues generally if there are other places
>> that require this new language. It has been mentioned that
>> Gnocchi could
>> potentially be a case in the future but until that problem actually
>> comes up there isn't much benefit to us deciding on another language
>> anyone can use and there are plenty of downsides to doing so.
>>
>
> I really do not want to "special case" swift. It really doesn't go
> with the spirit of inclusion.
 
I am not sure how inclusion is related to special casing. Inclusion here
implies that some group is not being accepted in to our community. The
excluded group here would be one writing software not in python, which
is the same groups being excluded currently. This is an argument against
the status quo, not against special casing.
 
> I expect we will see a continued drive for whichever additional
> languages are supported once it's in place/allowed.
 
That is the problem. The social and debugging costs of adding another
language are a function of how much that other language is used. If one
component of one project is written in another language then these costs
should be fairly low. I agree that once we allow another language we
should expect many projects to begin using it, and IMO most if not all
of these cases except swift will not be warranted. Thus, we will be
paying the costs in development resources from several standpoints
without cause.
 
> I also worry "special casing" drives at a core thing we've been trying
> to dissuade: Swift is not a special case and is part of OpenStack. It
> should be considered as such and adding more onto that line just
> divides the community further.
>
> I think that filling the role of "native optimizations that are
> inappropriate in python" is something we will continue to have
> requests for especially if one project has blessing to use this
> native-tool-optimization-thing.
>
 
This is my main disagreement with the all-or-nothing premise - we seem
to have a bad case of fallacy of composition. AFAIK we do not have other
components of OpenStack which are needing to do a read on a large number
of on disk files in parallel from python, let alone do it in a highly
performant way. Just because we have identified one non trivial problem
for a language does not mean we have lots more to come. It is entirely
possible that another project will have this same problem or we will
identify another problem in the future which requires a similar
solution, but I don't know why we expect to or even want to solve that
issue now before that case is clearly defined. Hopefully we don't have
the need to write more databases as part of OpenStack...
 
> So in short, lets not special case a project here.
>
>> Cheers,
>> Greg
 
Cheers,
Greg
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160523/51cd674c/attachment.html>