Open Stack

On Mon, May 23, 2016 at 4:28 PM, Gregory Haynes <greg at greghaynes.net> wrote:

> On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:
>
>
>
> On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes <greg at greghaynes.net>
> wrote:
>
> On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:
> > John Dickinson wrote:
> > > [...]
> > >> So the real question we need to answer is... where does OpenStack
> > >> stop, and where does the wider open source community start ? If
> > >> OpenStack is purely an "integration engine", glue code for other
> > >> lower-level technologies like hypervisors, databases, or distributed
> > >> block storage, then the scope is limited, Python should be plenty
> > >> enough, and we don't need to fragment our community. If OpenStack is
> > >> "whatever it takes to reach our mission", then yes we need to add one
> > >> language to cover lower-level/native optimization, because we'll
> > >> need that... and we need to accept the community cost as a
> > >> consequence of that scope choice. Those are the only two options on
> > >> the table.
> > >>
> > >> I'm actually not sure what is the best answer. But I'm convinced we,
> > >> as a community, need to have a clear answer to that. We've been
> > >> avoiding that clear answer until now, creating tension between the
> > >> advocates of an ASF-like collection of tools and the advocates of a
> > >> tighter-integrated "openstack" product. We have created silos and
> > >> specialized areas as we got into the business of developing time-
> > >> series databases or SDNs. As a result, it's not "one community"
> > >> anymore. Should we further encourage that, or should we focus on
> > >> what the core of our mission is, what we have in common, this
> > >> integration engine that binds all those other open source projects
> > >> into one programmable infrastructure solution ?
> > >
> > > You said the answer in your question. OpenStack isn't defined as an
> > > integration engine[3]. The definition of OpenStack is whatever it
> > > takes to fulfill our mission[4][5]. I don't mean that as a tautology.
> > > I mean that we've already gone to the effort of defining OpenStack.
> It's
> > > our mission statement. We're all about building a cloud platform upon
> > > which people can run their apps ("cloud-native" or otherwise), so we
> > > write the software needed to do that.
> > >
> > > So where does OpenStack stop and the wider community start? OpenStack
> > > includes the projects needed to fulfill its mission.
> >
> > I'd totally agree with you if OpenStack was developed in a vacuum. But
> > there is a large number of open source projects and libraries that
> > OpenStack needs to fulfill its mission that are not in "OpenStack": they
> > are external open source projects we depend on. Python, MySQL, libvirt,
> > KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should
> > be included in OpenStack, and we are not NIHing replacements for those
> > in OpenStack either.
> >
> > So it is not as clear-cut as you present it, and you can approach this
> > dependency question from two directions.
> >
> > One is community-centric: "anything produced by our community is
> > OpenStack". If we are missing a lower-level piece to achieve our mission
> > and are developing it ourselves as a result, then it is OpenStack, even
> > if it ends up being a message queue or a database.
> >
> > The other approach is product-centric: "lower-level pieces are OpenStack
> > dependencies, rather than OpenStack itself". If we are missing a
> > lower-level piece to achieve our mission and are developing it as a
> > result, it could be developed on OpenStack infrastructure by members of
> > the OpenStack community but it is not "OpenStack the product", it's an
> > OpenStack *dependency*. It is not governed by the TC, it can use any
> > language and tool deemed necessary.
> >
> > On this second approach, there is the obvious question of where
> > "lower-level" starts, which as you explained above is not really
> > clear-cut. A good litmus test for it could be whenever Python is not
> > enough. If you can't develop it effectively with the language that is
> > currently sufficient for the rest of OpenStack, then developing it as an
> > OpenStack dependency in whatever language is appropriate might be the
> > solution...
> >
> > That is what I mean by 'scope': where does "OpenStack" stop, and where
> > do "OpenStack dependencies" start ? It is a lot easier and a lot less
> > community-costly to allow additional languages in OpenStack dependencies
> > (we already have plenty there).
> >
>
> I strongly agree with both of the points about what OpenStack is defined
> as. We are  a set of projects attempting to fulfill our mission. In
> doing so, we try to use outside dependencies to help us as much as
> possible. Sometimes we cannot find an outside dependency to satisfy a
> need whether due to optimization needs, licensing issues, usability
> problems, or simply because an outside project doesn't exist. That is
> when things become less clear-cut and we might need to develop software
> not purely/directly related to fulfilling our mission.
>
> In the product-centric approach I worry that we are going to be paying
> many of the costs which the existing TC resolutions hoped to prevent. We
> still will have to maintain and debug these dependencies and if the
> OpenStack community is the only ones doing so then the language
> fragmentation costs will come in to play. Our goal should be to only pay
> these costs when necessary, and I am not sure this type of distinction
> helps us do that.
>
> Additionally, I am not sure how this would solve the swift case. The
> object server is the component that has been rewritten so either it
> would need to live outside of OpenStack with the rest of Swift being
> part of OpenStack, or all of Swift would need to move outside. Neither
> of these seems ideal.
>
> This makes me wonder - does this decision need to be an all-or-nothing
> thing? Can we say that the swift object server is a special case, let it
> be written in another language, and leave it at that? My thinking is -
> there is only value in us deciding on and allowing an additional
> language for solving these issues generally if there are other places
> that require this new language. It has been mentioned that Gnocchi could
> potentially be a case in the future but until that problem actually
> comes up there isn't much benefit to us deciding on another language
> anyone can use and there are plenty of downsides to doing so.
>
>
>
> I really do not want to "special case" swift. It really doesn't go with
> the spirit of inclusion.
>
>
> I am not sure how inclusion is related to special casing. Inclusion here
> implies that some group is not being accepted in to our community. The
> excluded group here would be one writing software not in python, which is
> the same groups being excluded currently. This is an argument against the
> status quo, not against special casing.
>
>
> I expect we will see a continued drive for whichever additional languages
> are supported once it's in place/allowed.
>
>
> That is the problem. The social and debugging costs of adding another
> language are a function of how much that other language is used. If one
> component of one project is written in another language then these costs
> should be fairly low. I agree that once we allow another language we should
> expect many projects to begin using it, and IMO most if not all of these
> cases except swift will not be warranted. Thus, we will be paying the costs
> in development resources from several standpoints without cause.
>
>
> I also worry "special casing" drives at a core thing we've been trying to
> dissuade: Swift is not a special case and is part of OpenStack. It should
> be considered as such and adding more onto that line just divides the
> community further.
>
> I think that filling the role of "native optimizations that are
> inappropriate in python" is something we will continue to have requests for
> especially if one project has blessing to use this
> native-tool-optimization-thing.
>
>
>
> This is my main disagreement with the all-or-nothing premise - we seem to
> have a bad case of fallacy of composition. AFAIK we do not have other
> components of OpenStack which are needing to do a read on a large number of
> on disk files in parallel from python, let alone do it in a highly
> performant way. Just because we have identified one non trivial problem for
> a language does not mean we have lots more to come. It is entirely possible
> that another project will have this same problem or we will identify
> another problem in the future which requires a similar solution, but I
> don't know why we expect to or even want to solve that issue now before
> that case is clearly defined. Hopefully we don't have the need to write
> more databases as part of OpenStack...
>
>

I am fine with this in premise, but I just want to be clear where I am
coming from on  the inclusion of golang (or rust, or C++, or JavaScript, or
Java, etc, etc). If we are accepting golang, I want it to be clearly
documented that the expectation is it is used exclusively where there is a
demonstrable case (such as with swift) and not a carte blanche to use it
wherever-you-please.

I want this to be a social contract looked at and enforced by the
community, not special permissions that are granted by the TC (I don't want
the TC to need to step in an approve every single use case of golang, or
javascript ...). It's bottlenecking back to the TC for special permissions
or inclusion (see reasons for the dissolution of the "integrated release").

This isn't strictly an all or nothing case, this is a "how would we enforce
this?" type deal. Lean on infra to enforce that only projects with the
golang-is-ok-here tag are allowed to use it? I don't want people to write
their APIs in javascript (and node.js) nor in golang. I would like to see
most of the work continue with python as the primary language. I just think
it's unreasonable to lock tools behind a gate that is stronger than the
social / community contract (and outlined in the resolution including X
language).

So, I'm not disagreeing with you, I am disagreeing with needing to grant
special permissions to use a tool. If a project is doing something totally
crazy, we can look into/address from the technical committee (and yes,
writing your apis in golang for the sake of writing your apis in golang is
somewhat crazy).

> So in short, lets not special case a project here.
>
>
> Cheers,
> Greg
>
>
> Cheers,
> Greg
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160523/72aa2692/attachment-0001.html>