Open Stack

On Thu, May 19, 2016 at 6:19 AM, Thierry Carrez <thierry at openstack.org>
wrote:

> Hi everyone,
>
> The discussion on the addition of golang focuses on estimating community
> costs vs. technical benefits, so that the TC can make the right call for
> "OpenStack". From that discussion, we established that the costs for
> cross-project teams (Infra...) seem to be workable. There is still
> significant community fragmentation effects as we add another language,
> creating language-expert silos, duplicating efforts, and losing some
> productivity as some needlessly rewrite things in golang. But we seem
> generally ready to accept that cost because we are missing a tool in our
> toolbelt: a language that lets us do such native optimization.
>
> We have a number of projects in OpenStack that are more low-level than
> others and which require such optimization, and for those projects (or
> subprojects) our current mix of language is not satisfactory. The Swift
> team in particular has spent a lot of time trying to get the I/O behavior
> they require with hard disk access using Python, and they certainly did not
> jump on the golang bandwagon lightly.
>
>
I agree with this. it is totally reasonable to add golang (or a tool to
fill the need for native optimizations). The fragmentation concerns are
real, but filling this gap in the openstack tool-chain is important. It
will garner more interest in the project as it opens the doors to examine
the native level optimizations that are exceptionally hard in Python. While
this all could be done in Python, I would argue that there are legitimate
cases where a tool like golang is going to be a "better" fit for the job. I
would also argue that the TC should provide a strong guidance on where
golang should be used (initially). Specifically that it should be used in
cases where Python can demonstrably be shown to be the inferior tool for
the job and not as whole-sale replacement(s) for the core API/end-user
interactions. The type of break that the swift team is advocating for is a
great starting place. With the acknowledgement from the -infra team that
this is reasonable and workable, I am willing (with my TC hat on), agree to
include golang provided the recommendations are outlined for the initial
use-cases. As we see where things shake out with these initial use-cases I
am sure we will see expanded uses of golang that make sense. I do know that
policing such uses is difficult at best, and I don't think it is something
that should be done via technology but more of a social contract and
documented as the recommended for initial cases of golang (or other similar
tool-chain) inclusion.



> I believe the programming languages you need in OpenStack official
> projects are a function of the scope you define for OpenStack official
> projects. We wouldn't need to have JavaScript in the mix if we considered
> that web interfaces that purely consume OpenStack APIs are projects that
> consume OpenStack, rather than part of OpenStack itself.
>
> In the same vein, if we consider lower-level projects (which often require
> such native optimization) as part of "OpenStack", rather than as external
> open source projects that should be integrated by "OpenStack", then we need
> a language like golang in our toolbelt. There is basically no point in
> saying no to golang in OpenStack if we need lower-level native optimization
> in OpenStack: we'll have to accept the community cost that comes with such
> a community scope.
>
> So the real question we need to answer is... where does OpenStack stop,
> and where does the wider open source community start ? If OpenStack is
> purely an "integration engine", glue code for other lower-level
> technologies like hypervisors, databases, or distributed block storage,
> then the scope is limited, Python should be plenty enough, and we don't
> need to fragment our community. If OpenStack is "whatever it takes to reach
> our mission", then yes we need to add one language to cover
> lower-level/native optimization, because we'll need that... and we need to
> accept the community cost as a consequence of that scope choice. Those are
> the only two options on the table.
>
>
It would be disingenuous to say we are a pure integration engine. If swift
or a similar project is outside the scope of openstack, frankly, I view
Keystone as outside of the scope of openstack. Keystone (while in a
different position due to a lot of heavy requirements for auth in
openstack) is not a simple "integrate an underlying technology with other
things with some tracking of the resources". Keystone provides a real added
value, while perhaps not as low-level as swift, that extends significantly
beyond the scope of "glue" of it's individual components. I do believe the
lower level technologies have a place in "openstack's big tent" but should
be evaluated carefully before inclusion; I am not advocating we include
MySQL, PGSQL, or RabbitMQ as "part of the big tent". I think object storage
is an important thing within the cloud ecosystem, and by it's nature swift
belongs as part of the big tent.

The cost for golang, in my view after having a large number of
conversations, is worth it. This conversation will continue to come up in
different forms if we nix golang here, the low level optimizations are
relevant and the projects (initially swift) that are running up against
these issues are in-fact a part of OpenStack today.. I also want to make
sure we, as the TC and community, are not evaluating a specific project
here, but OpenStack as a whole. If we are evaluating one project
specifically and making a change based upon where it is in the ecosystem,
we should step way back and evaluate why this just now has become part of
the conversation and address that issue first


> I'm actually not sure what is the best answer. But I'm convinced we, as a
> community, need to have a clear answer to that. We've been avoiding that
> clear answer until now, creating tension between the advocates of an
> ASF-like collection of tools and the advocates of a tighter-integrated
> "openstack" product. We have created silos and specialized areas as we got
> into the business of developing time-series databases or SDNs. As a result,
> it's not "one community" anymore. Should we further encourage that, or
> should we focus on what the core of our mission is, what we have in common,
> this integration engine that binds all those other open source projects
> into one programmable infrastructure solution ?
>
> --
> Thierry Carrez (ttx)
>
>
I think we would do better as an ASF-like organization with a bias towards
programmable infrastructure. I do not think we should include more
time-series databases, RDBMSs, or even Message Brokers simply for the sake
of inclusion (there are examples of these in the big tent already). But we
should be careful about excluding and eliminating projects and scope as
well. We should include golang, and work on the other issues separately
(this sounds like something we should be working on setting proper
guideposts for the community, how we do that, etc, and revolves around
improving how the TC provides leadership). As part of improving the
leadership of OpenStack, we need to also work to have a clear
product-vision (and I do not mean "product" as in something specifically
sell-able). I think part of our issue and what is driving these
conversations is a lack of clear product vision which is part of the TC
providing the guideposts and improving leadership within OpenStack.

--
Morgan Fainberg (notmorgan)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160519/e4cd9d74/attachment.html>