Open Stack

Morgan Fainberg wrote:
>
>
> On Thu, May 19, 2016 at 6:19 AM, Thierry Carrez <thierry at openstack.org
> <mailto:thierry at openstack.org>> wrote:
>
>     Hi everyone,
>
>     The discussion on the addition of golang focuses on estimating
>     community costs vs. technical benefits, so that the TC can make the
>     right call for "OpenStack". From that discussion, we established
>     that the costs for cross-project teams (Infra...) seem to be
>     workable. There is still significant community fragmentation effects
>     as we add another language, creating language-expert silos,
>     duplicating efforts, and losing some productivity as some needlessly
>     rewrite things in golang. But we seem generally ready to accept that
>     cost because we are missing a tool in our toolbelt: a language that
>     lets us do such native optimization.
>
>     We have a number of projects in OpenStack that are more low-level
>     than others and which require such optimization, and for those
>     projects (or subprojects) our current mix of language is not
>     satisfactory. The Swift team in particular has spent a lot of time
>     trying to get the I/O behavior they require with hard disk access
>     using Python, and they certainly did not jump on the golang
>     bandwagon lightly.
>
> I agree with this. it is totally reasonable to add golang (or a tool to
> fill the need for native optimizations). The fragmentation concerns are
> real, but filling this gap in the openstack tool-chain is important. It
> will garner more interest in the project as it opens the doors to
> examine the native level optimizations that are exceptionally hard in
> Python. While this all could be done in Python, I would argue that there
> are legitimate cases where a tool like golang is going to be a "better"
> fit for the job. I would also argue that the TC should provide a strong
> guidance on where golang should be used (initially). Specifically that
> it should be used in cases where Python can demonstrably be shown to be
> the inferior tool for the job and not as whole-sale replacement(s) for
> the core API/end-user interactions. The type of break that the swift
> team is advocating for is a great starting place. With the
> acknowledgement from the -infra team that this is reasonable and
> workable, I am willing (with my TC hat on), agree to include golang
> provided the recommendations are outlined for the initial use-cases. As
> we see where things shake out with these initial use-cases I am sure we
> will see expanded uses of golang that make sense. I do know that
> policing such uses is difficult at best, and I don't think it is
> something that should be done via technology but more of a social
> contract and documented as the recommended for initial cases of golang
> (or other similar tool-chain) inclusion.
>
>     I believe the programming languages you need in OpenStack official
>     projects are a function of the scope you define for OpenStack
>     official projects. We wouldn't need to have JavaScript in the mix if
>     we considered that web interfaces that purely consume OpenStack APIs
>     are projects that consume OpenStack, rather than part of OpenStack
>     itself.
>
>     In the same vein, if we consider lower-level projects (which often
>     require such native optimization) as part of "OpenStack", rather
>     than as external open source projects that should be integrated by
>     "OpenStack", then we need a language like golang in our toolbelt.
>     There is basically no point in saying no to golang in OpenStack if
>     we need lower-level native optimization in OpenStack: we'll have to
>     accept the community cost that comes with such a community scope.
>
>     So the real question we need to answer is... where does OpenStack
>     stop, and where does the wider open source community start ? If
>     OpenStack is purely an "integration engine", glue code for other
>     lower-level technologies like hypervisors, databases, or distributed
>     block storage, then the scope is limited, Python should be plenty
>     enough, and we don't need to fragment our community. If OpenStack is
>     "whatever it takes to reach our mission", then yes we need to add
>     one language to cover lower-level/native optimization, because we'll
>     need that... and we need to accept the community cost as a
>     consequence of that scope choice. Those are the only two options on
>     the table.
>
>
> It would be disingenuous to say we are a pure integration engine. If
> swift or a similar project is outside the scope of openstack, frankly, I
> view Keystone as outside of the scope of openstack. Keystone (while in a
> different position due to a lot of heavy requirements for auth in
> openstack) is not a simple "integrate an underlying technology with
> other things with some tracking of the resources". Keystone provides a
> real added value, while perhaps not as low-level as swift, that extends
> significantly beyond the scope of "glue" of it's individual components.
> I do believe the lower level technologies have a place in "openstack's
> big tent" but should be evaluated carefully before inclusion; I am not
> advocating we include MySQL, PGSQL, or RabbitMQ as "part of the big
> tent". I think object storage is an important thing within the cloud
> ecosystem, and by it's nature swift belongs as part of the big tent.
> The cost for golang, in my view after having a large number of
> conversations, is worth it. This conversation will continue to come up
> in different forms if we nix golang here, the low level optimizations
> are relevant and the projects (initially swift) that are running up
> against these issues are in-fact a part of OpenStack today.. I also want
> to make sure we, as the TC and community, are not evaluating a specific
> project here, but OpenStack as a whole. If we are evaluating one project
> specifically and making a change based upon where it is in the
> ecosystem, we should step way back and evaluate why this just now has
> become part of the conversation and address that issue first
>

+1

I really dislike the words 'integration engine' because IMHO its not 
honest, its not interesting and I don't feel its really innovative and 
attractive to developers that are coming to work on a opensource project 
(for say there first time). If I was a new developer out of college and 
was going to be working on a 'integration engine' (and/or glue engine) 
I'd be running for the hills. We really need to work on a better terms 
here IMHO ;)

>     I'm actually not sure what is the best answer. But I'm convinced we,
>     as a community, need to have a clear answer to that. We've been
>     avoiding that clear answer until now, creating tension between the
>     advocates of an ASF-like collection of tools and the advocates of a
>     tighter-integrated "openstack" product. We have created silos and
>     specialized areas as we got into the business of developing
>     time-series databases or SDNs. As a result, it's not "one community"
>     anymore. Should we further encourage that, or should we focus on
>     what the core of our mission is, what we have in common, this
>     integration engine that binds all those other open source projects
>     into one programmable infrastructure solution ?
>
>     --
>     Thierry Carrez (ttx)
>
>
> I think we would do better as an ASF-like organization with a bias
> towards programmable infrastructure. I do not think we should include
> more time-series databases, RDBMSs, or even Message Brokers simply for
> the sake of inclusion (there are examples of these in the big tent
> already). But we should be careful about excluding and eliminating
> projects and scope as well. We should include golang, and work on the
> other issues separately (this sounds like something we should be working
> on setting proper guideposts for the community, how we do that, etc, and
> revolves around improving how the TC provides leadership). As part of
> improving the leadership of OpenStack, we need to also work to have a
> clear product-vision (and I do not mean "product" as in something
> specifically sell-able). I think part of our issue and what is driving
> these conversations is a lack of clear product vision which is part of
> the TC providing the guideposts and improving leadership within OpenStack.
>
> --
> Morgan Fainberg (notmorgan)
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev