<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body class="" style="word-wrap:break-word">
OpenStack is more then the sum of its various pieces. Often features need to cross more then one project. Cross project work is already extremely hard without having to change languages between. Language change should be done very carefully and deliberately.<br>
<br>
Thanks,<br>
Kevin <strong>
<div><font face="Tahoma" color="#000000" size="2"> </font></div>
</strong>
<hr tabindex="-1">
<font face="Tahoma" size="2"><b>From:</b> Geoff O'Callaghan<br>
<b>Sent:</b> Monday, May 23, 2016 5:59:13 PM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [all][tc] Languages vs. Scope of "OpenStack"<br>
</font><br>
<div></div>
<div>Surely openstack is defined by it’s capabilities and interfaces rather than it’s internals. Given the simplistic view that openstack is a collection of micro services connected by well defined api’s does it really matter what code is used inside that
micro service (or macro service )? If there is a community willing to bring and support code in language ‘x’, isn’t that better than worrying about the off chance of existing developers wishing to move projects and not knowing the target language? Is
there a fear that we’ll end up with a fork of nova (or others) written in rust ?
<div class="">If we’re not open to evolving then we’ll die.<br class="">
<div class=""><br class="">
</div>
<div class="">Just throwing in a different perspective.</div>
<div class=""><br class="">
</div>
<div class="">Sorry about the top-post but it applies in general to the below discussion</div>
<div class="">Tks</div>
<div class="">Geoff</div>
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On 24 May 2016, at 9:28 AM, Gregory Haynes <<a href="mailto:greg@greghaynes.net" class="">greg@greghaynes.net</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">
<div class="">On Mon, May 23, 2016, at 05:24 PM, Morgan Fainberg wrote:<br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class=""> <br class="">
</div>
<div class="">
<div class=""> <br class="">
</div>
<div class="">
<div class="">On Mon, May 23, 2016 at 2:57 PM, Gregory Haynes <span dir="ltr" class="">
<<a href="mailto:greg@greghaynes.net" class="">greg@greghaynes.net</a>></span> wrote:<br class="">
</div>
<blockquote class="" style="margin-top:0px; margin-right:0px; margin-bottom:0px; margin-left:0.8ex; border-left-width:1px; border-left-color:rgb(204,204,204); border-left-style:solid; padding-left:1ex">
<div class="">
<div class="">
<div class="">On Fri, May 20, 2016, at 07:48 AM, Thierry Carrez wrote:<br class="">
</div>
<div class="">> John Dickinson wrote:<br class="">
</div>
<div class="">> > [...]<br class="">
</div>
<div class="">> >> So the real question we need to answer is... where does OpenStack<br class="">
</div>
<div class="">> >> stop, and where does the wider open source community start ? If<br class="">
</div>
<div class="">> >> OpenStack is purely an "integration engine", glue code for other<br class="">
</div>
<div class="">> >> lower-level technologies like hypervisors, databases, or distributed<br class="">
</div>
<div class="">> >> block storage, then the scope is limited, Python should be plenty<br class="">
</div>
<div class="">> >> enough, and we don't need to fragment our community. If OpenStack is<br class="">
</div>
<div class="">> >> "whatever it takes to reach our mission", then yes we need to add one<br class="">
</div>
<div class="">> >> language to cover lower-level/native optimization, because we'll<br class="">
</div>
<div class="">> >> need that... and we need to accept the community cost as a<br class="">
</div>
<div class="">> >> consequence of that scope choice. Those are the only two options on<br class="">
</div>
<div class="">> >> the table.<br class="">
</div>
<div class="">> >><br class="">
</div>
<div class="">> >> I'm actually not sure what is the best answer. But I'm convinced we,<br class="">
</div>
<div class="">> >> as a community, need to have a clear answer to that. We've been<br class="">
</div>
<div class="">> >> avoiding that clear answer until now, creating tension between the<br class="">
</div>
<div class="">> >> advocates of an ASF-like collection of tools and the advocates of a<br class="">
</div>
<div class="">> >> tighter-integrated "openstack" product. We have created silos and<br class="">
</div>
<div class="">> >> specialized areas as we got into the business of developing time-<br class="">
</div>
<div class="">> >> series databases or SDNs. As a result, it's not "one community"<br class="">
</div>
<div class="">> >> anymore. Should we further encourage that, or should we focus on<br class="">
</div>
<div class="">> >> what the core of our mission is, what we have in common, this<br class="">
</div>
<div class="">> >> integration engine that binds all those other open source projects<br class="">
</div>
<div class="">> >> into one programmable infrastructure solution ?<br class="">
</div>
<div class="">> ><br class="">
</div>
<div class="">> > You said the answer in your question. OpenStack isn't defined as an<br class="">
</div>
<div class="">> > integration engine[3]. The definition of OpenStack is whatever it<br class="">
</div>
<div class="">> > takes to fulfill our mission[4][5]. I don't mean that as a tautology.<br class="">
</div>
<div class="">> > I mean that we've already gone to the effort of defining OpenStack. It's<br class="">
</div>
<div class="">> > our mission statement. We're all about building a cloud platform upon<br class="">
</div>
<div class="">> > which people can run their apps ("cloud-native" or otherwise), so we<br class="">
</div>
<div class="">> > write the software needed to do that.<br class="">
</div>
<div class="">> ><br class="">
</div>
<div class="">> > So where does OpenStack stop and the wider community start? OpenStack<br class="">
</div>
<div class="">> > includes the projects needed to fulfill its mission.<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> I'd totally agree with you if OpenStack was developed in a vacuum. But<br class="">
</div>
<div class="">> there is a large number of open source projects and libraries that<br class="">
</div>
<div class="">> OpenStack needs to fulfill its mission that are not in "OpenStack": they<br class="">
</div>
<div class="">> are external open source projects we depend on. Python, MySQL, libvirt,<br class="">
</div>
<div class="">> KVM, Ceph, OpenvSwitch, RabbitMQ... We are not asking that those should<br class="">
</div>
<div class="">> be included in OpenStack, and we are not NIHing replacements for those<br class="">
</div>
<div class="">> in OpenStack either.<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> So it is not as clear-cut as you present it, and you can approach this<br class="">
</div>
<div class="">> dependency question from two directions.<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> One is community-centric: "anything produced by our community is<br class="">
</div>
<div class="">> OpenStack". If we are missing a lower-level piece to achieve our mission<br class="">
</div>
<div class="">> and are developing it ourselves as a result, then it is OpenStack, even<br class="">
</div>
<div class="">> if it ends up being a message queue or a database.<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> The other approach is product-centric: "lower-level pieces are OpenStack<br class="">
</div>
<div class="">> dependencies, rather than OpenStack itself". If we are missing a<br class="">
</div>
<div class="">> lower-level piece to achieve our mission and are developing it as a<br class="">
</div>
<div class="">> result, it could be developed on OpenStack infrastructure by members of<br class="">
</div>
<div class="">> the OpenStack community but it is not "OpenStack the product", it's an<br class="">
</div>
<div class="">> OpenStack *dependency*. It is not governed by the TC, it can use any<br class="">
</div>
<div class="">> language and tool deemed necessary.<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> On this second approach, there is the obvious question of where<br class="">
</div>
<div class="">> "lower-level" starts, which as you explained above is not really<br class="">
</div>
<div class="">> clear-cut. A good litmus test for it could be whenever Python is not<br class="">
</div>
<div class="">> enough. If you can't develop it effectively with the language that is<br class="">
</div>
<div class="">> currently sufficient for the rest of OpenStack, then developing it as an<br class="">
</div>
<div class="">> OpenStack dependency in whatever language is appropriate might be the<br class="">
</div>
<div class="">> solution...<br class="">
</div>
<div class="">><br class="">
</div>
<div class="">> That is what I mean by 'scope': where does "OpenStack" stop, and where<br class="">
</div>
<div class="">> do "OpenStack dependencies" start ? It is a lot easier and a lot less<br class="">
</div>
<div class="">> community-costly to allow additional languages in OpenStack dependencies<br class="">
</div>
<div class="">> (we already have plenty there).<br class="">
</div>
<div class="">><br class="">
</div>
<div class=""> <br class="">
</div>
</div>
</div>
<div class="">I strongly agree with both of the points about what OpenStack is defined<br class="">
</div>
<div class="">as. We are a set of projects attempting to fulfill our mission. In<br class="">
</div>
<div class="">doing so, we try to use outside dependencies to help us as much as<br class="">
</div>
<div class="">possible. Sometimes we cannot find an outside dependency to satisfy a<br class="">
</div>
<div class="">need whether due to optimization needs, licensing issues, usability<br class="">
</div>
<div class="">problems, or simply because an outside project doesn't exist. That is<br class="">
</div>
<div class="">when things become less clear-cut and we might need to develop software<br class="">
</div>
<div class="">not purely/directly related to fulfilling our mission.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">In the product-centric approach I worry that we are going to be paying<br class="">
</div>
<div class="">many of the costs which the existing TC resolutions hoped to prevent. We<br class="">
</div>
<div class="">still will have to maintain and debug these dependencies and if the<br class="">
</div>
<div class="">OpenStack community is the only ones doing so then the language<br class="">
</div>
<div class="">fragmentation costs will come in to play. Our goal should be to only pay<br class="">
</div>
<div class="">these costs when necessary, and I am not sure this type of distinction<br class="">
</div>
<div class="">helps us do that.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">Additionally, I am not sure how this would solve the swift case. The<br class="">
</div>
<div class="">object server is the component that has been rewritten so either it<br class="">
</div>
<div class="">would need to live outside of OpenStack with the rest of Swift being<br class="">
</div>
<div class="">part of OpenStack, or all of Swift would need to move outside. Neither<br class="">
</div>
<div class="">of these seems ideal.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">This makes me wonder - does this decision need to be an all-or-nothing<br class="">
</div>
<div class="">thing? Can we say that the swift object server is a special case, let it<br class="">
</div>
<div class="">be written in another language, and leave it at that? My thinking is -<br class="">
</div>
<div class="">there is only value in us deciding on and allowing an additional<br class="">
</div>
<div class="">language for solving these issues generally if there are other places<br class="">
</div>
<div class="">that require this new language. It has been mentioned that Gnocchi could<br class="">
</div>
<div class="">potentially be a case in the future but until that problem actually<br class="">
</div>
<div class="">comes up there isn't much benefit to us deciding on another language<br class="">
</div>
<div class="">anyone can use and there are plenty of downsides to doing so.<br class="">
</div>
<div class=""> <br class="">
</div>
</blockquote>
<div class=""> <br class="">
</div>
<div class="">I really do not want to "special case" swift. It really doesn't go with the spirit of inclusion.<br class="">
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> <br class="">
</div>
<div class="">I am not sure how inclusion is related to special casing. Inclusion here implies that some group is not being accepted in to our community. The excluded group here would be one writing software not in python, which is the same groups being excluded
currently. This is an argument against the status quo, not against special casing.<br class="">
</div>
<div class=""> <br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div class="">I expect we will see a continued drive for whichever additional languages are supported once it's in place/allowed.<br class="">
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> <br class="">
</div>
<div class="">That is the problem. The social and debugging costs of adding another language are a function of how much that other language is used. If one component of one project is written in another language then these costs should be fairly low. I agree
that once we allow another language we should expect many projects to begin using it, and IMO most if not all of these cases except swift will not be warranted. Thus, we will be paying the costs in development resources from several standpoints without cause.<br class="">
</div>
<div class=""> <br class="">
</div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div class="">I also worry "special casing" drives at a core thing we've been trying to dissuade: Swift is not a special case and is part of OpenStack. It should be considered as such and adding more onto that line just divides the community further.<br class="">
</div>
<div class=""> <br class="">
</div>
<div class="">I think that filling the role of "native optimizations that are inappropriate in python" is something we will continue to have requests for especially if one project has blessing to use this native-tool-optimization-thing.<br class="">
</div>
<div class=""> <br class="">
</div>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">This is my main disagreement with the all-or-nothing premise - we seem to have a bad case of fallacy of composition. AFAIK we do not have other components of OpenStack which are needing to do a read on a large number of on disk files in parallel
from python, let alone do it in a highly performant way. Just because we have identified one non trivial problem for a language does not mean we have lots more to come. It is entirely possible that another project will have this same problem or we will identify
another problem in the future which requires a similar solution, but I don't know why we expect to or even want to solve that issue now before that case is clearly defined. Hopefully we don't have the need to write more databases as part of OpenStack...<br class="">
</div>
<div class=""> </div>
<blockquote type="cite" class="">
<div dir="ltr" class="">
<div class="">
<div class="">
<div class="">So in short, lets not special case a project here.<br class="">
</div>
<div class=""> <br class="">
</div>
<blockquote class="" style="margin-top:0px; margin-right:0px; margin-bottom:0px; margin-left:0.8ex; border-left-width:1px; border-left-color:rgb(204,204,204); border-left-style:solid; padding-left:1ex">
<div class="">Cheers,<br class="">
</div>
<div class="">Greg<br class="">
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
<div class=""> </div>
<div class="">Cheers,<br class="">
</div>
<div class="">Greg</div>
</div>
__________________________________________________________________________<br class="">
OpenStack Development Mailing List (not for usage questions)<br class="">
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" class="">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</body>
</html>