Open Stack

Excerpts from Flavio Percoco's message of 2016-11-07 11:32:06 -0800:
> On 07/11/16 12:16 -0600, Monty Taylor wrote:
> >On 11/07/2016 11:58 AM, Hayes, Graham wrote:
> >> On 07/11/2016 17:14, Flavio Percoco wrote:
> >>> Greetings,
> >>>
> >>> I literally just posted a thing on my blog with some thoughts of what I'd expect
> >>> any new language being proposed for OpenStack to cover before it can be
> >>> accepted.
> >>>
> >>> The goal is to set the expectations right for what's needed for new languages to
> >>> be accepted in the community. During the last evaluation of the Go proposal I
> >>> raised some concerns but I also said I was not closed to discussing this again
> >>> in the future. It's clear we've not documented expectations yet and this is a
> >>> first step to get that going before a new proposal comes up and we start
> >>> discussing this topic again.
> >>>
> >>> I don't think a blog post is the "way we should do this" but it was my way to
> >>> dump what's in my brain before working on something official (which I'll do
> >>> this/next week).
> >>>
> >>> I also don't think this list is perfect. It could either be too restrictive or
> >>> too open but it's a first step. I won't paste the content of my post in this
> >>> email but I'll provide a tl;dr and eventually come back with the actual
> >>> reference document patch. I thought I'd drop this here in case people read my
> >>> post and were confused about what's going on there.
> >>>
> >>> Ok, here's the TL;DR of what I believe we should know/do before accepting a new
> >>> language into the community:
> >>
> >> Its a great starting point, but there is one issue:
> >>
> >> This is a *huge* amount of work to get into place, for the TC to still
> >> potentially refuse the language. (I know you covered this in your blog
> >> post, but I think there is a level of underestimation there.)
> >
> >Yes, it is absolutely a huge amount of work for sure, and I think your
> >point is important.
> >
> >>
> >>> - Define a way to share code/libraries for projects using the language
> >>
> >> ++ - Definitely needed
> >>
> >>> - Work on a basic set of libraries for OpenStack base services
> >>
> >> What do we include here? You called out these:
> >>
> >> keystoneauth / keystone-client
> >> oslo.config
> >> oslo.db
> >> oslo.messaging
> >>
> >> We need to also include
> >>
> >> oslo.log
> >>
> >> Do they *all* need to be implemented? Just some? Do they need feature
> >> parity?
> >
> >To me the most important piece is feature parity on the operator
> >interaction side.
> >
> >Concretely - oslo.config is important because it shapes what the config
> >files look like. The implementation language of a particular service
> >shouldn't change what our config files look like, yeah?
> >
> >Similarly keystoneauth - not because getting a token from keystone with
> >a username and password is necessarily hard- but because we're trying to
> >drive more service-service interactions through the catalog to reduce
> >the number of things an operator needs to configure directly - and also
> >because keystone has auth plugins that need to be supported in the new
> >language too. (it's a common fault in non-python client implementations
> >that non-password auth plugins are not covered at all)
> >
> >oslo.log has similar needs - the logging for an operator needs to be
> >able to be routed to the same places and be in the same format as the
> >existing things.
> >
> >oslo.messaging and oslo.db have needs where they intersect with operator
> >config.
> 
> 
> I agree with Monty here. If I were to "prioritize" the work on these libraries,
> I'd say parity on the operator's side of things should be the priority. The rest
> of the libraries could be implemented as they are needed.
> 
> I don't think they all need to be implemented but I do think some of them
> should, especially because I believe the process of adding a new language should
> also involve building the ecosystem within the openstack community for that
> language to exist and be consumed.
> 
> Wether this should be a hard requirement or not is something that we can talk
> more about. What I would like to avoid is a scenario were the service/team
> requesting the addition of the language doesn't use any of the common tools that
> exist today and we end up adding a new language and dropping all the work on
> compatibility to teams coming after. I believe this is exactly the scenario that
> we're in right now.

Right. It's much more important to me that a system be put in place
to produce and manage the libraries than that they are created in
any particular order. Oslo came after Swift and Nova, but we've
learned that building shared libraries is an important part of our
community's answer to the call for consistency that needs to extend
to new languages from the beginning of their adoption.

> >> For example the previous discussion about Go had 2 components that would
> >> have required at most 2 of these base libraries (and I think that was
> >> mainly on the Designate side - the swift implementation did not need
> >> any (I think - I am open to correction)
> >>
> >>> - Define how the deliverables are distributed
> >>
> >> ++ - but this needs to be done with the release management teams input.
> >> What I think is reasonable may not be something that they are interested
> >> in supporting.
> 
> Absolutely. I mentioned in the post that working with the release team is key
> for this.
> 
> >>> - Define how stable maintenance will work
> >>
> >> ++
> >>
> >>> - Setup the CI pipelines for the new language
> >>
> >> This requires the -infra team dedicate (what are already stretched)
> >> resources to a theoretical situation, doing work that may be thrown
> >> away if the TC rejects a language.
> >>
> 
> Ditto... Regardless of how swamped the infra team is, I believe this requires
> working with them. Actually, I don't see the infra team being streched as a
> problem here, it just means that we need to fit the addition of the new language
> into the community priorities (including the infra's team time), which should
> help us with making the right calls at the right time.
> 
> In other words, if we can't fit adding a new language into the infra's team
> priorities, then it might not be the right time for this addition to happen.
> 
> As far as the language being rejected, what I'm trying to do by laying down this
> requirements is building a process for which we can make more objective calls
> for adding new languages. It might very well mean that these process should be
> split into 2 phases. The first phase to evaluate the need of the new language
> and the second phase to check that these requirements have been met.
> 
> If the first phase is approved, then the second phase should always end up in
> the language being approved if the requirements are met.
> 
> >> I foresee these requirements as a whole being overly onerous, and
> >> having the same result as saying "no new languages".
> >>
> >> I think that there should be base research into all of these, but the
> >> requirements for some of these to be fully completed could be basically
> >> impossible.
> >
> >Having a solution for deliverable distribution, stable maintenance,
> >shared requirements management and caching/mirroring for the gate are
> >things I personally think are pre-requisites. They're conceptually hard
> >but not necessarily a giant amount of hands-on-ground work. (some work,
> >but not an excessive amount) They represent the parts of the
> >intra-developer social contract that exists within the structures of how
> >we operate on repositories.
> >
> 
> ++
> 
> >I would think that library coverage needs a plan, but doesn't need to be
> >_finished_ - until such a time as the deliverable in question needs to
> >be delivered. Or, put another way - we need to have a clear list of
> >things and a mechanism for doing them - but we don't need a logging
> >library until we have a thing that needs to log. That said, the instant
> >we have a thing in the new language that needs to log, it needs to be
> >understood that it needs to be done in a logging-library manner and that
> >the design needs to be strongly tied to the existing operator interface
> >for existing OpenStack components.
> >
> >I agree that actually writing duplicates of all the oslo libraries
> >before doing any coding is way too high of a barrier.
> >
> >On the other hand, before patches can land, someone needs to know how
> >zuul is going to properly clone the git repos associated with the
> >project, or else the code in question will not be properly tested and
> >we'll be in a painful position, no?
> 
> ++
> 
> As I mentioned in my post, I do know this is quite some work, which is why I
> think this is likely going to be a multi-cycle effort. However, I believe lots
> of this work is critical for any language to be successfully used within
> OpenStack. It took us quite a bit to get here with Python and we need to
> guarantee that any new language addition starts more or less from the same point
> we're at with Python.
> 
> Flavio
>