Open Stack

Openstack-common could be great. There are lots of use cases that make a lot of sense to put in openstack common. Configuration loading, context, some aspects of logging, wsgi middleware, some parts of utils--those seem to me like great opportunities to save time and effort, both writing and reading code, through code reuse.

Unfortunately, openstack-common could also be horrible. When a problematic module makes it into openstack-common it gets enshrined in "rough consensus" and is much harder to change or replace. If there are divergent use cases for that module, no doubt the common implementation of it will be riddled with conditional code paths, with each path only truly being exercised by one project (illusory reuse). If the module frequently has to change, then backwards compatibility requirements will make the code even more complicated and harder to understand and change. The requirement that "there is no other API in OpenStack competing for this consensus" means that, when a spaghetti-code module ought to be simply replaced or removed from common, fewer people will be willing to do this work because they now have to change the calling code in N projects.

We need to think carefully about how we will avoid and address this problem. I have a few proposals. I'm not 100% certain of any of these, but I'd like for you to consider them.

1) The process of adding code to common should require input from every core project. This might take the form of requiring one core member from each to approve the merge.

2) We should plan to be restrictive about what lands in common. We should plan to fight complexity and illusory reuse before something lands in common rather than relying exclusively on iterating on it after it is in common.

3) Rather than forking jkoelker/openstack-common, we should create a new project and merge sections in piecemeal. This creates the opportunity to review each part separately.

4) Above and beyond #2 and #3, our first steps should err on the side of safety.

  So while I feel that a lot of the modules in the current openstack-common github repo definitely belong there, I am really worried about the web framework and serialization code. That code in particular feels likely to change and likely to have different uses in different projects. As such it will probably bite us with complexity both from backwards compatibility and illusory reuse. Maybe we can refactor and rewrite to avoid these problems, but we should probably do that before adding it to common rather than after.

As always, just my 2 cents, but I hope there are others out there who are similarly concerned that openstack-common might not be worth it if we just try to go full-tilt on code reuse. 

"Mark McLoughlin" <markmc at redhat.com> said:

> Hey,
> 
> As Jason says - another year, another openstack-common thread! :-)
> 
> I've just written up the plan Jason and I have for openstack-common:
> 
>    http://wiki.openstack.org/CommonLibrary
> 
> (also pasted below to make it easier to reply to)
> 
> I guess what we're trying to do is quickly get this thing into good
> enough shape to do a first release. Even if that release only contains a
> handful of APIs, but they meet the criteria below, then we'll have a
> really solid foundation to build on.
> 
> Thoughts?
> 
> Cheers,
> Mark.
> 
> The openstack-common project intends to produce a python library containing
> infrastructure code shared by OpenStack projects. The APIs provided by the
> project should be high quality, stable, consistent and generally useful.
> 
> The existence of an API in openstack-common should be indicitative of rough
> consensus across the project on that API's design. New OpenStack projects should
> be able to use an API in the library safe in the knowledge that, by doing so,
> the project is being a good OpenStack citizen and building upon established
> best practice.
> 
> To that end, a number of principles should be adhered to when considering any
> proposed API for openstack-common:
> 
>   1) The API is generally useful and is a "good fit" - e.g. it doesn't encode
>      some assumptions specific to the project it originated from, it should
>      follow a style consistent with other openstack-common APIs and should
>      fit generally in a theme like error handling, configuration options,
>      time and date, notifications, WSGI, etc.
> 
>   2) The API is already in use by a number of OpenStack projects
> 
>   3) There is a commitment to adopt the API in all other OpenStack projects
>      (where appropriate) and there are no known major blockers to that adoption
> 
>   4) The API represents the "rough consensus" across OpenStack projects
> 
>   5) There is no other API in OpenStack competing for this "rough consensus"
> 
>   6) It should be possible for the API to evolve while continuing to maintain
>      backwards compatibility with older versions for a reasonable period - e.g.
>      compatibility with an API deprecated in release N may only be removed in
>      release N+2
> 
> There have been several attempts at kick-starting openstack-common, each attempt
> beginning with moving a number of existing APIs from Glance or Nova into a new
> repository. None of these attempts have quite managed to reach the point where
> they were ready for other OpenStack projects to depend on the library.
> 
> This proposal marks the beginning of yet another attempt. The idea is to create
> a new openstack-common repository, seed it with Jason Kölker's excellent
> infrastructure from his repository[1] and begin importing individual
> APIs while applying the principles above during the review process for
> each proposed API.
> 
> [1] - https://github.com/jkoelker/openstack-common
> 
> 
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to     : openstack at lists.launchpad.net
> Unsubscribe : https://launchpad.net/~openstack
> More help   : https://help.launchpad.net/ListHelp
>