Open Stack

I like the idea of a Quantum ecosystem repo because it provides a catalog of available software and fits within the notion that OpenStack should have a focused core interface: compute, networking, and storage.  The ecosystem also promotes a healthy platform layer that is built upon the core.

The ecosystem repo should require minimal set of tests: pep8, unit tests,  minimum coverage threshold, and basic certification test(s).  The certification tests should check for compliance with the service/plugin interface we're developing.  For example,  a VPN or LBaaS module will load and can be inspected for it's capabilities with some default set of configuration for the test runner.  The certification test won't check for functionality because I think that's outside the scope of things we can test.

The ecosystem solves several current issues:
	- Plugin reviews take time away from core and community bugs/blueprints
	- Plugins/drivers that require physical devices are hard to realistically validate.
	- An ecosystem repo places the onus on quality/working code back on the vendor/project promoting it.
	- Code bloat within the main repository as vendor push code for visibility.
	- Platform projects do not have a visible place to grow around Quantum (e.g. DNSaaS).


Separate repos would be the ideal setup, but that will take significant resources. We could start with a contrib directory within Quantum.  The contrib directory would have the testing requirements from above with the following additional rules:
	- Quantum core code cannot import anything directly from contrib.  (Imports must be done via optional configuration options).
	- Clearly indicate that contrib projects can be removed between releases for failing to maintain compatibility with current release.
	- Each contrib project should have a designated maintainer responsible for bug fixes and compatibility.  The maintainer (or designate) should be present for online Quantum meetings during a release cycle. 
	- Only a single core dev is required to approve changes to contrib if it is submitted by the maintainer.  Otherwise the maintainer must +1 the change first.
	- License must be Apache 2 with contributors completing OpenStack CLA process.  This will make it easy for contrib projects to be merged into Quantum core if they gain widespread community support.

mark

On Nov 5, 2012, at 3:46 PM, Dan Wendlandt <dan at nicira.com> wrote:

> Hi everyone,
> 
> We started discussing this at the last quantum team meeting, but figured it would benefit from a broader audience. 
> 
> There is a lot of interest from networking vendors and some open source project to integrate with OpenStack via Quantum.  This is great, as this was one of the key goals of Quantum.  The point of this thread is to discuss how we make sure we enable this while avoiding some potential negative side-effects.  
> 
> The primary concerns seem to be: 
> 
> 1) concern that the core dev team will be overwhelmed having to review and potentially fix large amounts of vendor-specific code, taking aware their cycles from other community tasks.  The concern is both that this may not be a good use of core dev time, and that the core devs may not be sufficiently familiar with the vendor backend to meaningfully review the code, and may be unable to test the code if they don't have access to the required hardware/software.  
> 
> 2) concern that vendors may contribute code, but not follow up to fix bugs, answer questions on the ML, update code when needed, etc.  Basically, bit-rot.  
> 
> These are not new concerns, and to date the Quantum team has had the following policy posted on the wiki (decided on during a team meeting during the Folsom cycle): 
> 
> "Quantum plugins may be open source or proprietary. Some of the open source plugins are maintained in the main Quantum repository (http://www.github.com/openstack/quantum), while others are maintained in their own separate repository. As a project, we encourage plugins to be kept in the main repository for several reasons (a more cohesive developer community, better code sharing across plugins, and more uniform testing of plugins). However, we must also make sure that any plugins in the main repository remain well maintained. As a result, any plugin that is in the main repo must have at least one active member of the quantum core developer team willing to maintain it. This may be achieved by having an existing core dev agree to maintain a plugin, or by having one of the developers of the plugin become a member of the core developer team." 
> 
> Essentially, the policy is to strongly encourage people to add their plugins to the main repo, but to make sure that they also provide value to the community in return.   If they do not go down this path, we still allow them to "advertise" their plugin on the Quantum wiki page, though it is hosted in an external repo, and the core team does not try to answer questions about it on the mailing list.  
> 
> In terms of current status, the OVS, Cisco, Linux Bridge, NVP, and NEC plugins all have core devs backing them up.  The Metaplugin is maintained by someone who is not a core dev, but is very active in the community in many areas.  The Ryu plugin is actively maintained, though the maintainers contributions are mostly focused on Ryu changes, and often has trouble getting core reviews.  
> 
> So to date, I think we've been reasonable successfully with the current policy.  I'm writing this email more because I want to get ahead of potential issues that will pop-up in grizzly, giving the exploding interest in Quantum core plugins as well as plugins/drivers for the LBaaS side of things.  For example, there are several plugins already proposed (e.g., Brocade plugin, Ruijie General Operation System, Juniper, bigswitch/floodlight, hyper-v, etc) as well as many LB vendors (riverbed, radware, F5, vmware, etc.) looking to integrate.  
> 
> I think the existing policy will result in us turning away some such contributions, which is sub-optimal.  However, changing the policy to allow all plugins might swamp core developers, prevent them from focusing on improving the platform as a while (and ultimately making it much less attractive to be a core developer, which is VERY bad).  
> 
> If we're not happy with the current state of things, one option is to keep the existing policy, but also establish a 'quantum-ecosystem' repo, that let's people contribute their code to openstack quantum, though without having to go through the standard review process (instead, we'd probably have a simple model where someone is the maintainer for a particular plugin/driver and can solicit reviews from everyone, but is still able to commit regardless).  This let's vendors say that they have contributed the code to quantum, without burdening the core team.  This repo could have automated tests that pull in the primary quantum code as a dependency and run the same unit tests.  We'd have to decide whether failures in these unit tests are just warnings, or if they would actually block commits.  Distros would decide whether to include certain plugins/drivers based on demand.  
> 
> A variation on this model could be keep all such code in the main Quantum repo, but enforcing different review criteria based on whether it is scoped to a single plugin or affects the whole community.  This is not all that different from what happens today in practice, in that usually the core maintainer makes sure there was a thorough review from someone familiar with the plugin, then gets a second core review that focuses more on general python coding guidelines.  
> 
> I'm sure there are other ideas on this as well, so feel free to offer your own.  Thanks for reading :)
> 
> Dan
> 
> 
> 
> -- 
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dan Wendlandt 
> Nicira, Inc: www.nicira.com
> twitter: danwendlandt
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121106/9a8211f6/attachment.html>