[openstack-dev] [all][tc] Equal Chances for all projects (was Plugins for all)

Doug Hellmann doug at doughellmann.com
Tue Jul 26 12:51:17 UTC 2016


Excerpts from Hayes, Graham's message of 2016-07-25 21:36:29 +0000:
> Top posting - this is a recap of what has been said, and some
> clarifications
> 
> I realise that I was not very clear at the beginning of this process, so
> here is my effort to clarify things, from the ML thread, and the review.
> 
>  >   ... does this also include plugins within projects, like storage
>  >   backends in cinder and hypervisor drivers in nova?
> 
> No - this was not clear enough. This change is aimed at projects that are
> points of significant cross project interaction. While, in the future 
> there may
> come a point where Nova Compute Drivers are developed out of tree (though
> I doubt it), that is not happening today. As a result, there is no 
> projects in
> the list of projects that would need to integrate with Nova.
> 
>  >   Could you please clarify: do you advocate for a generic plugin 
> interface for
>  >   every project, or that each project should expose a plugin 
> interface that
>  >   allows plugin to behave as in-tree components? Because the latter 
> is what
>  >   happens with Tempest, and I see the former a bit complicated.
> 
> For every project that has cross project interaction - tempest is a good
> example.
> 
> For these projects, they should allow all projects in tree (like Nova,
> Neutron, Cinder etc are today), or they should have a plugin interface
> (like they currently do), but all projects *must* use it, and not use
> parts of tempest that are not exposed in that interface.
> 
> This would mean that tempest would move the nova, neutron, etc tests to
> use the plugin interface.
> 
> Now, that plugin could be kept in the tempest repo, and still maintained
> by the QA team, but should use the same interface as the other plugins
> that are not in that repository.
> 
> Of course, it is not just tempest - an incomplete list looks like:
> 
> * Tempest
> * Devstack
> * Grende
> * Horizon
> * OpenStack Client
> * OpenStack SDK
> * Searchlight
> * Heat
> * Mistral
> * Celiometer
> * Rally
> * Documentation
> 
> And I am sure I have missed some obvious ones. (if you see a project missing
> let me know on the review)
> 
>  >   I think I disagree here. The root cause is being addressed: 
> external tests can
>  >   use the Tempest plugin interface, and use the API, which is being 
> stabilized.
>  >   The fact that the Tempest API is partially unstable is a temporary 
> things, due
>  >   to the origin of the project and the way the scope was redefined, 
> but again
>  >   it's temporary.
> 
> This seems to be the core of a lot of the disagreement - this is only 
> temporary,
> it will all be fixed in the future, and it should stay this way.
> 
> Unfortunately the discrepancy between projects is not temporary. The 
> specific
> problems I have highlighted in the thread for one of the projects is 
> temporary,
> but I beleive the only long-term solution is to remove the difference 
> between
> projects.
> 
>  >   Before we start making lots of specific rules about how teams
>  >   coordinate, I would like to understand the problem those rules are 
> meant
>  >   to solve, so thank you for providing that example.
>  >   ...
>  >   It's not clear yet whether there needs to be a new policy to change the
>  >   existing intent, or if a discussion just hasn't happened, or if someone
>  >   simply needs to edit some code.
> 
> Unfortunately there is a big push back on editing code to help plugins from
> some of the projects. Again, having the differing access between 
> projects will
> continue to exacerbate the problem.
> 
>  >   "Change the name of the resolution"
> 
> That was done in the last patchset. I think the Level Playing Field title
> bounced around my head from the other resolution that was titled Level 
> Playing
> Field. It may have been confusing alright.
> 
> I feel like I have been using tempest as an example a little too much, 
> as it captures
> the current issues perfectly, and a large number of the community have some
> knowledge of it, and how it works.
> 
> There is other areas across OpenStack where plugins need attention as well:
> 
> Horizon
> -------
> 
> Horizon privileged projects have access to much more panels than
> plugins (service status, quotas, overviews etc).

Is this by intent, lack of foresight, or changing requirements?

> Plugins have to rely on tarballs of horizon

It sounds like we need to figure out how to tag intermediate releases of
horizon and publish them to PyPI so projects can use them in tests. I'd
be happy to work with the Horizon team on that.

> 
> OpenStack Client
> ----------------
> 
> OpenStack CLI privileged projects have access to more commands, as
> plugins cannot hook in to them (e.g. quotas)

I think Steve addressed this point elsewhere on the thread (it's not by
design, but due to lack of contributor resources).

> 
> Grenade
> -------
> 
> Plugins may or may not have tempest tests ran (I think that patch
> merged), they have to use parts of tempest I was told explicitly
> plugins should not use to get the tests to run at that point.

This was acknowledged as a bug, and as you mention there's already
a fix at least in process if not merged.

> 
> Docs
> ----
> 
> We can now add install guides and hook into the API Reference, and API
> guides. This is great - and I am really happy about it. We still have
> issues trying to integrate with other areas in docs, and most non docs
> privileged projects end up with massive amounts of users docs in
> docs.openstack.org/developer/<project> , which is not ideal.

This is another thing in process. As Lana has pointed out, the doc
team is experimenting with tools for improving publishing support.
I think it makes sense to see how those experiments play out a bit
before we commit to a specific path (either publishing individual
manuals or working on automation to publish integrated manuals or
whatever other options they've come up with).


Given the amount of in-progress work to address the issue you've
raised, I'm not convinced we need a global rule or policy. All of
the teams mentioned are working toward the goal of providing stable
APIs already, and no one seems to be arguing against that goal. The
teams doing the work are not dragging their feet, and a rule or
policy wouldn't make the work go any faster.

The directions for cross-project teams were given when the bit tent
change went into effect were to "support all official teams" and
definitely not "do the work for all official teams." There was also
no requirement that the support be exactly equal, and it would be
a mistake to try to require that because the issue is almost always
lack of resources and not lack of desire.  Volunteers to contribute
to the work that's needed will do more to help than a one-size-fits-all
policy.

Doug

> 
> 
> Thanks,
> 
> Graham
> 
> On 14/07/2016 20:25, Hayes, Graham wrote:
> > I just proposed a review to openstack/governance repo [0] that aims
> > to have everything across OpenStack be plugin based for all cross
> > project interaction, or allow all projects access to the same internal
> > APIs and I wanted to give a bit of background on my motivation, and how
> > it came about.
> >
> > Coming from a smaller project, I can see issues for new projects,
> > smaller projects, and projects that may not be seen as "important".
> >
> > As a smaller project trying to fit into cross project initiatives,
> > (and yes, make sure our software looks at least OK in the
> > Project Navigator) the process can be difficult.
> >
> > A lot of projects / repositories have plugin interfaces, but also
> > have project integrations in tree, that do not follow the plugin
> > interface. This makes it difficult to see what a plugin can, and
> > should do.
> >
> > When we moved to the big tent, we wanted as a community to move to
> > a flatter model, removing the old integrated status.
> >
> > Unfortunately we still have areas when some projects are more equal -
> > there is a lingering set of projects who were integrated at the point
> > in time that we moved, and have preferential status.
> >
> > A lot of the effects are hard to see, and are not insurmountable, but
> > do cause projects to re-invent the wheel.
> >
> > For example, quotas - there is no way for a project that is not nova,
> > neutron, cinder to hook into the standard CLI, or UI for setting
> > quotas. They can be done as either extra commands
> > (openstack dns quota set --foo bar) or as custom panels, but not
> > the way other quotas get set.
> >
> > Tempest plugins are another example. Approximately 30 of the 36
> > current plugins are using resources that are not supposed to be
> > used, and are an unstable interface. Projects in tree in tempest
> > are at a much better position, as any change to the internal
> > API will have to be fixed before the gate merges, but other
> > out of tree plugins are in a place where they can be broken at any
> > point.
> >
> > None of this is meant to single out projects, or teams. A lot
> > of the projects that are in this situation have inordinate amounts of
> > work placed on them by the big-tent, and I can emphasize with why things
> > are this way. These were the examples that currently stick out
> > in my mind, and I think we have come to a point where we need to make
> > a change as a community.
> >
> > By moving to a "plugins for all" model, these issues are reduced.
> > It undoubtedly will cause more, but it is closer to our goal
> > of Recognizing all our community is part of OpenStack, and
> > differentiate projects by tags.
> >
> > This won't be a change that happens tomorrow, next week, or even next
> > cycle, but think as a goal, we should start moving in this direction
> > as soon as we can, and start building momentum.
> >
> > Thanks,
> >
> > Graham
> >
> > 0 - https://review.openstack.org/342366
> >
> > __________________________________________________________________________
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 



More information about the OpenStack-dev mailing list