Open Stack

Top posting - this is a recap of what has been said, and some
clarifications

I realise that I was not very clear at the beginning of this process, so
here is my effort to clarify things, from the ML thread, and the review.

 >   ... does this also include plugins within projects, like storage
 >   backends in cinder and hypervisor drivers in nova?

No - this was not clear enough. This change is aimed at projects that are
points of significant cross project interaction. While, in the future 
there may
come a point where Nova Compute Drivers are developed out of tree (though
I doubt it), that is not happening today. As a result, there is no 
projects in
the list of projects that would need to integrate with Nova.

 >   Could you please clarify: do you advocate for a generic plugin 
interface for
 >   every project, or that each project should expose a plugin 
interface that
 >   allows plugin to behave as in-tree components? Because the latter 
is what
 >   happens with Tempest, and I see the former a bit complicated.

For every project that has cross project interaction - tempest is a good
example.

For these projects, they should allow all projects in tree (like Nova,
Neutron, Cinder etc are today), or they should have a plugin interface
(like they currently do), but all projects *must* use it, and not use
parts of tempest that are not exposed in that interface.

This would mean that tempest would move the nova, neutron, etc tests to
use the plugin interface.

Now, that plugin could be kept in the tempest repo, and still maintained
by the QA team, but should use the same interface as the other plugins
that are not in that repository.

Of course, it is not just tempest - an incomplete list looks like:

* Tempest
* Devstack
* Grende
* Horizon
* OpenStack Client
* OpenStack SDK
* Searchlight
* Heat
* Mistral
* Celiometer
* Rally
* Documentation

And I am sure I have missed some obvious ones. (if you see a project missing
let me know on the review)


 >   I think I disagree here. The root cause is being addressed: 
external tests can
 >   use the Tempest plugin interface, and use the API, which is being 
stabilized.
 >   The fact that the Tempest API is partially unstable is a temporary 
things, due
 >   to the origin of the project and the way the scope was redefined, 
but again
 >   it's temporary.

This seems to be the core of a lot of the disagreement - this is only 
temporary,
it will all be fixed in the future, and it should stay this way.

Unfortunately the discrepancy between projects is not temporary. The 
specific
problems I have highlighted in the thread for one of the projects is 
temporary,
but I beleive the only long-term solution is to remove the difference 
between
projects.

 >   Before we start making lots of specific rules about how teams
 >   coordinate, I would like to understand the problem those rules are 
meant
 >   to solve, so thank you for providing that example.
 >   ...
 >   It's not clear yet whether there needs to be a new policy to change the
 >   existing intent, or if a discussion just hasn't happened, or if someone
 >   simply needs to edit some code.

Unfortunately there is a big push back on editing code to help plugins from
some of the projects. Again, having the differing access between 
projects will
continue to exacerbate the problem.


 >   "Change the name of the resolution"

That was done in the last patchset. I think the Level Playing Field title
bounced around my head from the other resolution that was titled Level 
Playing
Field. It may have been confusing alright.

I feel like I have been using tempest as an example a little too much, 
as it captures
the current issues perfectly, and a large number of the community have some
knowledge of it, and how it works.

There is other areas across OpenStack where plugins need attention as well:

Horizon
-------

Horizon privileged projects have access to much more panels than
plugins (service status, quotas, overviews etc).
Plugins have to rely on tarballs of horizon

OpenStack Client
----------------

OpenStack CLI privileged projects have access to more commands, as
plugins cannot hook in to them (e.g. quotas)

Grenade
-------

Plugins may or may not have tempest tests ran (I think that patch
merged), they have to use parts of tempest I was told explicitly
plugins should not use to get the tests to run at that point.

Docs
----

We can now add install guides and hook into the API Reference, and API
guides. This is great - and I am really happy about it. We still have
issues trying to integrate with other areas in docs, and most non docs
privileged projects end up with massive amounts of users docs in
docs.openstack.org/developer/<project> , which is not ideal.


Thanks,

Graham

On 14/07/2016 20:25, Hayes, Graham wrote:
> I just proposed a review to openstack/governance repo [0] that aims
> to have everything across OpenStack be plugin based for all cross
> project interaction, or allow all projects access to the same internal
> APIs and I wanted to give a bit of background on my motivation, and how
> it came about.
>
> Coming from a smaller project, I can see issues for new projects,
> smaller projects, and projects that may not be seen as "important".
>
> As a smaller project trying to fit into cross project initiatives,
> (and yes, make sure our software looks at least OK in the
> Project Navigator) the process can be difficult.
>
> A lot of projects / repositories have plugin interfaces, but also
> have project integrations in tree, that do not follow the plugin
> interface. This makes it difficult to see what a plugin can, and
> should do.
>
> When we moved to the big tent, we wanted as a community to move to
> a flatter model, removing the old integrated status.
>
> Unfortunately we still have areas when some projects are more equal -
> there is a lingering set of projects who were integrated at the point
> in time that we moved, and have preferential status.
>
> A lot of the effects are hard to see, and are not insurmountable, but
> do cause projects to re-invent the wheel.
>
> For example, quotas - there is no way for a project that is not nova,
> neutron, cinder to hook into the standard CLI, or UI for setting
> quotas. They can be done as either extra commands
> (openstack dns quota set --foo bar) or as custom panels, but not
> the way other quotas get set.
>
> Tempest plugins are another example. Approximately 30 of the 36
> current plugins are using resources that are not supposed to be
> used, and are an unstable interface. Projects in tree in tempest
> are at a much better position, as any change to the internal
> API will have to be fixed before the gate merges, but other
> out of tree plugins are in a place where they can be broken at any
> point.
>
> None of this is meant to single out projects, or teams. A lot
> of the projects that are in this situation have inordinate amounts of
> work placed on them by the big-tent, and I can emphasize with why things
> are this way. These were the examples that currently stick out
> in my mind, and I think we have come to a point where we need to make
> a change as a community.
>
> By moving to a "plugins for all" model, these issues are reduced.
> It undoubtedly will cause more, but it is closer to our goal
> of Recognizing all our community is part of OpenStack, and
> differentiate projects by tags.
>
> This won't be a change that happens tomorrow, next week, or even next
> cycle, but think as a goal, we should start moving in this direction
> as soon as we can, and start building momentum.
>
> Thanks,
>
> Graham
>
> 0 - https://review.openstack.org/342366
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>