Open Stack

On 15/07/2016 18:24, Luigi Toscano wrote:
> On Friday, 15 July 2016 17:05:41 CEST Hayes, Graham wrote:
>> On 15/07/2016 17:52, Luigi Toscano wrote:
>>> On Friday, 15 July 2016 16:42:22 CEST Hayes, Graham wrote:
>>>> On 15/07/2016 17:10, Andrew Laski wrote:
>>>>>> Tempest plugins are another example. Approximately 30 of the 36
>>>>>> current plugins are using resources that are not supposed to be
>>>>>> used, and are an unstable interface. Projects in tree in tempest
>>>>>> are at a much better position, as any change to the internal
>>>>>> API will have to be fixed before the gate merges, but other
>>>>>> out of tree plugins are in a place where they can be broken at any
>>>>>> point.
>>>>>
>>>>> Has there been an attempt to elevate these internal interfaces into
>>>>> stable and publicly consumable interfaces? Was there resistance to such
>>>>> an effort?
>>>>
>>>> When we have asked previously, we have been told that certain parts
>>>> of tempest "are not really meant for plugins".
>>>>
>>>> The main part that is used that is not part of the tempest stable
>>>> interface is the base test class.
>>>>
>>>> This is the bit that sets up credentials, clients, and other useful
>>>> things.
>>>>
>>>> There is a base test class in the tempest lib - but it is very sparse -
>>>> meaning any project using it would have to re-invent creating users,
>>>> resources, and clients.
>>>>
>>>> https://github.com/openstack/tempest/blob/master/tempest/test.py#L203
>>>> vs
>>>> https://github.com/openstack/tempest/blob/master/tempest/lib/base.py#L22
>>>
>>> This is a known situation, but it is being addressed right now. It's not
>>> like no one wants to have a stable Tempest interface, but it had to be
>>> cleanly built.
>>> There is a spec and work in progress to make the client manager interface
>>> stable:
>>>
>>> http://specs.openstack.org/openstack/qa-specs/specs/tempest/client-manager
>>> -refactor.html
>>>
>>> So yes, almost existing plugins are using unstable interfaces right now,
>>> but again this is not meant to be the long term scenario.
>>>
>>> Ciao
>>
>> Yeap - I have seen the spec.
>>
>> My point is that if all projects had to use the same plugin interface,
>> this would not be a problem.
>
> Could you please clarify: do you advocate for a generic plugin interface for
> every project, or that each project should expose a plugin interface that
> allows plugin to behave as in-tree components? Because the latter is what
> happens with Tempest, and I see the former a bit complicated.

For every project that has cross project interaction - tempest is a good 
example.

For these projects, they should allow all projects in tree (like Nova, 
Neutron, Cinder etc are today), or they should have a plugin interface 
(like they currently do), but all projects *must* use it, and not use
parts of tempest that are not exposed in that interface.

This would mean that tempest would move the nova, neutron, etc tests to
use the plugin interface.

Now, that plugin could be kept in the tempest repo, and still maintained
by the QA team, but should use the same interface as the other plugins
that are not in that repository.

The point is that we were supposed to be a level field as a community
but if we have examples like this, there is not a level playing field.

>>
>> If we have this as our default position as the community continues to
>> build more and more things like tempest, OSC, devstack, grenade we
>> should build it so there is not a discrepancy between projects.
>>
>> The root cause is not being addressed, as features can still land in
>> tempest, but not tempest.lib, and then can only be used by the projects
>> that tempest keeps as built in.
>
>
> I think I disagree here. The root cause is being addressed: external tests can
> use the Tempest plugin interface, and use the API, which is being stabilized.
> The fact that the Tempest API is partially unstable is a temporary things, due
> to the origin of the project and the way the scope was redefined, but again
> it's temporary.
>

But the situation could very easily re-occur. If there is a new feature
that is added to tempest, but not tempest.lib plugins will have a choice
of re implementing it, or relying on an unstable interface.