Open Stack

On 03/01/2017 07:35 AM, Jordan Pittier wrote:
> 
> 
> On Wed, Mar 1, 2017 at 3:57 AM, Ghanshyam Mann <ghanshyammann at gmail.com
> <mailto:ghanshyammann at gmail.com>> wrote:
> 
>     Doing gradual refactoring and fixing plugins time to time needs lot
>     of wait and sync.
> 
>     That needs:
>     1. Plugins to switch from current method usage. Plugins to have some
>     other function or same copy paste code what current scenario base
>     class has.
>     2. Tempest patch to wait for plugin fix.
>     3. Plugins to switch back to stable interface once Tempest going to
>     provide those. 
> 
>     This needs lot of sync between tempest and plugins and we have to
>     wait for tempest refactoring patch for long. 
> 
>     To make it more efficient, how about this:
>     1. Keep the scenario manger copy in tempest as it is. for plugins
>     usage only.
> 
> Given that the refactoring effort "started" a year ago, at the current
> speed it may take 2 years to complete. In the mean time we will have a
> massive code duplication and a maintenance burden. 
> 
>     2. Start refactoring the scenario framework by adding more and more
>     helper function under /common or lib.
> 
> Starting a "framework" (each time I see that word, I have a bad feeling)
> from scratch without users and usage is very very difficult. How do we
> know what we need in that framework and what will be actually used in
> tests ?  

I'm with Jordan, honestly, common functions and inheritance aren't going
to help anyway. My feeling is that the only way to sanely take things
forward is to start pivoting towards fixtures. And basically error if
inheritance is used in tests. The functions on base classes is what is
causing all the pain of rewind, because it's not a contract, but it's
also magically there, so people try to use it.

This would mean that every test would have more setUp with a bunch of
explicit fixtures, but that's fine, it's a little more time thinking
through the writing of the tests, and a lot easier to read them to know
what is going on, and to delete them later. Basically get rid of all
class/superclass magic.

> The effort was called scenario refactoring and I think that's what we
> should do. We should not do "start from scratch scenarios" or "copy all
> the code and see what happens".
> 
> There's no problem with plugins. We committed to have a stable interface
> which is documented and agreed upon. It's clearly written
> here https://docs.openstack.org/developer/tempest/plugin.html#stable-tempest-apis-plugins-may-use
> The rest is private, for Tempest internal use. If Tempest cores disagree
> with that, then we should first of all put a spec and rewrite that
> document. 

It does seem like there is a tough love moment coming about the fact
that APIs were consumed that were specifically out of bounds. Either the
team commits to them inbounds, and there is the contract, or moves
forward under the existing contract.

Though, I expect the issue is that because all the Tempest tests work in
this super class way, this is the only stuff people understood and copy
pasted. So it does feel like there has to be at least a couple of
instances in the Tempest tree of how the team *wants* other projects to
write tests that they can point people to.

	-Sean

-- 
Sean Dague
http://dague.net