Open Stack

On Wed, Mar 1, 2017 at 12:54 PM Sean Dague <sean at dague.net> wrote:

> On 03/01/2017 07:35 AM, Jordan Pittier wrote:
> >
> >
> > On Wed, Mar 1, 2017 at 3:57 AM, Ghanshyam Mann <ghanshyammann at gmail.com
> > <mailto:ghanshyammann at gmail.com>> wrote:
> >
> >     Doing gradual refactoring and fixing plugins time to time needs lot
> >     of wait and sync.
> >
> >     That needs:
> >     1. Plugins to switch from current method usage. Plugins to have some
> >     other function or same copy paste code what current scenario base
> >     class has.
> >     2. Tempest patch to wait for plugin fix.
> >     3. Plugins to switch back to stable interface once Tempest going to
> >     provide those.
> >
> >     This needs lot of sync between tempest and plugins and we have to
> >     wait for tempest refactoring patch for long.
> >
> >     To make it more efficient, how about this:
> >     1. Keep the scenario manger copy in tempest as it is. for plugins
> >     usage only.
> >
> > Given that the refactoring effort "started" a year ago, at the current
> > speed it may take 2 years to complete. In the mean time we will have a
> > massive code duplication and a maintenance burden.
> >
> >     2. Start refactoring the scenario framework by adding more and more
> >     helper function under /common or lib.
> >
> > Starting a "framework" (each time I see that word, I have a bad feeling)
> > from scratch without users and usage is very very difficult. How do we
> > know what we need in that framework and what will be actually used in
> > tests ?
>
> I'm with Jordan, honestly, common functions and inheritance aren't going
> to help anyway. My feeling is that the only way to sanely take things
> forward is to start pivoting towards fixtures. And basically error if
> inheritance is used in tests. The functions on base classes is what is
> causing all the pain of rewind, because it's not a contract, but it's
> also magically there, so people try to use it.
>
> This would mean that every test would have more setUp with a bunch of
> explicit fixtures, but that's fine, it's a little more time thinking
> through the writing of the tests, and a lot easier to read them to know
> what is going on, and to delete them later. Basically get rid of all
> class/superclass magic.
>
> > The effort was called scenario refactoring and I think that's what we
> > should do. We should not do "start from scratch scenarios" or "copy all
> > the code and see what happens".
> >
> > There's no problem with plugins. We committed to have a stable interface
> > which is documented and agreed upon. It's clearly written
> > here
> https://docs.openstack.org/developer/tempest/plugin.html#stable-tempest-apis-plugins-may-use
> > The rest is private, for Tempest internal use. If Tempest cores disagree
> > with that, then we should first of all put a spec and rewrite that
> > document.
>
> It does seem like there is a tough love moment coming about the fact
> that APIs were consumed that were specifically out of bounds. Either the
> team commits to them inbounds, and there is the contract, or moves
> forward under the existing contract.
>
> Though, I expect the issue is that because all the Tempest tests work in
> this super class way, this is the only stuff people understood and copy
> pasted. So it does feel like there has to be at least a couple of
> instances in the Tempest tree of how the team *wants* other projects to
> write tests that they can point people to.
>

+1 on this. Which is why we need to refactor our scenario test to a sane
code
base that people can use as examples to copy paste.

The basic credentials provider services that comes with test.py is something
that is useful to have I think - and we are working on make them a contract
people
can rely on.

We had a discussion at the PTG about how to deal with those, and the
conclusion
was that we are going to move credentials providers to tempest.lib and keep
test.py more or less as it is and declare it a stable interface for plugins.

Alternatives like dropping class fixtures for good, and move everything to
be test
fixtures, involve too much code churn / refactor for an advantage which
is too little - and we wouldn't have the bandwidth to do that anyways.


>
>         -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170301/2a75696c/attachment.html>