Open Stack

On Fri, Mar 10, 2017 at 3:12 PM Lance Bragstad <lbragstad at gmail.com> wrote:

> On Fri, Mar 10, 2017 at 8:49 AM, Andrea Frittoli <
> andrea.frittoli at gmail.com> wrote:
>
>
>
> On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann <doug at doughellmann.com>
> wrote:
>
> Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:
> > On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad <lbragstad at gmail.com>
> wrote:
> >
> > >
> > >
> > > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann <doug at doughellmann.com>
> > > wrote:
> > >
> > >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +0000:
> > >> > Hi folks,
> > >> >
> > >> > I'm trying to figure out what's the best approach to fade out
> testing of
> > >> > deprecated API versions.
> > >> > We currently host in Tempest API tests for Glance API v1, Keystone
> API
> > >> v2
> > >> > and Cinder API v1.
> > >> >
> > >> > According to the guidelines for the "follow-standard-deprecation"
> tag
> > >> [0],
> > >> > when projects that have that tag deprecate a feature:
> > >> >
> > >> > "Code will be frozen and only receive minimal maintenance (just so
> that
> > >> it
> > >> > continues to work as-is)."
> > >> >
> > >> > I interpret this so that projects should maintain some level of
> testing
> > >> of
> > >> > the deprecated feature, including a deprecated API version.
> > >> > The QA team does not see value in testing deprecated API versions
> in the
> > >> > common gate jobs, so my question is what do to with those tests.
> > >> >
> > >> > One option is to maintain them in Tempest until the API version is
> > >> removed,
> > >> > and run them in dedicated project jobs.
> > >> > This means that tempest would have to run those jobs as well, so
> three
> > >> > extra jobs, until the API version is removed.
> > >> >
> > >> > The other option is to move those tests out of Tempest, into the
> > >> projects.
> > >> > This would imply back porting them to all relevant branches as well,
> > >> but it
> > >> > would have the advantage of decoupling them from Tempest. It should
> be
> > >> no
> > >> > concern from an API stability POV since the code for that API will
> be
> > >> > frozen.
> > >> > Tests for deprecated APIs in cinder, keystone and glance are all -
> as
> > >> far
> > >> > as I can tell - removed or deprecated from interoperability
> guidelines,
> > >> so
> > >> > moving the tests out of Tempest would not be an issue in that sense.
> > >> >
> > >> > The 2nd option involves a bit more initial overhead for the removal
> of
> > >> > tests, but I think it would works for the best on the long term.
> > >> >
> > >> > There is a 3rd option as well, which is to stop running integration
> > >> testing
> > >> > on deprecated API versions before they are actually removed, but I
> feel
> > >> > that would not meet the criteria defined by the
> > >> follow-standard-deprecation
> > >> > tag.
> > >> >
> > >> > Thoughts?
> > >> >
> > >> > andrea
> > >>
> > >> Are any of those tests used by the interoperability working group
> > >> (formerly DefCore)?
> > >>
> > >>
> > > That's a good question. I was very curious about this because last I
> > > checked keystone had v2.0 calls required for defcore. Looks like that
> might
> > > not be the case anymore [0]? I started a similar thread to this after
> the
> > > PTG since that was something our group talked about extensively during
> the
> > > deprecation session [1].
> > >
> > > ​Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore
> > 2017.01.json [0]​. But there are some compute tests which internally use
> > glance v1 API call [2]. But on mentioned flagged action- Nova already
> moved
> > to v2 APIs and tempest part is pending which can be fixed to make call on
> > v2 APIs only (which can be part of this work and quick).
> >
> > ​From options about deprecated APIs testing, I am with options 2 which
> > really take out the load of Tempest tests maintenance and gate.   ​
> >
> > ​But another question is about stable branch testing of those API, like
> > glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.
> > ​As Tempest is responsible of testing all stable branch behavior too,
> Should
> >  we keep testing them till all Mitaka EOL (till APIs are in deprecated
> > state in all stable branch) ?​
>
> Excellent point.
>
>
> As far as I can tell:
> - Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's
> deprecated in all supported releases.
> - Glance v1 has been deprecated in Newton, so it's deprecated in all
> supported releases
> - Keystone v2 has been deprecated in Mitaka, so testing *must* stay in
> Tempest until Mitaka EOL, which is in a month from now
>
> We should stop testing these three api versions in the common gate
> including stable branches now (except for keystone v2 on stable/mitaka
> which can run for one more month).
>
> Are cinder / glance / keystone willing to take over the API tests and run
> them in their own gate until removal of the API version?
>
>
> Would this process consist of porting those tests directly to the project
> and hooking them in like we do for tempest/devstack plugins?
>
>
Yes, that's what I had in mind.

We are going to make test.py a stable interface soon-ish, which will mean
identity v2 tests only depend on tempest stable interfaces - i.e. no gate
breakages.


>
> Doug
>
> >
> > >
> > > [0] https://github.com/openstack/defcore/blob/master/2017.01.json
> > > [1] http://lists.openstack.org/pipermail/openstack-dev/
> > > 2017-March/113166.html
> > >
> > > ​[2]
> > https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294
> ​
> >
> > >
> > >
> > >> Doug
> > >>
> > >> >
> > >> > [0]
> > >> > https://governance.openstack.org/tc/reference/tags/assert_fo
> > >> llows-standard-deprecation.html
> > >>
> > >> ____________________________________________________________
> > >> ______________
> > >> OpenStack Development Mailing List (not for usage questions)
> > >> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
> > >> e
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >>
> > >
> > >
> > >
> __________________________________________________________________________
> > > OpenStack Development Mailing List (not for usage questions)
> > > Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >
> > >
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170310/237bb31d/attachment.html>