<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_extra"><div class="gmail_quote">On Sat, Mar 11, 2017 at 12:10 AM, Andrea Frittoli <span dir="ltr"><<a href="mailto:andrea.frittoli@gmail.com" target="_blank">andrea.frittoli@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, Mar 10, 2017 at 2:49 PM Andrea Frittoli <<a href="mailto:andrea.frittoli@gmail.com" target="_blank">andrea.frittoli@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg"><div class="gmail_quote gmail-m_-4561560613520678628gmail_msg"><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg">On Fri, Mar 10, 2017 at 2:24 PM Doug Hellmann <<a href="mailto:doug@doughellmann.com" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">doug@doughellmann.com</a>> wrote:<br class="gmail-m_-4561560613520678628gmail_msg"></div><blockquote class="gmail_quote gmail-m_-4561560613520678628gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Excerpts from Ghanshyam Mann's message of 2017-03-10 10:55:25 +0900:<br class="gmail-m_-4561560613520678628gmail_msg">
> On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad <<a href="mailto:lbragstad@gmail.com" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">lbragstad@gmail.com</a>> wrote:<br class="gmail-m_-4561560613520678628gmail_msg">
><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> > On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann <<a href="mailto:doug@doughellmann.com" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">doug@doughellmann.com</a>><br class="gmail-m_-4561560613520678628gmail_msg">
> > wrote:<br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +0000:<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > Hi folks,<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > I'm trying to figure out what's the best approach to fade out testing of<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > deprecated API versions.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > We currently host in Tempest API tests for Glance API v1, Keystone API<br class="gmail-m_-4561560613520678628gmail_msg">
> >> v2<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > and Cinder API v1.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > According to the guidelines for the "follow-standard-deprecation" tag<br class="gmail-m_-4561560613520678628gmail_msg">
> >> [0],<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > when projects that have that tag deprecate a feature:<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > "Code will be frozen and only receive minimal maintenance (just so that<br class="gmail-m_-4561560613520678628gmail_msg">
> >> it<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > continues to work as-is)."<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > I interpret this so that projects should maintain some level of testing<br class="gmail-m_-4561560613520678628gmail_msg">
> >> of<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > the deprecated feature, including a deprecated API version.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > The QA team does not see value in testing deprecated API versions in the<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > common gate jobs, so my question is what do to with those tests.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > One option is to maintain them in Tempest until the API version is<br class="gmail-m_-4561560613520678628gmail_msg">
> >> removed,<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > and run them in dedicated project jobs.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > This means that tempest would have to run those jobs as well, so three<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > extra jobs, until the API version is removed.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > The other option is to move those tests out of Tempest, into the<br class="gmail-m_-4561560613520678628gmail_msg">
> >> projects.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > This would imply back porting them to all relevant branches as well,<br class="gmail-m_-4561560613520678628gmail_msg">
> >> but it<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > would have the advantage of decoupling them from Tempest. It should be<br class="gmail-m_-4561560613520678628gmail_msg">
> >> no<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > concern from an API stability POV since the code for that API will be<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > frozen.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > Tests for deprecated APIs in cinder, keystone and glance are all - as<br class="gmail-m_-4561560613520678628gmail_msg">
> >> far<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > as I can tell - removed or deprecated from interoperability guidelines,<br class="gmail-m_-4561560613520678628gmail_msg">
> >> so<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > moving the tests out of Tempest would not be an issue in that sense.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > The 2nd option involves a bit more initial overhead for the removal of<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > tests, but I think it would works for the best on the long term.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > There is a 3rd option as well, which is to stop running integration<br class="gmail-m_-4561560613520678628gmail_msg">
> >> testing<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > on deprecated API versions before they are actually removed, but I feel<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > that would not meet the criteria defined by the<br class="gmail-m_-4561560613520678628gmail_msg">
> >> follow-standard-deprecation<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > tag.<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > Thoughts?<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > andrea<br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> >> Are any of those tests used by the interoperability working group<br class="gmail-m_-4561560613520678628gmail_msg">
> >> (formerly DefCore)?<br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> > That's a good question. I was very curious about this because last I<br class="gmail-m_-4561560613520678628gmail_msg">
> > checked keystone had v2.0 calls required for defcore. Looks like that might<br class="gmail-m_-4561560613520678628gmail_msg">
> > not be the case anymore [0]? I started a similar thread to this after the<br class="gmail-m_-4561560613520678628gmail_msg">
> > PTG since that was something our group talked about extensively during the<br class="gmail-m_-4561560613520678628gmail_msg">
> > deprecation session [1].<br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> > Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore<br class="gmail-m_-4561560613520678628gmail_msg">
> 2017.01.json [0]. But there are some compute tests which internally use<br class="gmail-m_-4561560613520678628gmail_msg">
> glance v1 API call [2]. But on mentioned flagged action- Nova already moved<br class="gmail-m_-4561560613520678628gmail_msg">
> to v2 APIs and tempest part is pending which can be fixed to make call on<br class="gmail-m_-4561560613520678628gmail_msg">
> v2 APIs only (which can be part of this work and quick).<br class="gmail-m_-4561560613520678628gmail_msg">
><br class="gmail-m_-4561560613520678628gmail_msg">
> From options about deprecated APIs testing, I am with options 2 which<br class="gmail-m_-4561560613520678628gmail_msg">
> really take out the load of Tempest tests maintenance and gate. <br class="gmail-m_-4561560613520678628gmail_msg">
><br class="gmail-m_-4561560613520678628gmail_msg">
> But another question is about stable branch testing of those API, like<br class="gmail-m_-4561560613520678628gmail_msg">
> glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka.<br class="gmail-m_-4561560613520678628gmail_msg">
> As Tempest is responsible of testing all stable branch behavior too, Should<br class="gmail-m_-4561560613520678628gmail_msg">
> we keep testing them till all Mitaka EOL (till APIs are in deprecated<br class="gmail-m_-4561560613520678628gmail_msg">
> state in all stable branch) ?<br class="gmail-m_-4561560613520678628gmail_msg">
<br class="gmail-m_-4561560613520678628gmail_msg">
Excellent point.<br class="gmail-m_-4561560613520678628gmail_msg">
<br class="gmail-m_-4561560613520678628gmail_msg"></blockquote><div class="gmail-m_-4561560613520678628gmail_msg"><br class="gmail-m_-4561560613520678628gmail_msg"></div></div></div><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg"><div class="gmail_quote gmail-m_-4561560613520678628gmail_msg"><div class="gmail-m_-4561560613520678628gmail_msg">As far as I can tell:</div><div class="gmail-m_-4561560613520678628gmail_msg">- Cinder v1 if I'm not mistaken has been deprecated in Juno, so it's deprecated in all supported releases.<br class="gmail-m_-4561560613520678628gmail_msg"></div><div class="gmail-m_-4561560613520678628gmail_msg">- Glance v1 has been deprecated in Newton, so it's deprecated in all supported releases</div></div></div></blockquote><div><br></div><div>Of course Glance v1 still has to run on the stable/newton gate jobs, until Newton EOL (TBD), so tests will stay in Tempest for a cycle more at least.<br></div><div>I guess I shouldn't be sending emails on a Friday afternoon? <br></div></div></div></blockquote><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">humm, Till Mitaka right? Newton version of glance is with v1 API as deprecated. And Mitaka is v1 as supported APIs so we only need to care about Mitaka where v1 APIs were supported and keep tests till Mitaka EOL.</div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><div><br></div><div>The question remains for Cinder and Keystone and in general - how we want to deal with integration tests for deprecated APIs.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg"><div class="gmail_quote gmail-m_-4561560613520678628gmail_msg"><div class="gmail-m_-4561560613520678628gmail_msg">- Keystone v2 has been deprecated in Mitaka, so testing *must* stay in Tempest until Mitaka EOL, which is in a month from now</div><div class="gmail-m_-4561560613520678628gmail_msg"> </div><div class="gmail-m_-4561560613520678628gmail_msg">We should stop testing these three api versions in the common gate including stable branches now (except for keystone v2 on stable/mitaka which can run for one more month).</div></div></div></blockquote></div></div></blockquote><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">But I am little bit confused about version deprecation as version discovery shows v2 as stable in Mitaka [0]. Which does not match with release notes, so is v2 deprecated in Mitaka or Newton ?<br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">If it is officially marked as deprecated in Mitaka (with version discovery bug) then, tests can go to project side without any wait.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">Tempest will not leave v2 APIs untested where v2 is stable API. </div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg"><div class="gmail_quote gmail-m_-4561560613520678628gmail_msg"><div class="gmail-m_-4561560613520678628gmail_msg"><br class="gmail-m_-4561560613520678628gmail_msg"></div><div class="gmail-m_-4561560613520678628gmail_msg">Are cinder / glance / keystone willing to take over the API tests and run them in their own gate until removal of the API version?</div></div></div><div dir="ltr" class="gmail-m_-4561560613520678628gmail_msg"><div class="gmail_quote gmail-m_-4561560613520678628gmail_msg"><div class="gmail-m_-4561560613520678628gmail_msg"><br class="gmail-m_-4561560613520678628gmail_msg"></div><blockquote class="gmail_quote gmail-m_-4561560613520678628gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Doug<br class="gmail-m_-4561560613520678628gmail_msg">
<br class="gmail-m_-4561560613520678628gmail_msg">
><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> > [0] <a href="https://github.com/openstack/defcore/blob/master/2017.01.json" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">https://github.com/openstack/<wbr>defcore/blob/master/2017.01.<wbr>json</a><br class="gmail-m_-4561560613520678628gmail_msg">
> > [1] <a href="http://lists.openstack.org/pipermail/openstack-dev/" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">http://lists.openstack.org/<wbr>pipermail/openstack-dev/</a><br class="gmail-m_-4561560613520678628gmail_msg">
> > 2017-March/113166.html<br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> > [2]<br class="gmail-m_-4561560613520678628gmail_msg">
> <a href="https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">https://git.openstack.org/<wbr>cgit/openstack/defcore/tree/<wbr>2017.01.json#n294</a> <br class="gmail-m_-4561560613520678628gmail_msg">
><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> Doug<br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> >> ><br class="gmail-m_-4561560613520678628gmail_msg">
> >> > [0]<br class="gmail-m_-4561560613520678628gmail_msg">
> >> > <a href="https://governance.openstack.org/tc/reference/tags/assert_fo" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">https://governance.openstack.<wbr>org/tc/reference/tags/assert_<wbr>fo</a><br class="gmail-m_-4561560613520678628gmail_msg">
> >> llows-standard-deprecation.<wbr>html<br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> >> ______________________________<wbr>______________________________<br class="gmail-m_-4561560613520678628gmail_msg">
> >> ______________<br class="gmail-m_-4561560613520678628gmail_msg">
> >> OpenStack Development Mailing List (not for usage questions)<br class="gmail-m_-4561560613520678628gmail_msg">
> >> Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscrib" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscrib</a><br class="gmail-m_-4561560613520678628gmail_msg">
> >> e<br class="gmail-m_-4561560613520678628gmail_msg">
> >> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br class="gmail-m_-4561560613520678628gmail_msg">
> >><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> > ______________________________<wbr>______________________________<wbr>______________<br class="gmail-m_-4561560613520678628gmail_msg">
> > OpenStack Development Mailing List (not for usage questions)<br class="gmail-m_-4561560613520678628gmail_msg">
> > Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br class="gmail-m_-4561560613520678628gmail_msg">
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
> ><br class="gmail-m_-4561560613520678628gmail_msg">
<br class="gmail-m_-4561560613520678628gmail_msg">
______________________________<wbr>______________________________<wbr>______________<br class="gmail-m_-4561560613520678628gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail-m_-4561560613520678628gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br class="gmail-m_-4561560613520678628gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail-m_-4561560613520678628gmail_msg" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br class="gmail-m_-4561560613520678628gmail_msg">
</blockquote></div></div></blockquote></div></div>
<br></blockquote><div><br></div><div><br></div><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">..0 <a href="https://git.openstack.org/cgit/openstack/keystone/tree/keystone/version/controllers.py?h=stable/mitaka#n124">https://git.openstack.org/cgit/openstack/keystone/tree/keystone/version/controllers.py?h=stable/mitaka#n124</a></div><br></div><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">Thanks</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">gmann</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br></blockquote></div><br></div></div>