<div dir="ltr"><div dir="ltr" class="gmail_msg"><div class="gmail_quote gmail_msg"><div dir="ltr" class="gmail_msg">On Fri, Mar 10, 2017 at 1:59 AM Ghanshyam Mann <<a href="mailto:ghanshyammann@gmail.com" class="gmail_msg" target="_blank">ghanshyammann@gmail.com</a>> wrote:<br class="gmail_msg"></div><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_default gmail_msg" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)"><span style="font-family:arial,sans-serif;color:rgb(34,34,34)" class="gmail_msg">On Fri, Mar 10, 2017 at 7:23 AM, Lance Bragstad </span><span dir="ltr" style="font-family:arial,sans-serif;color:rgb(34,34,34)" class="gmail_msg"><<a href="mailto:lbragstad@gmail.com" class="gmail_msg" target="_blank">lbragstad@gmail.com</a>></span><span style="font-family:arial,sans-serif;color:rgb(34,34,34)" class="gmail_msg"> wrote:</span><br class="gmail_msg"></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg"><br class="gmail_msg"><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg">On Thu, Mar 9, 2017 at 3:46 PM, Doug Hellmann <span dir="ltr" class="gmail_msg"><<a href="mailto:doug@doughellmann.com" class="gmail_msg" target="_blank">doug@doughellmann.com</a>></span> wrote:<br class="gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Excerpts from Andrea Frittoli's message of 2017-03-09 20:53:54 +0000:<br class="gmail_msg">
<div class="gmail_msg"><div class="m_-3949551666231403183m_-2799762232257982024gmail-m_5619289470930054799gmail-m_-7127463423234097919gmail-h5 gmail_msg">> Hi folks,<br class="gmail_msg">
><br class="gmail_msg">
> I'm trying to figure out what's the best approach to fade out testing of<br class="gmail_msg">
> deprecated API versions.<br class="gmail_msg">
> We currently host in Tempest API tests for Glance API v1, Keystone API v2<br class="gmail_msg">
> and Cinder API v1.<br class="gmail_msg">
><br class="gmail_msg">
> According to the guidelines for the "follow-standard-deprecation" tag [0],<br class="gmail_msg">
> when projects that have that tag deprecate a feature:<br class="gmail_msg">
><br class="gmail_msg">
> "Code will be frozen and only receive minimal maintenance (just so that it<br class="gmail_msg">
> continues to work as-is)."<br class="gmail_msg">
><br class="gmail_msg">
> I interpret this so that projects should maintain some level of testing of<br class="gmail_msg">
> the deprecated feature, including a deprecated API version.<br class="gmail_msg">
> The QA team does not see value in testing deprecated API versions in the<br class="gmail_msg">
> common gate jobs, so my question is what do to with those tests.<br class="gmail_msg">
><br class="gmail_msg">
> One option is to maintain them in Tempest until the API version is removed,<br class="gmail_msg">
> and run them in dedicated project jobs.<br class="gmail_msg">
> This means that tempest would have to run those jobs as well, so three<br class="gmail_msg">
> extra jobs, until the API version is removed.<br class="gmail_msg">
><br class="gmail_msg">
> The other option is to move those tests out of Tempest, into the projects.<br class="gmail_msg">
> This would imply back porting them to all relevant branches as well, but it<br class="gmail_msg">
> would have the advantage of decoupling them from Tempest. It should be no<br class="gmail_msg">
> concern from an API stability POV since the code for that API will be<br class="gmail_msg">
> frozen.<br class="gmail_msg">
> Tests for deprecated APIs in cinder, keystone and glance are all - as far<br class="gmail_msg">
> as I can tell - removed or deprecated from interoperability guidelines, so<br class="gmail_msg">
> moving the tests out of Tempest would not be an issue in that sense.<br class="gmail_msg">
><br class="gmail_msg">
> The 2nd option involves a bit more initial overhead for the removal of<br class="gmail_msg">
> tests, but I think it would works for the best on the long term.<br class="gmail_msg">
><br class="gmail_msg">
> There is a 3rd option as well, which is to stop running integration testing<br class="gmail_msg">
> on deprecated API versions before they are actually removed, but I feel<br class="gmail_msg">
> that would not meet the criteria defined by the follow-standard-deprecation<br class="gmail_msg">
> tag.<br class="gmail_msg">
><br class="gmail_msg">
> Thoughts?<br class="gmail_msg">
><br class="gmail_msg">
> andrea<br class="gmail_msg">
<br class="gmail_msg">
</div></div>Are any of those tests used by the interoperability working group<br class="gmail_msg">
(formerly DefCore)?<br class="gmail_msg">
<br class="gmail_msg"></blockquote><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">That's a good question. I was very curious about this because last I checked keystone had v2.0 calls required for defcore. Looks like that might not be the case anymore [0]? I started a similar thread to this after the PTG since that was something our group talked about extensively during the deprecation session [1].</div><div class="gmail_msg"><br class="gmail_msg"></div></div></div></div></blockquote></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><div class="gmail_default gmail_msg" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">Yes, it seems no Volume v1 and Keystone v2 tests usage in defcore 2017.01.json [0]. But there are some compute tests which internally use glance v1 API call [2]. But on mentioned flagged action- Nova already moved to v2 APIs and tempest part is pending which can be fixed to make call on v2 APIs only (which can be part of this work and quick).</div></div></div></div></div></blockquote><div class="gmail_msg"><br class="gmail_msg"></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg">I just checked all non-glance tests that invoke glance v1 in the gate right now.</div><div class="gmail_msg">None of them is in the 2017.01 guideline [0], and all of them will run with glance v2 as long as v1 is not marked as enabled.</div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><div>FlavorsV2NegativeTest:test_boot_with_low_ram</div><div>ServerActionsTestJSON:test_create_backup</div><div>TestMinimumBasicScenario:test_minimum_basic_scenario</div><div>TestVolumeBootPattern:test_create_ebs_image_and_check_boot</div><div>VolumesV2ActionsTest:test_volume_upload</div><div><br></div></div><div class="gmail_msg">[0] <a href="https://refstack.openstack.org/api/v1/guidelines/2017.01/tests?target=platform&type=required&alias=true&flag=false">https://refstack.openstack.org/api/v1/guidelines/2017.01/tests?target=platform&type=required&alias=true&flag=false</a></div><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><div class="gmail_default gmail_msg" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">From options about deprecated APIs testing, I am with options 2 which really take out the load of Tempest tests maintenance and gate. </div><br class="gmail_msg"></div><div class="gmail_msg"><font color="#000000" face="arial, helvetica, sans-serif" class="gmail_msg">But another question is about stable branch testing of those API, like glance v1 and identity v2 APIs are supported (not deprecated) in Mitaka. </font><div class="gmail_default gmail_msg" style="display:inline"><font color="#000000" face="arial, helvetica, sans-serif" class="gmail_msg">As Tempest is responsible of testing all stable branch behavior too, Should</font></div><font color="#000000" face="arial, helvetica, sans-serif" class="gmail_msg"> we keep testing them till all Mitaka EOL (till APIs are in deprecated state in all stable branch) ?</font><br class="gmail_msg"></div></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">[0] <a href="https://github.com/openstack/defcore/blob/master/2017.01.json" class="gmail_msg" target="_blank">https://github.com/openstack/defcore/blob/master/2017.01.json</a></div><div class="gmail_msg">[1] <a href="http://lists.openstack.org/pipermail/openstack-dev/2017-March/113166.html" class="gmail_msg" target="_blank">http://lists.openstack.org/pipermail/openstack-dev/2017-March/113166.html</a></div><div class="gmail_msg"><br class="gmail_msg"></div></div></div></div></blockquote></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_default gmail_msg" style="font-family:arial,helvetica,sans-serif;color:rgb(0,0,0)">[2] <a href="https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294" class="gmail_msg" target="_blank">https://git.openstack.org/cgit/openstack/defcore/tree/2017.01.json#n294</a> </div></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><div class="gmail_msg"></div><div class="gmail_msg"> </div><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Doug<br class="gmail_msg">
<br class="gmail_msg">
><br class="gmail_msg">
> [0]<br class="gmail_msg">
> <a href="https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html" rel="noreferrer" class="gmail_msg" target="_blank">https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html</a><br class="gmail_msg">
<br class="gmail_msg">
__________________________________________________________________________<br class="gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail_msg" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="gmail_msg">
</blockquote></div><br class="gmail_msg"></div></div>
<br class="gmail_msg">__________________________________________________________________________<br class="gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail_msg" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="gmail_msg">
<br class="gmail_msg"></blockquote></div></div></div>
__________________________________________________________________________<br class="gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail_msg" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="gmail_msg">
</blockquote></div></div></div>