<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1777095846;
mso-list-template-ids:1483365532;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi Egle,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks a lot for the info!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards<o:p></o:p></p>
<p class="MsoNormal">Georg<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Egle Sigler [mailto:egle.sigler@rackspace.com] <br>
<b>Sent:</b> Monday, June 26, 2017 11:36 PM<br>
<b>To:</b> Georg Kunz <georg.kunz@ericsson.com>; interop-wg@lists.openstack.org<br>
<b>Subject:</b> Re: [Interop-wg] Tempest scenario tests<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Hello Georg,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">We had tried to make sure that tests test only one capability and not several. We had a spec that describes what we are looking for in test:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><a href="https://github.com/openstack/interop/blob/master/working_materials/interop_test_spec.rst">https://github.com/openstack/interop/blob/master/working_materials/interop_test_spec.rst</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Test guidelines:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-top:12.0pt;background:white"><b><i><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Guidelines<o:p></o:p></span></i></b></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Interop API functional tests should target and validate one capability. Some capabilities may require discovery calls and these should be included
also in the test. Each capability should have at least one interop API functional test.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Interop API functional tests should not use non-required capabilities.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Interop scenario tests may use several capabilities to test end to end functionality and none of the capabilities used may be non-required capabilities.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">4.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Tests must be reliable; race conditions must be avoided; tests should not be time dependent. If any of these problems occur the test will be flagged
and either fixed or removed from the guidelines.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">5.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Tests must be able to run consistently across all the releases covered by the Guidelines which are considered valid by the OpenStack Foundation for
the OpenStack Powered program.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">6.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Tests should not require more credentials than required to test the capability itself.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">7.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Tests must not be harmful to the end user's environment, that is no changing credentials nor destroying user's data nor altering any other user resource
other than those specifically created as part of the test.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-top:3.0pt;mso-margin-bottom-alt:auto;margin-left:1.0in;text-indent:-.25in;mso-list:l0 level1 lfo1;background:white">
<![if !supportLists]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E"><span style="mso-list:Ignore">8.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:12.0pt;font-family:"Segoe UI",sans-serif;color:#24292E">Test cases must setup any data they need to run and clean up after themselves. It is not acceptable to corrupt the user's environment or leave any
test data or test configuration behind.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Let us know if you have any questions!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">-Egle<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">Georg Kunz <<a href="mailto:georg.kunz@ericsson.com">georg.kunz@ericsson.com</a>><br>
<b>Date: </b>Monday, June 26, 2017 at 2:54 AM<br>
<b>To: </b>"<a href="mailto:interop-wg@lists.openstack.org">interop-wg@lists.openstack.org</a>" <<a href="mailto:interop-wg@lists.openstack.org">interop-wg@lists.openstack.org</a>><br>
<b>Subject: </b>[Interop-wg] Tempest scenario tests<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">Hi Egle, Rocky,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Thank you again for joining last week’s OPNFV Dovetail call. Your input was very valuable and I believe we agreed across the board that a closer collaboration between OPNFV Dovatail and the OpenStack Interop WG
is fundamental for both communities.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">I have one follow-up question: the test suites of the three OpenStack certification programs currently include only tempest API tests (located in tempest.api.X) and none of the tempest scenarios (located in tempest.scenario.X).
Is there a specific reason why those more complex end-to-end tests are currently not included?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">Georg<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>