<div dir="ltr">Hi team,<div><br></div><div>here is a short update:</div><div><br></div><div>1) The QA user story for destructive testing of OpenStack cloud is on review [1].</div><div>2) The spec for a new framework which will focus on HA/failover and destructive testing is no review [2].</div><div>3) The commit for the new repository is on review [3] as well.</div><div><br></div><div>[1] <a href="https://review.openstack.org/#/c/396142">https://review.openstack.org/#/c/396142</a></div><div>[2] <a href="https://review.openstack.org/#/c/399618">https://review.openstack.org/#/c/399618</a></div><div>[3] <a href="https://review.openstack.org/#/c/374667">https://review.openstack.org/#/c/374667</a></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Oct 14, 2016 at 3:47 AM, Ghanshyam Mann <span dir="ltr"><<a href="mailto:ghanshyam.mann@nectechnologies.in" target="_blank">ghanshyam.mann@nectechnologies.in</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-IN" link="blue" vlink="purple">
<div class="m_1539674485926932036WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I like os-faults library which can provide the abstraction over different destructive actions like reboot/poweroff node etc.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">But not much clear about Stepler that what all tests it will contain. Tempest do have scenario tests which can hit the production env
with significant way of testing scenario.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">It can cover the end user scenario also which involves the interaction of public OpenStack APIs and always in enhancement state by
adding more and more scenario tests.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Few query over Stepler as separate project:<u></u><u></u></span></p>
<p class="m_1539674485926932036MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><span>1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Is Stepler will cover only tests which hits the node level action(reboot, HA etc)?
<u></u><u></u></span></p>
<p class="m_1539674485926932036MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><span>2.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">This should not mix the scenario tests which are in Tempest scope because that can make confusion for developers (where to
add scenario tests) as well as for tester(from where to run what scenario tests).<u></u><u></u></span></p>
<p class="m_1539674485926932036MsoListParagraph"><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><span>3.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">How we make sure those tests run fine or at least run while adding.<u></u><u></u></span></p>
<p class="m_1539674485926932036MsoListParagraph" style="margin-left:72.0pt">
<u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><span>a.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">I think devstack enhancement for multi-node etc can do this as mentioned by you also.<u></u><u></u></span></p>
<p class="m_1539674485926932036MsoListParagraph" style="margin-left:72.0pt">
<u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><span>b.<span style="font:7.0pt "Times New Roman"">
</span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">If so then why not adding those tests in Tempest only using os-faults lib ?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Overall I feel os-faults as lib is really nice idea but tests scope can go in Tempest under HW_scenario (or something else) till
it does not break basic principle of Tempest. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Thanks<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">gmann</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Timur Nurlygayanov [mailto:<a href="mailto:tnurlygayanov@mirantis.com" target="_blank">tnurlygayanov@<wbr>mirantis.com</a>]
<br>
<b>Sent:</b> 06 October 2016 20:09<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions) <<a href="mailto:openstack-dev@lists.openstack.org" target="_blank">openstack-dev@lists.<wbr>openstack.org</a>><br>
<b>Subject:</b> Re: [openstack-dev] [QA] The end-user test suite for OpenStack clusters<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Ken, it is a good idea!<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The plan is to develop os-faults as a library which will be able<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">to manage cluster nodes and OpenStack services on these nodes.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">It is a good idea to add some Tempest tests which will use<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">os-faults library as well for some API tests.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The Stepler framework [1] will use os-faults to perform all destructive<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">actions in the clouds (the reboot of nodes, restart of OpenStack services,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">enable/disable network interfaces or some firewall rules and etc).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">We need to get +1 from you in [1] to create the repository with<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">advanced end-user scenario tests.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Thank you!<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">[1] <a href="https://review.openstack.org/#/c/374667/" target="_blank"><span style="font-size:9.5pt">https://review.openstack.<wbr>org/#/c/374667/</span></a><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov <<a href="mailto:ylobankov@mirantis.com" target="_blank">ylobankov@mirantis.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal">Hi Ken,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">OS-Faults doesn't have any scenarios in the tree yet (the project is two months old), but you can find some examples of the use in the os-faults/examples directory.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Yaroslav Lobankov.<u></u><u></u></p>
</div>
</div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi <<a href="mailto:ken1ohmichi@gmail.com" target="_blank">ken1ohmichi@gmail.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal">Hi Timur,<br>
<br>
Thanks for your explanation.<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov <<a href="mailto:tnurlygayanov@mirantis.com" target="_blank">tnurlygayanov@mirantis.com</a>>:<br>
><br>
>> I am guessing the above "restart nodes" is for verifying each<br>
>> OpenStack service restarts successfully, right?<br>
><br>
> Yes, this is right. And we also will check that HA logic for these<br>
> services works correctly (for example, rescheduling of L3 Neutron<br>
> agents for networks).<br>
><br>
>> But these service scripts are provided by distributors, and Devstack<br>
>> itself doesn't contain service scripts IIUC.<br>
>> So I'd like to know how to verify it on Devstack clouds.<br>
><br>
> Yes, DevStack doesn't support many scenarios which are actual<br>
> and should be supported on the production clouds.<br>
> It will be not possible to run all advanced test scenarios for DevStack<br>
> clouds,<br>
> just because DevStack can't deploy OpenStack cloud with 3 controllers<br>
> now (so, probably it will be possible in the future).<br>
><br>
> Of course, some advanced scenarios will support DevStack clouds,<br>
> for example, some test scenarios which are based on customer-found<br>
> issues from the real production clouds, like upload of the large images<br>
> (100+ Gb)<br>
> to Glance with Swift backend. Such cases are important for verification of<br>
> pre-production environments, but not very important for CI gate jobs.<br>
><br>
> It is also important to note that in these advanced cases we are targeting<br>
> to check not only the logic of Python code, but also the correct<br>
> configuration<br>
> of all OpenStack components on some pre-production OpenStack clusters.<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal">I guessed some part of os-faults can be moved to Tempest if os-faults<br>
contains API tests for enabling/disabling OpenStack services.<br>
Then, os-faults would be able to concentrate on more destructive tests<br>
like rebooting physical nodes, etc.<br>
However, I could not find any actual scenarios on current os-faults<br>
(<a href="https://github.com/openstack/os-faults" target="_blank">https://github.com/openstack/<wbr>os-faults</a>).<br>
That seems to just contain some abstraction layers and unit tests. Can<br>
we see actual test scenarios of os-faults ?<br>
Maybe I missed something.<br>
<br>
Thanks<br>
<span class="m_1539674485926932036m8889648732652088501hoenzb"><span style="color:#888888">Ken Ohmichi</span></span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Timur,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Senior QA Manager<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">OpenStack Projects<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Mirantis Inc<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div></div></div>
</div>
</div>
<br>______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><font color="#888888"><font color="#888888"><br></font></font><div style="font-family:arial;font-size:small">Timur,</div><div style="font-family:arial;font-size:small">QA Manager</div><div style="font-family:arial;font-size:small">OpenStack Projects</div><div style="font-family:arial;font-size:small">Mirantis Inc</div></div></div></div></div></div></div></div></div></div></div>
</div>