<html><body><div style="font-family: times new roman, new york, times, serif; font-size: 12pt; color: #000000"><br><div><br></div><br><div><br></div><hr id="zwchr"><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>From: </b>"Fei Long Wang" <feilong@catalyst.net.nz><br><b>To: </b>"OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org><br><b>Cc: </b>bruno@catalyst.net.nz<br><b>Sent: </b>Tuesday, June 24, 2014 8:29:03 AM<br><b>Subject: </b>[openstack-dev] [QA] Questions about test policy for scenario test<br><div><br></div>Greetings,<br><div><br></div>We're leveraging the scenario test of Tempest to do the end-to-end<br>functional test to make sure everything work great after upgrade,<br>patching, etc. And We're happy to fill the gaps we found. However, I'm a<br>little bit confused about the test policy from the scenario test<br>perspective, especially comparing with the API test. IMHO, scenario test<br>will cover some typical work flows of one specific service or mixed<br>services, and it would be nice to make sure the function is really<br>working instead of just checking the object status from OpenStack<br>perspective. Is that correct?<br><div><br></div>For example, live migration of Nova, it has been covered in API test of<br>Tempest (see<br>https://github.com/openstack/tempest/blob/master/tempest/api/compute/test_live_block_migration.py).<br>But as you see, it just checks if the instance is Active or not instead<br>of checking if the instance can be login/ssh successfully</blockquote><div>Seems to me, that what you want is to add migration test to</div><div><a href="https://github.com/openstack/tempest/blob/master/tempest/scenario/test_network_advanced_server_ops.py">https://github.com/openstack/tempest/blob/master/tempest/scenario/test_network_advanced_server_ops.py</a></div><div>This scenario does exactly what you are looking for</div><div>1. check VM connectivity</div><div>2. mess with VM (reboot, resize, or in your case - migrate)</div><div>3. check VM connectivity</div><blockquote style="border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;">. Obviously,<br>from an real world view, we'd like to check if it's working indeed. So<br>the question is, should this be improved? If so, the enhanced code<br>should be in API test, scenario test or any other places? Thanks you.<br><div><br></div>-- <br>Cheers & Best regards,<br>Fei Long Wang (王飞龙)<br>--------------------------------------------------------------------------<br>Senior Cloud Software Engineer<br>Tel: +64-48032246<br>Email: flwang@catalyst.net.nz<br>Catalyst IT Limited<br>Level 6, Catalyst House, 150 Willis Street, Wellington<br>-------------------------------------------------------------------------- <br><div><br></div><br>_______________________________________________<br>OpenStack-dev mailing list<br>OpenStack-dev@lists.openstack.org<br>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br></blockquote><br></div></body></html>