[openstack-dev] [QA] Questions about test policy for scenario test

Fei Long Wang feilong at catalyst.net.nz
Wed Jun 25 22:06:52 UTC 2014


Good to know. I think it's a good idea to implement a common compute
verifier after instances booted. Maybe we can define different checking
levels so that it can be leveraged by different test cases. I will see
what I can do.

On 24/06/14 22:27, Sean Dague wrote:
> On 06/24/2014 01:29 AM, Fei Long Wang wrote:
>> Greetings,
>>
>> We're leveraging the scenario test of Tempest to do the end-to-end
>> functional test to make sure everything work great after upgrade,
>> patching, etc. And We're happy to fill the gaps we found. However, I'm a
>> little bit confused about the test policy from the scenario test
>> perspective, especially comparing with the API test. IMHO, scenario test
>> will cover some typical work flows of one specific service or mixed
>> services, and it would be nice to make sure the function is really
>> working instead of just checking the object status from OpenStack
>> perspective. Is that correct?
>>
>> For example, live migration of Nova, it has been covered in API test of
>> Tempest (see
>> https://github.com/openstack/tempest/blob/master/tempest/api/compute/test_live_block_migration.py).
>> But as you see, it just checks if the instance is Active or not instead
>> of checking if the instance can be login/ssh successfully. Obviously,
>> from an real world view, we'd like to check if it's working indeed. So
>> the question is, should this be improved? If so, the enhanced code
>> should be in API test, scenario test or any other places? Thanks you.
> The fact that computes aren't verified fully during the API testing is
> mostly historical. I think they should be. The run_ssh flag used to be
> used for this, however because of some long standing race conditions in
> the networking stack, that wasn't able to be turned on in upstream
> testing. My guess is that it's rotted now.
>
> We've had some conversations in the QA team about a compute verifier
> that would be run after any of the compute jobs to make sure they booted
> correctly, and more importantly, did a very consistent set of debug
> capture when they didn't. Would be great if that's something you'd like
> to help out with.
>
> 	-Sean
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
Cheers & Best regards,
Fei Long Wang (???)
--------------------------------------------------------------------------
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flwang at catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
-------------------------------------------------------------------------- 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140626/4e4f354d/attachment.html>


More information about the OpenStack-dev mailing list