Open Stack

Hi Chris,

Thank you for picking it up,

> -----Original Message-----
> From: Christopher Yeoh [mailto:cbkyeoh at gmail.com]
> Sent: Thursday, March 13, 2014 1:56 PM
> To: OpenStack Development Mailing List
> Subject: [openstack-dev] [qa][tempest] Where to do response body validation
> 
> The new tempest body response validation is being added to individual
> testcases. See this as an example:
> 
> https://review.openstack.org/#/c/78149
> 
> After having a look at https://review.openstack.org/#/c/80174/
> I'm now thinking that perhaps we should be doing the response validation
> in the tempest/services/compute classes. And only check the
> response body if the status code is a success code (and then check that
> it is an appropriate success code).
> 
> I think this will lead to fewer changes needed in the end as the
> response body checking will not needed to be added to individual tests.
> 
> There may be some complications with handling extensions, but I think
> they are all implement backwards compatible behaviour so should be ok.
> 
> Anyone have any thoughts about this alternative approach?

I like the above idea that the response body validation will be operated
in REST client. Tempest will be able to check response body anytime and
reduce the test code.

One concern is that Nova API returns different response body when admin
user or not. I'd like to check response body containing attributes what
admin user can get. For example, "get server info" API returns a response
including "OS-EXT-STS" and "OS-EXT-SRV-ATT" attributes when an admin user.

So how about operating the basic validation(without special attributes)
only in each REST client?
The the special validation(such as the above admin info) would be operated
in each test. The schema size for the special cases could be reduced.


Thanks
Ken'ichi Ohmichi