Open Stack

On Thu, 13 Mar 2014 06:40:58 +0000
Kenichi Oomichi <oomichi at mxs.nes.nec.co.jp> wrote:
> 
> Hi Chris,
> 
> Thank you for picking it up,
> 
> > -----Original Message-----
> > From: Christopher Yeoh [mailto:cbkyeoh at gmail.com]
> > Sent: Thursday, March 13, 2014 1:56 PM
> > To: OpenStack Development Mailing List
> > Subject: [openstack-dev] [qa][tempest] Where to do response body
> > validation
> > 
> > The new tempest body response validation is being added to
> > individual testcases. See this as an example:
> > 
> > https://review.openstack.org/#/c/78149
> > 
> > After having a look at https://review.openstack.org/#/c/80174/
> > I'm now thinking that perhaps we should be doing the response
> > validation in the tempest/services/compute classes. And only check
> > the response body if the status code is a success code (and then
> > check that it is an appropriate success code).
> > 
> > I think this will lead to fewer changes needed in the end as the
> > response body checking will not needed to be added to individual
> > tests.
> > 
> > There may be some complications with handling extensions, but I
> > think they are all implement backwards compatible behaviour so
> > should be ok.
> > 
> > Anyone have any thoughts about this alternative approach?
> 
> I like the above idea that the response body validation will be
> operated in REST client. Tempest will be able to check response body
> anytime and reduce the test code.
> 
> One concern is that Nova API returns different response body when
> admin user or not. I'd like to check response body containing
> attributes what admin user can get. For example, "get server info"
> API returns a response including "OS-EXT-STS" and "OS-EXT-SRV-ATT"
> attributes when an admin user.

Yes that's true, I think that's a case of extensions and policy changing
what is returned. 

> 
> So how about operating the basic validation(without special
> attributes) only in each REST client?
> The the special validation(such as the above admin info) would be
> operated in each test. The schema size for the special cases could be
> reduced.

That sounds reasonable to me. We'd just check for the additional
parameters that we expect to be present.

Chris