[openstack-dev] [qa][tempest] Where to do response body validation
cbkyeoh at gmail.com
Thu Mar 13 07:08:04 UTC 2014
On Thu, 13 Mar 2014 06:40:58 +0000
Kenichi Oomichi <oomichi at mxs.nes.nec.co.jp> wrote:
> Hi Chris,
> Thank you for picking it up,
> > -----Original Message-----
> > From: Christopher Yeoh [mailto:cbkyeoh at gmail.com]
> > Sent: Thursday, March 13, 2014 1:56 PM
> > To: OpenStack Development Mailing List
> > Subject: [openstack-dev] [qa][tempest] Where to do response body
> > validation
> > The new tempest body response validation is being added to
> > individual testcases. See this as an example:
> > https://review.openstack.org/#/c/78149
> > After having a look at https://review.openstack.org/#/c/80174/
> > I'm now thinking that perhaps we should be doing the response
> > validation in the tempest/services/compute classes. And only check
> > the response body if the status code is a success code (and then
> > check that it is an appropriate success code).
> > I think this will lead to fewer changes needed in the end as the
> > response body checking will not needed to be added to individual
> > tests.
> > There may be some complications with handling extensions, but I
> > think they are all implement backwards compatible behaviour so
> > should be ok.
> > Anyone have any thoughts about this alternative approach?
> I like the above idea that the response body validation will be
> operated in REST client. Tempest will be able to check response body
> anytime and reduce the test code.
> One concern is that Nova API returns different response body when
> admin user or not. I'd like to check response body containing
> attributes what admin user can get. For example, "get server info"
> API returns a response including "OS-EXT-STS" and "OS-EXT-SRV-ATT"
> attributes when an admin user.
Yes that's true, I think that's a case of extensions and policy changing
what is returned.
> So how about operating the basic validation(without special
> attributes) only in each REST client?
> The the special validation(such as the above admin info) would be
> operated in each test. The schema size for the special cases could be
That sounds reasonable to me. We'd just check for the additional
parameters that we expect to be present.
More information about the OpenStack-dev