[openstack-dev] [nova][placement] Add API Sample tests for Placement APIs
cdent+os at anticdent.org
Thu Jun 22 09:29:56 UTC 2017
On Wed, 21 Jun 2017, Shewale, Bhagyashri wrote:
> I would like to write functional tests to check the exact req/resp for each placement API for all supported versions similar
> to what is already done for other APIs under nova/tests/functional/api_sample_tests/api_samples/*.
> These request/response json samples can be used by the api.openstack.org and in the manuals.
> There are already functional tests written for placement APIs under nova/tests/functional/api/openstack/placement,
> but these tests doesn't check the entire HTTP response for each API for all supported versions.
> I think adding such functional tests for checking response for each placement API would be beneficial to the project.
> If there is an interest to create such functional tests, I can file a new blueprint for this activity.
At Matt points out elsewhere, we made a choice to not use the
api_samples format when developing placement. There were a few
different reasons for this:
* we wanted to limit the amount of nova code used in placement, to
ease the eventual extraction of placement to its own code
* some of us wanted to use gabbi  as the testing framework as it is
nicely declarative  and keeps the request and response in the same
* we were building the api framework from scratch and doing what
amounts to test driven development  using functional tests and
gabbi works well for that
* testing the full response isn't actually a great way to test an
API in a granular way; the info is useful to have but it isn't a
useful test (from a development standpoint)
But, as you've noted, this means there isn't a single place to go to
see a collection of a full request and response bodies. That
information can be extracted from the gabbi tests, but it's a) not
immediately obvious, b) requires interpretation.
Quite some time ago I started a gabbi-based full request and
response suite of tests  but it was never finished and now is
very out of date.
If the end goal is to have a set of documents that pair all the
possible requests (with bodies) with all possible responses (with
bodies), gabbi could easily create this in its "verbose" mode 
when run as functional tests or with the gabbi-run  command that
can run against a running service.
So I would suggest that we more completely explain the goal or goals
that you're trying to satisfy and then see how we can use the
existing tooling to fill them. Some questions related to that:
* who or what needs to consume these JSON samples?
* do they need to be tests against current code, or are they
primarily reference info?
* what are the value propositions associated with fully validating
the structure and content of the response bodies?
We can relatively easily figure out some way to drive gabbi to
produce the desired information, but first we want to make sure that
the information produced is going to be the right info (that is,
will satisfy the needs of whoever wants it).
I am, as Matt mentioned, on holiday at the moment so my response to
any future messages may be delayed, but I'll catch up as I'm able.
 verbose mode can print out request and response headers, bodies,
or both. If the bodies are JSON, it will be pretty printed.
Chris Dent ┬──┬◡ﾉ(° -°ﾉ) https://anticdent.org/
freenode: cdent tw: @anticdent
More information about the OpenStack-dev