Open Stack

What is our strategy about testing combination of API versions?

 

If we start testing all possible combinations of API the gate time will grow
exponentially - e.g. all tests would be run using v2 and v3 auth, all tests
using a VM would create it via nova v2 and v3, all tests using an image
would fetch it via glance v1 and v2.

 

My idea would be to:

-          Test all available versions only in the dedicated tests. For
instance identity tests shall verify keystone v2 and v3, compute tests shall
verify nova v2 and v3.

-          In the gate each tests shall use only the latest stable API
version from other components.  This means that all tests shall authenticate
via keystone v2 as long v3 is completed and stable. All tests which require
a VM will set it up via nova v2 as long as nova v3 is completed and stable.

-          We could also have a non-gating job which uses the latest
available API for each component in each tests, to prepare for the version
switch in the gate.

 

Whichever approach we go for, we need our tests and base classes to be
designed to allow for easily switching between API versions for one of the
APIs.

Any given test should be executable with any chosen combination of API
versions, so that we can select which combination we want to use in the
gate.

 

Thoughts?

 

Thanks

 

andrea

 

 

-- 

Andrea Frittoli

IaaS Systems Engineering Team

HP Cloud Services

Tel: +44(0)11731 62324

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-qa/attachments/20130619/bbbbd59c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6202 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-qa/attachments/20130619/bbbbd59c/attachment.bin>