Open Stack

 ---- On Thu, 03 Jan 2019 07:29:27 +0900 Jeremy Freudberg <jeremyfreudberg at gmail.com> wrote ---- 
 > Hey Luigi. 
 >  
 > I poked around in Tempest and saw these code bits: 
 > https://github.com/openstack/tempest/blob/master/tempest/lib/common/rest_client.py#L210 
 > https://github.com/openstack/tempest/blob/f9650269a32800fdcb873ff63f366b7bc914b3d7/tempest/lib/auth.py#L53 
 >  
 > Here's a patch which takes advantage of those bits to append the 
 > version to the unversioned base URL: 
 > https://review.openstack.org/#/c/628056/ 
 >  
 > Hope it works without regression (I'm a bit worried since Tempest does 
 > its own URL mangling rather than nicely use keystoneauth...) 

Yeah, that is the code where service client can tell which version of API tests needs to use.
This was kind of hack we do in Tempest for different versioned API with same endpoint of service
Other way to test different API version in Tempest is via catalog_type. You can define the different
jobs running the same test for different versioned endpoints defined in tempest's config options catalog_type.
But if you want to run the both versions test in same job then api_version is the way to do that.

 >  
 > On Wed, Jan 2, 2019 at 5:19 AM Luigi Toscano <ltoscano at redhat.com> wrote: 
 > > 
 > > Hi all, 
 > > 
 > > I'm working on adding support for APIv2 to the Sahara tempest plugin. 
 > > 
 > > If I get it correctly, there are two main steps 
 > > 
 > > 1) Make sure that that tempest client works with APIv2 (and don't regress with 
 > > APIv1.1). 
 > > 
 > > This mainly mean implementing the tempest client for Sahara APIv2, which 
 > > should not be too complicated. 
 > > 
 > > On the other hand, we hit an issue with the v1.1 client in an APIv2 
 > > environment. 
 > > A change associated with API v2 is usage of an unversioned endpoint for the 
 > > deployment (see https://review.openstack.org/#/c/622330/ , without the /v1,1/$ 
 > > (tenant_id) suffix) which should magically work with both API variants, but it 
 > > seems that the current tempest client fails in this case: 
 > > 
 > > http://logs.openstack.org/30/622330/1/check/sahara-tests-tempest/7e02114/job-output.txt.gz#_2018-12-05_21_20_23_535544 
 > > 
 > > Does anyone know if this is an issue with the code of the tempest tests (which 
 > > should maybe have some logic to build the expected endpoint when it's 
 > > unversioned, like saharaclient does) or somewhere else? 
 > > 
 > > 
 > > 2) fix the tests to support APIv2. 
 > > 
 > > Should I duplicate the tests for APIv1.1 and APIv2? Other projects which 
 > > supports different APIs seems to do this. 
 > > But can I freely move the existing tests under a subdirectory 
 > > (sahara_tempest_plugins/tests/api/ -> sahara_tempest_plugins/tests/api/v1/), 
 > > or are there any compatibility concerns? Are the test ID enough to ensure that 
 > > everything works as before? 

It depends on compatibility and state of version v1.1 and v2. If both are supposed to
be compatible at least feature wise then you should not duplicate the test instead
run the same set of tests against both version either in same job or in different job.

We do that for nova, cinder, image etc where we run same set of the test against 
1. compute v2.0 and v2.1, 2. volume v2 and v3. We have done that testing those
in different jobs with defining different catalog_type(version endpoints). 

Duplicating the tests has two drawbacks, 1. maintenance 2. easy to loose the coverage
against specific version. 


-gmann

 > > 
 > > And what about CLI tests currently under sahara_tempest_plugin/tests/cli/ ? 
 > > They supports both API versions through a configuration flag. Should they be 
 > > duplicated as well? 
 > > 
 > > 
 > > Ciao 
 > > (and happy new year if you have a new one in your calendar!) 
 > > -- 
 > > Luigi 
 > > 
 > > 
 > > 
 >  
 >