Open Stack

Can you move the authentication to the tempest/clients.py,
and let the rest_clients to get a token and service_url parameter
 instead of user/tenant/passwd/(domain) + auth_url ?




----- Original Message -----
> From: "Andrea Frittoli (Cloud Services)" <frittoli at hp.com>
> To: "All Things QA." <openstack-qa at lists.openstack.org>
> Sent: Tuesday, June 25, 2013 12:09:52 PM
> Subject: Re: [openstack-qa] Multi API version testing
> 
> I filed
> https://blueprints.launchpad.net/tempest/+spec/multi-keystone-api-version-te
> sts about supporting multiple versions of keystone in non-keystone tests.
> 
> andrea
> 
> -----Original Message-----
> From: Frittoli, Andrea (Cloud Services)
> Sent: 21 June 2013 09:39
> To: All Things QA.
> Subject: Re: [openstack-qa] Multi API version testing
> 
> > -----Original Message-----
> > From: Matthew Treinish [mailto:mtreinish at kortar.org]
> > Sent: 20 June 2013 16:55
> > To: All Things QA.
> > Subject: Re: [openstack-qa] Multi API version testing
> > 
> > On Wed, Jun 19, 2013 at 07:38:17AM +0000, Frittoli, Andrea (Cloud
> Services) wrote:
> > > What is our strategy about testing combination of API versions?
> > > 
> > >  
> > > 
> > > If we start testing all possible combinations of API the gate time
> > > will grow exponentially - e.g. all tests would be run using v2 and
> > > v3 auth, all tests using a VM would create it via nova v2 and v3,
> > > all tests using an image would fetch it via glance v1 and v2.
> > > 
> > >  
> > > 
> > > My idea would be to:
> > > 
> > > -          Test all available versions only in the dedicated tests. For
> > > instance identity tests shall verify keystone v2 and v3, compute
> > > tests shall verify nova v2 and v3.
> > 
> > So in the api tests this is done for the most part I believe. I wrote
> > the
> glance tests to be a framework for doing multiple api version tests. The
> glance tests auto discover which version of the api are available and then
> test them appropriately. The underlying assumption is that we have to assume
> nothing will remain the same between API versions (because there isn't a
> guarantee of that) so we have to test everything in the new version (even if
> it remains unchanged) this is what I started to do for the glance testing,
> but there are probably still gaps in what is being tested. I'm sure that
> there are other gaps in which versions we test currently for the other apis
> too, so feel free to contribute to get us closer to full coverage.
> 
> +1 we need to retest the entire API set in each version
> 
> > 
> > > 
> > > -          In the gate each tests shall use only the latest stable API
> > > version from other components.  This means that all tests shall
> > > authenticate via keystone v2 as long v3 is completed and stable. All
> > > tests which require a VM will set it up via nova v2 as long as nova
> > > v3
> is completed and stable.
> > > 
> > > -          We could also have a non-gating job which uses the latest
> > > available API for each component in each tests, to prepare for the
> > > version switch in the gate.
> > > 
> > >  
> > > 
> > > Whichever approach we go for, we need our tests and base classes to
> > > be designed to allow for easily switching between API versions for
> > > one of the APIs.
> > > 
> > > Any given test should be executable with any chosen combination of
> > > API versions, so that we can select which combination we want to use
> > > in the gate.
> > 
> > So I don't think I follow this completely. With keystone being the
> exception (because it's used everywhere) do we have any tests that make
> straight api calls to another service? I know, for example, all the compute
> api tests for volumes and images are just using the nova api and api
> extensions. (which then use official python clients inside of nova) Then for
> things like the scenario tests we also use the python clients, and for most
> of the clients I don't think there is multiple api version support. So
> unless that gets implemented I think those are fine. But, if multiple
> version support becomes available for the clients we probably should revisit
> this for scenario and non-api testing.
> > 
> > I do agree though if we use a straight api call from another service
> > for
> any part of any test(not through an official client) it has to be able to
> support multiple versions(if there are multiple versions). I don't think
> that for keystone we aren't there yet, but I haven't looked at that code too
> closely so I'm not sure. So we definitely have a todo there, but I just
> don't think that this applies anywhere else yet.
> 
> Fair point, really the major use case is keystone. If we can implement that,
> we should be able to reuse the same concept for other APIs as well if
> needed.
> For instance if a volume tests requires a virtual machine, it probably does
> not matter much through which API version the VM is created, but still it
> would be good to be able to run the test with support from any compute api
> version.
> 
> The current approach is the code is two have additional client instances
> added to the manager using the keystone v3 api, e.g. from
> https://github.com/openstack/tempest/blob/master/tempest/clients.py
> 
>             if client_args_v3_auth:
>                 self.servers_client_v3_auth = SERVERS_CLIENTS[interface](
>                     *client_args_v3_auth)
>             else:
>                 self.servers_client_v3_auth = None
> 
> The client itself support working with both keystone versions, as
> authentication is ultimately delegated to the rest_client.py.
> 
> I can  file a blueprint to provide support for switching auth version used
> by all tests. It should quite small, most of the logic is in place already.
> 
> > 
> > I also agree that once we get to a point where multiple version
> > support is
> configurable for tests that we should have a separate gate job that runs
> with it.
> > But, I feel it should also be a gating job especially if the API is
> supported.
> 
> Perhaps we can run multiple jobs in parallel, each using a different version
> of the API?
> 
> >  
> > 
> > -Matt Treinish
> > 
> > _______________________________________________
> > openstack-qa mailing list
> > openstack-qa at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
> >
> 
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>