Open Stack

Hi Matthew and Yaniv,

Thanks for your reply. I will wait for a proper Tempest Image API client before submitting the tests.

Regards,
Kavan

-----Original Message-----
From: Matthew Treinish [mailto:treinish at linux.vnet.ibm.com] 
Sent: Tuesday, February 05, 2013 10:54 PM
To: All Things QA.
Subject: Re: [openstack-qa] About Glance Client in Tempest


Thanks for taking this out to the list, you're right this is probably the proper forum for discussing this.

On Tue, Feb 05, 2013 at 11:48:09AM +0200, Yaniv Kaul wrote:
> On 05/02/13 11:33, Patil, Kavan K (HPCS) wrote:
> >
> >Hi Matthew Treinish,
> >
> >[I am not sure how one could reply to comments on gerrit, so I am 
> >mailing this across. I am using the list as other people could 
> >express their thoughts here.]

For the record you can just hit 'r' or use the review button on the UI.
Gerrit comments are treated as code reviews, so just write your comments in a 0 review on your commit.

> >
> >Quoted Text from your review comment for :
> >https://review.openstack.org/#/c/20464/
> ><https://review.openstack.org/#/c/20464/>
> >
> >---------------
> >
> >“I also have some concerns about whether using http from glanceclient 
> >common is distinct enough from just testing glanceclient (like we do 
> >currently) for tempest. I'm not sure if we should use tempest's 
> >RestClient or another tempest specific http client for glance, or if 
> >using glanceclient's http lib is enough. If someone with more 
> >knowledge about this could chime in...
> >
> >Either way I think writing a client class for the API is the right 
> >way to do it, instead of baking the request formatting and headers 
> >into the tests like you did here. I have a WIP up for this at:
> >https://review.openstack.org/#/c/19963/ . It doesn't currently work 
> >because of issues with image upload, but I'll rework it ASAP 
> >depending on what the consensus is on which http client to use.
> >(Using glance client's would be easier) Once that's done I'll update 
> >the review so you can rebase the tests on top of it.”
> >
> >---------------
> >
> >About your first point, i.e. which http base client to use while 
> >writing tempest tests: I think we will be reinventing the wheel if we 
> >try to write our own http base class, especially for tricky things 
> >like SSL support. The glanceclient.common.http module has special 
> >consideration for SSL (https) based connections and we would also 
> >have to do the same thing if we want to write a http base client 
> >ourselves. So I would vote to just use glanceclient.common.http as 
> >the base http client.
> >
> 
> I'm not sure why SSL is complex. It's been done before, many times.
> 
The special case with glance actually isn't ssl, it's the chunked upload. The Tempest RestClient class uses httplib2 which does not support async (like
chunking) which is why that WIP image client I pushed doesn't work with image upload. However, my concern with using the http lib from glanceclient is whether that differentiates itself from just testing the client. In tempest we are trying to test the API which is why all the other projects have their own clients in Tempest, so we are testing the API. Using the clients can mask issues in the API, especially if they're both broken. So the open question, which I don't have the answer to, is whether using http lib from glanceclient is different enough from using glanceclient for tests? I tend to think so, but I think we need input from others before we can say definitively.

> >Next, I agree that we should try to have a API client wrapper (like 
> >the ones for Compute and Quantum), and use that in our tests. I 
> >didn’t do it because I thought that will again be like implementing 
> >glanceclient.v1.images, while we could do all the stuff through the 
> >base http client directly in tests as I have done in my proposal.
> >
> 
> Well, you could go even further and ask why not use the command line 
> tool? This way we are testing:
> CLI -> client -> API
> 
> (say we use 'pexpect' or something for the command line client, or 
> just parse it).
> 
> I think there are cons and pros to each direction - I like that we 
> test all those layers in one sweep, but we are definitely adding 
> complexity to the debugging process if there are issues.
> I'm mostly concerned that the clients do not have stable versions and 
> are just rolling along.
> 

We try to avoid using the official clients in tempest for testing. Since we are trying to test the API, using the official client can sometimes mask issues in the API. The rolling version is a concern, but is also unavoidable because we use tempest as a gate test. I think the plan is to start doing stable drops of Tempest with each release. But, I'm not sure what ever happened with that.
(There are branches for diablo and essex)


> >Still if we want to maintain consistency (w.r.t. other services) we 
> >could implement an Image v1 API client for Tempest, as you have done 
> >here: https://review.openstack.org/#/c/19963/ and use that in the 
> >test scripts instead of plain http.
> >
> 
> I'm in favor of a simple client, mainly because I don't 'trust' the 
> clients and want to maintain independence in implementation according 
> to the API, and not use an implementation that may have been tailored 
> between the client and the server.
> Y.
> 

Using a tempest images client is the right way to implement real glance testing in tempest. The only question is which http library do we use for that client.

Thanks,

-Matt Treinish


_______________________________________________
openstack-qa mailing list
openstack-qa at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa