Open Stack

Thanks for taking this out to the list, you're right this is probably the proper
forum for discussing this.

On Tue, Feb 05, 2013 at 11:48:09AM +0200, Yaniv Kaul wrote:
> On 05/02/13 11:33, Patil, Kavan K (HPCS) wrote:
> >
> >Hi Matthew Treinish,
> >
> >[I am not sure how one could reply to comments on gerrit, so I am
> >mailing this across. I am using the list as other people could
> >express their thoughts here.]

For the record you can just hit 'r' or use the review button on the UI.
Gerrit comments are treated as code reviews, so just write your comments
in a 0 review on your commit.

> >
> >Quoted Text from your review comment for :
> >https://review.openstack.org/#/c/20464/
> ><https://review.openstack.org/#/c/20464/>
> >
> >---------------
> >
> >“I also have some concerns about whether using http from
> >glanceclient common is distinct enough from just testing
> >glanceclient (like we do currently) for tempest. I'm not sure if
> >we should use tempest's RestClient or another tempest specific
> >http client for glance, or if using glanceclient's http lib is
> >enough. If someone with more knowledge about this could chime
> >in...
> >
> >Either way I think writing a client class for the API is the right
> >way to do it, instead of baking the request formatting and headers
> >into the tests like you did here. I have a WIP up for this at:
> >https://review.openstack.org/#/c/19963/ . It doesn't currently
> >work because of issues with image upload, but I'll rework it ASAP
> >depending on what the consensus is on which http client to use.
> >(Using glance client's would be easier) Once that's done I'll
> >update the review so you can rebase the tests on top of it.”
> >
> >---------------
> >
> >About your first point, i.e. which http base client to use while
> >writing tempest tests: I think we will be reinventing the wheel if
> >we try to write our own http base class, especially for tricky
> >things like SSL support. The glanceclient.common.http module has
> >special consideration for SSL (https) based connections and we
> >would also have to do the same thing if we want to write a http
> >base client ourselves. So I would vote to just use
> >glanceclient.common.http as the base http client.
> >
> 
> I'm not sure why SSL is complex. It's been done before, many times.
> 
The special case with glance actually isn't ssl, it's the chunked upload. The
Tempest RestClient class uses httplib2 which does not support async (like
chunking) which is why that WIP image client I pushed doesn't work with image
upload. However, my concern with using the http lib from glanceclient is whether
that differentiates itself from just testing the client. In tempest we are trying
to test the API which is why all the other projects have their own clients in
Tempest, so we are testing the API. Using the clients can mask issues in the
API, especially if they're both broken. So the open question, which I don't have
the answer to, is whether using http lib from glanceclient is different enough
from using glanceclient for tests? I tend to think so, but I think we need input
from others before we can say definitively.

> >Next, I agree that we should try to have a API client wrapper
> >(like the ones for Compute and Quantum), and use that in our
> >tests. I didn’t do it because I thought that will again be like
> >implementing glanceclient.v1.images, while we could do all the
> >stuff through the base http client directly in tests as I have
> >done in my proposal.
> >
> 
> Well, you could go even further and ask why not use the command line
> tool? This way we are testing:
> CLI -> client -> API
> 
> (say we use 'pexpect' or something for the command line client, or
> just parse it).
> 
> I think there are cons and pros to each direction - I like that we
> test all those layers in one sweep, but we are definitely adding
> complexity to the debugging process if there are issues.
> I'm mostly concerned that the clients do not have stable versions
> and are just rolling along.
> 

We try to avoid using the official clients in tempest for testing. Since we are
trying to test the API, using the official client can sometimes mask issues in
the API. The rolling version is a concern, but is also unavoidable because we
use tempest as a gate test. I think the plan is to start doing stable drops of
Tempest with each release. But, I'm not sure what ever happened with that.
(There are branches for diablo and essex)


> >Still if we want to maintain consistency (w.r.t. other services)
> >we could implement an Image v1 API client for Tempest, as you have
> >done here: https://review.openstack.org/#/c/19963/ and use that in
> >the test scripts instead of plain http.
> >
> 
> I'm in favor of a simple client, mainly because I don't 'trust' the
> clients and want to maintain independence in implementation
> according to the API, and not use an implementation that may have
> been tailored between the client and the server.
> Y.
> 

Using a tempest images client is the right way to implement real glance
testing in tempest. The only question is which http library do we use for that
client.

Thanks,

-Matt Treinish