Open Stack

There are a bunch of separable issues here so I'll give my 2 cents on 
each of them.

XML vs JSON
The current xml/json thing is a mess but it is really the fault of the 
APIs. There was a discussion on the dev list about whether, going 
forward, we should support xml at all. I'm not sure if there was 
agreement, but if we do support xml we should really do it in a way that 
supports auto-serialization back and forth. It might not be "good xml" 
to xml lovers but it would save us a maintenance nightmare. My view is 
that in Tempest we should leave this alone for the existing apis. It is 
what it is and we should focus on getting more tests. For new apis, 
hopefully we could use the serialize/deserialize code from the server 
project to the extent that we need to test both.

Tempest rest client vs official client
There are valid arguments on both sides of this issue. If we say that 
OpenStack apis are defined by the REST API, we have to test it. Users 
might use that API directly, or use a library from some other language. 
The existing official python clients are not enough. The official 
clients could have been written in a way that would support the needs of 
testing the REST API but it would have been a lot more work and I'm not 
sure any one even considered doing that.  On a related note, we don't 
have any tests in Tempest that make sure the official clients are not 
changing behavior. We have a start on cli tests but IMO the python 
library is even more important.

Resource management
There was a discussion last year about whether resource management 
should be in the client or in the test class. I will try to dig it up 
from the old launchpad qa list if I can. We ended up managing this in 
the test classes. This is a complicated issue. If you have a proposal 
for how to do it differently the best thing would be to put up a WIP 
patch with the essence of the base class structure and flow you are 
proposing for both client and test classes.

  -David


On 07/01/2013 11:57 AM, Martina Kollarova wrote:
> So, ignore the response code checking.
>
> How about the removal of XML vs JSON redundancy?
>
> How about resource management in the clients? (for example, keep track 
> of all created VMs, provide cleanup method that would remove them all).
>
> Martina
>
> On 07/01/2013 05:28 PM, Sean Dague wrote:
>> The point of things in the api/ directory is to test the API, which is
>> why we have those clients, and those tests. It's actually a core
>> purpose of the test suite. It's also a core purpose to be explicit
>> about return codes, so those have to be exposed to the tests. That's
>> part of the thing we are testing, for the API portion of the test
>> tree. It's not a side effect, it's actually the whole purpose of the
>> tests. It's double entry book keeping for our API, yes its effort, but
>> you can't have a stable API unless you do it.
>>
>> Anyway, we've been around in circles about this a few times now in the
>> last 2 months. Specific cleanups are welcomed, and will be reviewed.
>> So please be specific and not general in this space. Gutting API
>> testing is not welcomed or appropriate.
>>
>> Scenario/ is designed for tests that you want to use the official
>> clients. It's totally ok to add more complicated tests over there
>> which aren't API tests.
>>
>> Lastly, I'm actually out on vacation for the week, so expect
>> substantial, if not complete ignoring of threads here until I get
>> back.
>>
>> On Mon, Jul 1, 2013 at 11:07 AM, Daryl Walleck
>> <daryl.walleck at rackspace.com> wrote:
>>> There are certainly ways to accomplish this. The way that I've found 
>>> to be the most easy to maintain is to implement 
>>> serializers/deserializers rather that implement two separate 
>>> clients. It greatly simplifies the client logic and makes a clean 
>>> break between client and serialization logic. You can see an example 
>>> of that here:
>>>
>>> https://github.com/stackforge/cloudcafe/blob/master/cloudcafe/compute/servers_api/models/requests.py 
>>>
>>>
>>> Daryl
>>> ________________________________________
>>> From: Martina Kollarova [mkollaro at redhat.com]
>>> Sent: Monday, July 01, 2013 6:19 AM
>>> To: All Things QA.
>>> Subject: [openstack-qa] Refactoring Tempest service clients
>>>
>>> Problem: There is a lot of redundancy in the code around the XML and
>>> JSON API formats and API versions. The client classes are very 
>>> low-level
>>> and hard to use.
>>>
>>> There should be a single general class for each service that would
>>> contain as much common code as possible. Then there would be derived
>>> classes for JSON, XML and different API versions that would only
>>> overwrite the differences. Ideally, there could be derived class that
>>> would also handle the CLI.
>>> How to do this with the least amount of redundancy?
>>>
>>> This way, the tests would be written only once and most of them run for
>>> each API version/format. Then there would be tests for only a specific
>>> API version, when new API features would be tested.
>>>
>>> The general classes should also be more high-level. They could
>>> themselves manage the resources that were created with them and be able
>>> to clean them up. The classes would check the response codes themselves
>>> and throw exceptions, instead of having
>>> "assertEqual(resp['status'], '200')"
>>> everywhere in the test code.
>>>
>>>
>>> Thoughts?
>>>
>>> Martina Kollarova
>>> -- 
>>> irc: mkollaro
>>>
>>> _______________________________________________
>>> openstack-qa mailing list
>>> openstack-qa at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>
>>> _______________________________________________
>>> openstack-qa mailing list
>>> openstack-qa at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>
>>
>>
>