Open Stack

Thanks, this whole thread made me realize the reasons behind a lot of 
decisions around Tempest.

Resource management in clients doesn't make much sense when they are 
redundant as they are now, it would just create more code duplication.

Martina

On Tue 02 Jul 2013 04:01:04 PM CEST, David Kranz wrote:
> There are a bunch of separable issues here so I'll give my 2 cents on
> each of them.
>
> XML vs JSON
> The current xml/json thing is a mess but it is really the fault of the
> APIs. There was a discussion on the dev list about whether, going
> forward, we should support xml at all. I'm not sure if there was
> agreement, but if we do support xml we should really do it in a way
> that supports auto-serialization back and forth. It might not be "good
> xml" to xml lovers but it would save us a maintenance nightmare. My
> view is that in Tempest we should leave this alone for the existing
> apis. It is what it is and we should focus on getting more tests. For
> new apis, hopefully we could use the serialize/deserialize code from
> the server project to the extent that we need to test both.
>
> Tempest rest client vs official client
> There are valid arguments on both sides of this issue. If we say that
> OpenStack apis are defined by the REST API, we have to test it. Users
> might use that API directly, or use a library from some other
> language. The existing official python clients are not enough. The
> official clients could have been written in a way that would support
> the needs of testing the REST API but it would have been a lot more
> work and I'm not sure any one even considered doing that.  On a
> related note, we don't have any tests in Tempest that make sure the
> official clients are not changing behavior. We have a start on cli
> tests but IMO the python library is even more important.
>
> Resource management
> There was a discussion last year about whether resource management
> should be in the client or in the test class. I will try to dig it up
> from the old launchpad qa list if I can. We ended up managing this in
> the test classes. This is a complicated issue. If you have a proposal
> for how to do it differently the best thing would be to put up a WIP
> patch with the essence of the base class structure and flow you are
> proposing for both client and test classes.
>
>  -David
>
>
> On 07/01/2013 11:57 AM, Martina Kollarova wrote:
>> So, ignore the response code checking.
>>
>> How about the removal of XML vs JSON redundancy?
>>
>> How about resource management in the clients? (for example, keep
>> track of all created VMs, provide cleanup method that would remove
>> them all).
>>
>> Martina
>>
>> On 07/01/2013 05:28 PM, Sean Dague wrote:
>>> The point of things in the api/ directory is to test the API, which is
>>> why we have those clients, and those tests. It's actually a core
>>> purpose of the test suite. It's also a core purpose to be explicit
>>> about return codes, so those have to be exposed to the tests. That's
>>> part of the thing we are testing, for the API portion of the test
>>> tree. It's not a side effect, it's actually the whole purpose of the
>>> tests. It's double entry book keeping for our API, yes its effort, but
>>> you can't have a stable API unless you do it.
>>>
>>> Anyway, we've been around in circles about this a few times now in the
>>> last 2 months. Specific cleanups are welcomed, and will be reviewed.
>>> So please be specific and not general in this space. Gutting API
>>> testing is not welcomed or appropriate.
>>>
>>> Scenario/ is designed for tests that you want to use the official
>>> clients. It's totally ok to add more complicated tests over there
>>> which aren't API tests.
>>>
>>> Lastly, I'm actually out on vacation for the week, so expect
>>> substantial, if not complete ignoring of threads here until I get
>>> back.
>>>
>>> On Mon, Jul 1, 2013 at 11:07 AM, Daryl Walleck
>>> <daryl.walleck at rackspace.com> wrote:
>>>> There are certainly ways to accomplish this. The way that I've
>>>> found to be the most easy to maintain is to implement
>>>> serializers/deserializers rather that implement two separate
>>>> clients. It greatly simplifies the client logic and makes a clean
>>>> break between client and serialization logic. You can see an
>>>> example of that here:
>>>>
>>>> https://github.com/stackforge/cloudcafe/blob/master/cloudcafe/compute/servers_api/models/requests.py
>>>>
>>>>
>>>> Daryl
>>>> ________________________________________
>>>> From: Martina Kollarova [mkollaro at redhat.com]
>>>> Sent: Monday, July 01, 2013 6:19 AM
>>>> To: All Things QA.
>>>> Subject: [openstack-qa] Refactoring Tempest service clients
>>>>
>>>> Problem: There is a lot of redundancy in the code around the XML and
>>>> JSON API formats and API versions. The client classes are very
>>>> low-level
>>>> and hard to use.
>>>>
>>>> There should be a single general class for each service that would
>>>> contain as much common code as possible. Then there would be derived
>>>> classes for JSON, XML and different API versions that would only
>>>> overwrite the differences. Ideally, there could be derived class that
>>>> would also handle the CLI.
>>>> How to do this with the least amount of redundancy?
>>>>
>>>> This way, the tests would be written only once and most of them run
>>>> for
>>>> each API version/format. Then there would be tests for only a specific
>>>> API version, when new API features would be tested.
>>>>
>>>> The general classes should also be more high-level. They could
>>>> themselves manage the resources that were created with them and be
>>>> able
>>>> to clean them up. The classes would check the response codes
>>>> themselves
>>>> and throw exceptions, instead of having
>>>> "assertEqual(resp['status'], '200')"
>>>> everywhere in the test code.
>>>>
>>>>
>>>> Thoughts?
>>>>
>>>> Martina Kollarova
>>>> --
>>>> irc: mkollaro
>>>>
>>>> _______________________________________________
>>>> openstack-qa mailing list
>>>> openstack-qa at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>>
>>>> _______________________________________________
>>>> openstack-qa mailing list
>>>> openstack-qa at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>
>>>
>>>
>>
>
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa

--
irc: mkollaro