Open Stack

Cool,

As a favor, could you propose updates to the in tree documentation for 
tempest,

tempest/doc/source/index.rst

That summarizes it? I agree that a lot of Tempest rationale came up in 
this thread, and it would be awesome if we captured that into our docs 
so that in future we can point folks there. :)

	-Sean

On 07/02/2013 11:03 AM, Martina Kollarova wrote:
> Thanks, this whole thread made me realize the reasons behind a lot of
> decisions around Tempest.
>
> Resource management in clients doesn't make much sense when they are
> redundant as they are now, it would just create more code duplication.
>
> Martina
>
> On Tue 02 Jul 2013 04:01:04 PM CEST, David Kranz wrote:
>> There are a bunch of separable issues here so I'll give my 2 cents on
>> each of them.
>>
>> XML vs JSON
>> The current xml/json thing is a mess but it is really the fault of the
>> APIs. There was a discussion on the dev list about whether, going
>> forward, we should support xml at all. I'm not sure if there was
>> agreement, but if we do support xml we should really do it in a way
>> that supports auto-serialization back and forth. It might not be "good
>> xml" to xml lovers but it would save us a maintenance nightmare. My
>> view is that in Tempest we should leave this alone for the existing
>> apis. It is what it is and we should focus on getting more tests. For
>> new apis, hopefully we could use the serialize/deserialize code from
>> the server project to the extent that we need to test both.
>>
>> Tempest rest client vs official client
>> There are valid arguments on both sides of this issue. If we say that
>> OpenStack apis are defined by the REST API, we have to test it. Users
>> might use that API directly, or use a library from some other
>> language. The existing official python clients are not enough. The
>> official clients could have been written in a way that would support
>> the needs of testing the REST API but it would have been a lot more
>> work and I'm not sure any one even considered doing that.  On a
>> related note, we don't have any tests in Tempest that make sure the
>> official clients are not changing behavior. We have a start on cli
>> tests but IMO the python library is even more important.
>>
>> Resource management
>> There was a discussion last year about whether resource management
>> should be in the client or in the test class. I will try to dig it up
>> from the old launchpad qa list if I can. We ended up managing this in
>> the test classes. This is a complicated issue. If you have a proposal
>> for how to do it differently the best thing would be to put up a WIP
>> patch with the essence of the base class structure and flow you are
>> proposing for both client and test classes.
>>
>>  -David
>>
>>
>> On 07/01/2013 11:57 AM, Martina Kollarova wrote:
>>> So, ignore the response code checking.
>>>
>>> How about the removal of XML vs JSON redundancy?
>>>
>>> How about resource management in the clients? (for example, keep
>>> track of all created VMs, provide cleanup method that would remove
>>> them all).
>>>
>>> Martina
>>>
>>> On 07/01/2013 05:28 PM, Sean Dague wrote:
>>>> The point of things in the api/ directory is to test the API, which is
>>>> why we have those clients, and those tests. It's actually a core
>>>> purpose of the test suite. It's also a core purpose to be explicit
>>>> about return codes, so those have to be exposed to the tests. That's
>>>> part of the thing we are testing, for the API portion of the test
>>>> tree. It's not a side effect, it's actually the whole purpose of the
>>>> tests. It's double entry book keeping for our API, yes its effort, but
>>>> you can't have a stable API unless you do it.
>>>>
>>>> Anyway, we've been around in circles about this a few times now in the
>>>> last 2 months. Specific cleanups are welcomed, and will be reviewed.
>>>> So please be specific and not general in this space. Gutting API
>>>> testing is not welcomed or appropriate.
>>>>
>>>> Scenario/ is designed for tests that you want to use the official
>>>> clients. It's totally ok to add more complicated tests over there
>>>> which aren't API tests.
>>>>
>>>> Lastly, I'm actually out on vacation for the week, so expect
>>>> substantial, if not complete ignoring of threads here until I get
>>>> back.
>>>>
>>>> On Mon, Jul 1, 2013 at 11:07 AM, Daryl Walleck
>>>> <daryl.walleck at rackspace.com> wrote:
>>>>> There are certainly ways to accomplish this. The way that I've
>>>>> found to be the most easy to maintain is to implement
>>>>> serializers/deserializers rather that implement two separate
>>>>> clients. It greatly simplifies the client logic and makes a clean
>>>>> break between client and serialization logic. You can see an
>>>>> example of that here:
>>>>>
>>>>> https://github.com/stackforge/cloudcafe/blob/master/cloudcafe/compute/servers_api/models/requests.py
>>>>>
>>>>>
>>>>>
>>>>> Daryl
>>>>> ________________________________________
>>>>> From: Martina Kollarova [mkollaro at redhat.com]
>>>>> Sent: Monday, July 01, 2013 6:19 AM
>>>>> To: All Things QA.
>>>>> Subject: [openstack-qa] Refactoring Tempest service clients
>>>>>
>>>>> Problem: There is a lot of redundancy in the code around the XML and
>>>>> JSON API formats and API versions. The client classes are very
>>>>> low-level
>>>>> and hard to use.
>>>>>
>>>>> There should be a single general class for each service that would
>>>>> contain as much common code as possible. Then there would be derived
>>>>> classes for JSON, XML and different API versions that would only
>>>>> overwrite the differences. Ideally, there could be derived class that
>>>>> would also handle the CLI.
>>>>> How to do this with the least amount of redundancy?
>>>>>
>>>>> This way, the tests would be written only once and most of them run
>>>>> for
>>>>> each API version/format. Then there would be tests for only a specific
>>>>> API version, when new API features would be tested.
>>>>>
>>>>> The general classes should also be more high-level. They could
>>>>> themselves manage the resources that were created with them and be
>>>>> able
>>>>> to clean them up. The classes would check the response codes
>>>>> themselves
>>>>> and throw exceptions, instead of having
>>>>> "assertEqual(resp['status'], '200')"
>>>>> everywhere in the test code.
>>>>>
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> Martina Kollarova
>>>>> --
>>>>> irc: mkollaro
>>>>>
>>>>> _______________________________________________
>>>>> openstack-qa mailing list
>>>>> openstack-qa at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>>>
>>>>> _______________________________________________
>>>>> openstack-qa mailing list
>>>>> openstack-qa at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>>>>
>>>>
>>>>
>>>
>>
>>
>> _______________________________________________
>> openstack-qa mailing list
>> openstack-qa at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa
>
> --
> irc: mkollaro
>
> _______________________________________________
> openstack-qa mailing list
> openstack-qa at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa


-- 
Sean Dague
http://dague.net