[openstack-qa] Refactoring the rest_client and Services

Attila Fazekas afazekas at redhat.com
Wed Feb 13 11:24:18 UTC 2013


I would like JUST summarize the key ideas of a different object hierarchy, which mostly related to the RestClient usage.
The plan is not fully complete yet, but it is ready for starting a discussion.

1. Using Composition/Aggregation over inheritance

    Very frequently we see someone has to implement new type of "rest client" for testing something different.
    List of several clients which overriding the request method:
     - TokenClientXML
     - ObjectClientCustomizedHeader
     - AccountClientCustomizedHeader
    Client makes decision based on the base url:
     - https://review.openstack.org/#/c/20688/

    These are clear indicators of the current model has issues.

    Every RestClient instance does his own authentication and caches it until his lifetime. We usually using multiple RestClient instance in a test class, and rest_client's lifetime ends at the end of the fixture.
    The authentication should not be part of the base rest client, the base rest client should be as pure as possible.
    Clients using the same credentials, should use a common authentication mechanism, with single token "cache".

    The services (tempest/services), should not read and process endpoint configuration settings, it limits their usability. Some "manager" "objects" should do it in most cases.

    The services should not use any rest_client as base class.
    A service constructor (__init__) should have 2(+1) arguments:
     - A rest_client (it could be an authenticating one or not authenticating one, with the same interface
     - A base_url

2. RestClient functionality chain (filters?)

    We should have at least on conceptional level, something similar as on the server side. So we should have one "application"(servlet) a very basic rest client without any smart thing even without logging.
    We should configure an in/out filters for the requests and responses on the base rest client:
     - authentication
     - logging (deduplicating version could exist too)
     - generic response validation

   Implementation types:
   1. (Aggregation) Constructor should get filter lists
   2. (Aggregation) Constructor gets another rest_client instance
   3. Inheritance, with different constructor parameters

3. Services (tempest/services)

   - The XML and JSON common behavior should be inherited from base Service class instead from a regular RestClient class.
   - Services should be responsible for basic validation ie. status code processing, testing the appearance of MUST field  in the response.
   - If a remote object has Read, Update, Delete functionality (CRUD) it should be implemented in the same class (or module, or..)

4. Ducks - Interface

    We should give the same name to the methods which does the same function, but on different resource type:
    - deletes a remote object
    - get fresh information about the remote object
    - replace the remote object's content
    If the remote object has embedded individually manageable complex attribute (metadata), they should be implemented in the same class/module.., and should have similar interface, in all occurrence in tempest.

5. Some of the services methods should return with a specific object instead of a dict

   - The services should return with an object on creation or read action
   - The detailed lists should return with a list of objects
   The objects should know:
     - their type (MUST)
     - How to delete the remote object (absolute url, and a reference to the RestClient)
     - How to get fresh status information (read)
     - How update the remote object
     - They should be able to use multiple API (xml, json, (maybe ec2, or they should be able to convert to ec2 object, and constructable from ec2 object))
     - They should have a default switchable API beckend (xml, json)

6. Managed Resources (testresources.TestResourceManager) should embed the objects described above (5.)
     - XML, JSON test fixtures should use the appropriate interface

7. Eliminate long redundant absolute names

     Examples:
       tempest.services.compute.admin.json.quotas_client.AdminQuotasClientJSON.update_quota_set (quota 3 times, admin 2 times, json 2 times, client 2 times)
       tempest.services.volume.xml.snapshots_client.SnapshotsClientXML.create_snapshot (snapshot 3 times, xml 2 times, client 2 times)
       tempest.services.volume.xml.volumes_client.VolumesClientXML.create_volume (volume 4 times, xml 2 times, client 2 times)

     - These are preventing leveraging common interfaces (duck typing).
     - They are preventing higher level generalization (very frequently a CRUD client implementation differs only in 2-4 words)
     - Causing very long import lines in the tempest/clients.py.
     - It is not a Python style.
     - They looks dumb when you follow the import modules only "rule".


8. Compute Admin and provided users (and other provided resources)
    The provided users(resources) lifetime is longer than tempest lifetime, if we ever have any knowledge connected to these resources (user -> keypairs), should be stored in single location, even if we have multiple thread or we using them in different fixtures.
    Since we do not have any well controlled and cheap before and after all test hook point, they should be accessed through a singleton and they should have dedicated classes.

    The Compute Admin might need different roles as the identity admin, he should be created only ONCE in most cases, by using a configured role information in the tempest config (it will be new configuration attribute).


It is just the short version of one concept. 
Feel free to ask for details or criticize it.

PS.: the swift objects should contain the full entity: entity_body and entity-headers (all header not fall into other type (generic, request, response)).



More information about the openstack-qa mailing list