Open Stack

-----Original Message-----
From: Kekane, Abhishek [mailto:Abhishek.Kekane at nttdata.com] 
Sent: 10 December 2015 12:56
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [python-glanceclient] Return request-id to caller



-----Original Message-----
From: stuart.mclaren at hp.com [mailto:stuart.mclaren at hp.com]
Sent: 09 December 2015 23:54
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [python-glanceclient] Return request-id to caller

> Excerpts from Flavio Percoco's message of 2015-12-09 09:09:10 -0430:
>> On 09/12/15 11:33 +0000, Kekane, Abhishek wrote:
>>> Hi Devs,
>>>
>>>
>>>
>>> We are adding support for returning ?x-openstack-request-id?  to the 
>>> caller as per the design proposed in cross-project specs:
>>>
>>> http://specs.openstack.org/openstack/openstack-specs/specs/
>>> return-request-id.html
>>>
>>>
>>>
>>> Problem Description:
>>>
>>> Cannot add a new property of list type to the warlock.model object.
>>>
>>>
>>>
>>> How is a model object created:
>>>
>>> Let?s take an example of glanceclient.api.v2.images.get() call [1]:
>>>
>>>
>>>
>>> Here after getting the response we call model() method. This model() 
>>> does the job of creating a warlock.model object(essentially a dict) 
>>> based on the schema given as argument (image schema retrieved from 
>>> glance in this case). Inside
>>> model() the raw() method simply return the image schema as JSON 
>>> object. The advantage of this warlock.model object over a simple 
>>> dict is that it validates any changes to object based on the rules specified in the reference schema.
>>> The keys of this  model object are available as object properties to 
>>> the caller.
>>>
>>>
>>>
>>> Underlying reason:
>>>
>>> The schema for different sub APIs is returned a bit differently. For 
>>> images, metadef APIs glance.schema.Schema.raw() is used which 
>>> returns a schema containing ?additionalProperties?: {?type?: 
>>> ?string?}. Whereas for members and tasks APIs 
>>> glance.schema.Schema.minimal() is used to return schema object which does not contain ?additionalProperties?.
>>>
>>>
>>>
>>> So we can add extra properties of any type to the model object 
>>> returned from members or tasks API but for images and metadef APIs 
>>> we can only add properties which can be of type string. Also for the 
>>> latter case we depend on the glance configuration to allow additional properties.
>>>
>>>
>>>
>>> As per our analysis we have come up with two approaches for 
>>> resolving this
>>> issue:
>>>
>>>
>>>
>>> Approach #1:  Inject request_ids property in the warlock model 
>>> object in glance client
>>>
>>> Here we do the following:
>>>
>>> 1. Inject the ?request_ids? as additional property into the model 
>>> object (returned from model())
>>>
>>> 2. Return the model object which now contains request_ids property
>>>
>>>
>>>
>>> Limitations:
>>>
>>> 1. Because the glance schemas for images and metadef only allows 
>>> additional properties of type string, so even though natural type of 
>>> request_ids should be list we have to make it as a comma separated 
>>> ?string? of request ids as a compromise.
>>>
>>> 2. Lot of extra code is needed to wrap objects returned from the 
>>> client API so that the caller can get request ids. For example we 
>>> need to write wrapper classes for dict, list, str, tuple, generator.
>>>
>>> 3. Not a good design as we are adding a property which should 
>>> actually be a base property but added as additional property as a compromise.
>>>
>>> 4. There is a dependency on glance whether to allow 
>>> custom/additional properties or not. [2]
>>>
>>>
>>>
>>> Approach #2:  Add ?request_ids? property to all schema definitions 
>>> in glance
>>>
>>>
>>>
>>> Here we add  ?request_ids? property as follows to the various APIs (schema):
>>>
>>>
>>>
>>> ?request_ids?: {
>>>
>>>  "type": "array",
>>>
>>>  "items": {
>>>
>>>    "type": "string"
>>>
>>>  }
>>>
>>> }
>>>
>>>
>>>
>>> Doing this will make changes in glance client very simple as 
>>> compared to approach#1.
>>>
>>> This also looks a better design as it will be consistent.
>>>
>>> We simply need to modify the request_ids property in  various API 
>>> calls for example glanceclient.v2.images.get().
>>>
>>
>> Hey Abhishek,
>>
>> thanks for working on this.
>>
>> To be honest, I'm a bit confused on why the request_id needs to be an 
>> attribute of the image. Isn't it passed as a header? Does it have to 
>> be an attribute so we can "print" it?
>
> The requirement they're trying to meet is to make the request id 
> available to the user of the client library [1]. The user typically 
> doesn't have access to the headers, so the request id needs to be part 
> of the payload returned from each method. In other clients

> Will this work if the payload is image data?

I think yes, let me test this as well

> that work with simple data types, they've subclassed dict, list, etc. 
> to add the extra property. This adds the request id to the return 
> value without making a breaking change to the API of the client 
> library.
>
> Abhishek, would it be possible to add the request id information to 
> the schema data in glance client, before giving it to warlock?
> I don't know whether warlock asks for the schema or what form that 
> data takes (dictionary, JSON blob, etc.). If it's a dictionary visible 
> to the client code it would be straightforward to add data to it.
>
Yes, it is possible to add request-id to schema before giving it to warlock, but since it's a contract IMO it doesn't look good to modify schema at client side.

> Failing that, is it possible to change warlock to allow extra 
> properties with arbitrary types to be added to objects? Because 
> validating inputs to the constructor is all well and good, but 
> breaking the ability to add data to an object is a bit un-pythonic.
>
IMO there is no point to change warlock as it is a 3rd party module.

> If we end up having to change the schema definitions in the Glance 
> API, that also means changing those API calls to add the request id to 
> the return value, right?
>
IMO there will be no changing API calls as request-id will be injected in glanceclient and it doesn't have any impact in glance.
Also we can make this request-id as non-mandatory if required.

> Doug
>
> [1] 
> http://specs.openstack.org/openstack/openstack-specs/specs/return-requ
> est-id.html
>
>>
>> As it is presented in your email, I'd probably go with option #2 but 
>> I'm curious to know the answer to my question.
>>
IMO approach #2 is better and it will be consistent to all api's to have request-id as an attribute in schema so that it will be consistent.

So we will make change in glance to add request-id as base property in schema and inject request-id in glanceclient from response headers.

The code change will be look like,

iff --git a/glance/api/v2/images.py b/glance/api/v2/images.py index bb7949c..2a760a7 100644
--- a/glance/api/v2/images.py
+++ b/glance/api/v2/images.py
@@ -807,6 +807,10 @@ class ResponseSerializer(wsgi.JSONResponseSerializer):

 def get_base_properties():
     return {
+        'request_ids': {
+            'type': 'array',
+            'items': {'type': 'string'}
+        },
         'id': {
             'type': 'string',
             'description': _('An identifier for the image'),

Changes in glanceclient to assign request-id from response headers

openstack at openstack-136:~/python-glanceclient$ git diff diff --git a/glanceclient/v2/images.py b/glanceclient/v2/images.py index 4fdcea2..65b0d6c 100644
--- a/glanceclient/v2/images.py
+++ b/glanceclient/v2/images.py
@@ -182,6 +182,7 @@ class Controller(object):
         # NOTE(bcwaldon): remove 'self' for now until we have an elegant
         # way to pass it into the model constructor without conflict
         body.pop('self', None)
+        body['request_ids'] = [resp.headers['x-openstack-request-id']]
         return self.model(**body)

     def data(self, image_id, do_checksum=True):

Output:

>>> import glanceclient
>>> glance = glanceclient.Client('2',
>>> endpoint='http://10.69.4.136:9292/',
>>> token='16038d125b804eef805c7020bbebc769')
>>> get = glance.images.get('a00b6125-94d9-43a8-a497-839cf25a8fdd')
>>> get
{u'status': u'active', u'tags': [], u'container_format': u'aki', u'min_ram': 0, u'updated_at': u'2015-11-18T13:04:18Z', u'visibility': u'public', 'request_ids': ['req-68926f34-4434-45dc-822c-c4eb94506c63'], u'owner': u'd1ee7fd5dcc341c3973f19f790238e63', u'file': u'/v2/images/a00b6125-94d9-43a8-a497-839cf25a8fdd/file', u'min_disk': 0, u'virtual_size': None, u'id': u'a00b6125-94d9-43a8-a497-839cf25a8fdd', u'size': 4979632, u'name': u'cirros-0.3.4-x86_64-uec-kernel', u'checksum': u'8a40c862b5735975d82605c1dd395796', u'created_at': u'2015-11-18T13:04:18Z', u'disk_format': u'aki', u'protected': False, u'schema': u'/v2/schemas/image'}
>>> get.request_ids
['req-68926f34-4434-45dc-822c-c4eb94506c63']
>>>

Hi Flavio, Glance Cores,

To implement this requirement, I need make changes in glance schema to add request_id attribute as a property.
Do I need to submit a glance-specs for this change or just blueprint is fine as cross-project specs is already approved.

Please suggest.

Thank You,

Abhishek

>> Cheers,
>> Flavio
>>
>>>
>>>
>>> Please let us know which approach is better or any suggestions for the same.
>>>
>>>
>>>
>>> [1] 
>>> https://github.com/openstack/python-glanceclient/blob/master/glancec
>>> lient/
>>> v2/images.py#L179
>>>
>>> [2] 
>>> https://github.com/openstack/glance/blob/master/glance/api/v2/images
>>> .py#
>>> L944
>>>
>>>
>>> ____________________________________________________________________
>>> __
>>> Disclaimer: This email and any attachments are sent in strictest 
>>> confidence for the sole use of the addressee and may contain legally 
>>> privileged, confidential, and proprietary data. If you are not the 
>>> intended recipient, please advise the sender by replying promptly to 
>>> this email and then delete and destroy this email and any 
>>> attachments without any further use, copying or forwarding.
>>
>>> ____________________________________________________________________
>>> ______ OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: 
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>

______________________________________________________________________
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.