Open Stack

-----Original Message-----
From: Doug Hellmann [mailto:doug at doughellmann.com] 
Sent: 09 December 2015 19:28
To: openstack-dev
Subject: Re: [openstack-dev] [python-glanceclient] Return request-id to caller

Excerpts from Flavio Percoco's message of 2015-12-09 09:09:10 -0430:
> On 09/12/15 11:33 +0000, Kekane, Abhishek wrote:
> >Hi Devs,
> >
> > 
> >
> >We are adding support for returning ‘x-openstack-request-id’  to the 
> >caller as per the design proposed in cross-project specs:
> >
> >http://specs.openstack.org/openstack/openstack-specs/specs/
> >return-request-id.html
> >
> > 
> >
> >Problem Description:
> >
> >Cannot add a new property of list type to the warlock.model object.
> >
> > 
> >
> >How is a model object created:
> >
> >Let’s take an example of glanceclient.api.v2.images.get() call [1]:
> >
> > 
> >
> >Here after getting the response we call model() method. This model() 
> >does the job of creating a warlock.model object(essentially a dict) 
> >based on the schema given as argument (image schema retrieved from 
> >glance in this case). Inside
> >model() the raw() method simply return the image schema as JSON 
> >object. The advantage of this warlock.model object over a simple dict 
> >is that it validates any changes to object based on the rules specified in the reference schema.
> >The keys of this  model object are available as object properties to 
> >the caller.
> >
> > 
> >
> >Underlying reason:
> >
> >The schema for different sub APIs is returned a bit differently. For 
> >images, metadef APIs glance.schema.Schema.raw() is used which returns 
> >a schema containing “additionalProperties”: {“type”: “string”}. 
> >Whereas for members and tasks APIs glance.schema.Schema.minimal() is 
> >used to return schema object which does not contain “additionalProperties”.
> >
> > 
> >
> >So we can add extra properties of any type to the model object 
> >returned from members or tasks API but for images and metadef APIs we 
> >can only add properties which can be of type string. Also for the 
> >latter case we depend on the glance configuration to allow additional properties.
> >
> > 
> >
> >As per our analysis we have come up with two approaches for resolving 
> >this
> >issue:
> >
> > 
> >
> >Approach #1:  Inject request_ids property in the warlock model object 
> >in glance client
> >
> >Here we do the following:
> >
> >1. Inject the ‘request_ids’ as additional property into the model 
> >object (returned from model())
> >
> >2. Return the model object which now contains request_ids property
> >
> > 
> >
> >Limitations:
> >
> >1. Because the glance schemas for images and metadef only allows 
> >additional properties of type string, so even though natural type of 
> >request_ids should be list we have to make it as a comma separated 
> >‘string’ of request ids as a compromise.
> >
> >2. Lot of extra code is needed to wrap objects returned from the 
> >client API so that the caller can get request ids. For example we 
> >need to write wrapper classes for dict, list, str, tuple, generator.
> >
> >3. Not a good design as we are adding a property which should 
> >actually be a base property but added as additional property as a compromise.
> >
> >4. There is a dependency on glance whether to allow custom/additional 
> >properties or not. [2]
> >
> > 
> >
> >Approach #2:  Add ‘request_ids’ property to all schema definitions in 
> >glance
> >
> > 
> >
> >Here we add  ‘request_ids’ property as follows to the various APIs (schema):
> >
> > 
> >
> >“request_ids”: {
> >
> >  "type": "array",
> >
> >  "items": {
> >
> >    "type": "string"
> >
> >  }
> >
> >}
> >
> > 
> >
> >Doing this will make changes in glance client very simple as compared 
> >to approach#1.
> >
> >This also looks a better design as it will be consistent.
> >
> >We simply need to modify the request_ids property in  various API 
> >calls for example glanceclient.v2.images.get().
> >
> 
> Hey Abhishek,
> 
> thanks for working on this.
> 
> To be honest, I'm a bit confused on why the request_id needs to be an 
> attribute of the image. Isn't it passed as a header? Does it have to 
> be an attribute so we can "print" it?

The requirement they're trying to meet is to make the request id available to the user of the client library [1]. The user typically doesn't have access to the headers, so the request id needs to be part of the payload returned from each method. In other clients that work with simple data types, they've subclassed dict, list, etc. to add the extra property. This adds the request id to the return value without making a breaking change to the API of the client library.

Abhishek, would it be possible to add the request id information to the schema data in glance client, before giving it to warlock?
I don't know whether warlock asks for the schema or what form that data takes (dictionary, JSON blob, etc.). If it's a dictionary visible to the client code it would be straightforward to add data to it.

Yes, it is possible to add request-id to schema before giving it to warlock, but since it’s a contract IMO it doesn't look good to modify schema at client side.

Failing that, is it possible to change warlock to allow extra properties with arbitrary types to be added to objects? Because validating inputs to the constructor is all well and good, but breaking the ability to add data to an object is a bit un-pythonic.
IMO there is no point to change warlock as it is a 3rd party module.

If we end up having to change the schema definitions in the Glance API, that also means changing those API calls to add the request id to the return value, right?
IMO there will be no changing API calls as request-id will be injected in glanceclient and it doesn't have any impact in glance.
Also we can make this request-id as non-mandatory if required.

Doug

[1] http://specs.openstack.org/openstack/openstack-specs/specs/return-request-id.html

> 
> As it is presented in your email, I'd probably go with option #2 but 
> I'm curious to know the answer to my question.

IMO approach #2 is better and it will be consistent to all api's to have request-id as an attribute in schema so that it will be consistent.

So we will make change in glance to add request-id as base property in schema and inject request-id in glanceclient from response headers.

The code change will be look like,

iff --git a/glance/api/v2/images.py b/glance/api/v2/images.py
index bb7949c..2a760a7 100644
--- a/glance/api/v2/images.py
+++ b/glance/api/v2/images.py
@@ -807,6 +807,10 @@ class ResponseSerializer(wsgi.JSONResponseSerializer):

 def get_base_properties():
     return {
+        'request_ids': {
+            'type': 'array',
+            'items': {'type': 'string'}
+        },
         'id': {
             'type': 'string',
             'description': _('An identifier for the image'),

Changes in glanceclient to assign request-id from response headers

openstack at openstack-136:~/python-glanceclient$ git diff
diff --git a/glanceclient/v2/images.py b/glanceclient/v2/images.py
index 4fdcea2..65b0d6c 100644
--- a/glanceclient/v2/images.py
+++ b/glanceclient/v2/images.py
@@ -182,6 +182,7 @@ class Controller(object):
         # NOTE(bcwaldon): remove 'self' for now until we have an elegant
         # way to pass it into the model constructor without conflict
         body.pop('self', None)
+        body['request_ids'] = [resp.headers['x-openstack-request-id']]
         return self.model(**body)

     def data(self, image_id, do_checksum=True):

Output:

>>> import glanceclient
>>> glance = glanceclient.Client('2', endpoint='http://10.69.4.136:9292/', token='16038d125b804eef805c7020bbebc769')
>>> get = glance.images.get('a00b6125-94d9-43a8-a497-839cf25a8fdd')
>>> get
{u'status': u'active', u'tags': [], u'container_format': u'aki', u'min_ram': 0, u'updated_at': u'2015-11-18T13:04:18Z', u'visibility': u'public', 'request_ids': ['req-68926f34-4434-45dc-822c-c4eb94506c63'], u'owner': u'd1ee7fd5dcc341c3973f19f790238e63', u'file': u'/v2/images/a00b6125-94d9-43a8-a497-839cf25a8fdd/file', u'min_disk': 0, u'virtual_size': None, u'id': u'a00b6125-94d9-43a8-a497-839cf25a8fdd', u'size': 4979632, u'name': u'cirros-0.3.4-x86_64-uec-kernel', u'checksum': u'8a40c862b5735975d82605c1dd395796', u'created_at': u'2015-11-18T13:04:18Z', u'disk_format': u'aki', u'protected': False, u'schema': u'/v2/schemas/image'}
>>> get.request_ids
['req-68926f34-4434-45dc-822c-c4eb94506c63']
>>>

Please suggest.

Thank You,

Abhishek

> 
> Cheers,
> Flavio
> 
> > 
> >
> >Please let us know which approach is better or any suggestions for the same.
> >
> > 
> >
> >[1] 
> >https://github.com/openstack/python-glanceclient/blob/master/glancecl
> >ient/
> >v2/images.py#L179
> >
> >[2] 
> >https://github.com/openstack/glance/blob/master/glance/api/v2/images.
> >py#
> >L944
> >
> >
> >_____________________________________________________________________
> >_
> >Disclaimer: This email and any attachments are sent in strictest 
> >confidence for the sole use of the addressee and may contain legally 
> >privileged, confidential, and proprietary data. If you are not the 
> >intended recipient, please advise the sender by replying promptly to 
> >this email and then delete and destroy this email and any attachments 
> >without any further use, copying or forwarding.
> 
> >_____________________________________________________________________
> >_____ OpenStack Development Mailing List (not for usage questions)
> >Unsubscribe: 
> >OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

______________________________________________________________________
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.