[glance] Global Request ID issues in Glance

Abhishek Kekane akekane at redhat.com
Mon Jul 6 13:39:24 UTC 2020


Hi Markus,

Thank you for detailed analysis.
Both cases you pointed out are valid bugs. Could you please report this to
launchpad?

Thanks & Best Regards,

Abhishek Kekane


On Fri, Jun 26, 2020 at 6:33 PM Markus Hentsch <markus.hentsch at secustack.com>
wrote:

> Hello everyone,
>
> while I was experimenting with the Global Request ID functionality of
> OpenStack [1], I identified two issues in Glance related to this topic.
> I have written my findings below and would appreciate it if you could
> take a look and confirm whether those are intended behaviors or indeed
> issues with the implementation.
>
> In case of the latter please advice me which bug tracker to report them
> to.
>
>
> 1. The Glance client does not correctly forward the global ID
>
> When the SessionClient class is used, the global_request_id is removed
> from kwargs in the constructor using pop() [2]. Directly after this,
> the parent constructor is called using super(), which in this case is
> Adapter from the keystoneauth1 library. Therein the global_request_id
> is set again [3] but since it has been removed from the kwargs, it
> defaults to None as specified in the Adapter's __init__() header. Thus,
> the global_request_id passed to the SessionClient constructor never
> actually makes it to the Glance API. This is in contrast to the
> HTTPClient class, where get() is used instead of pop() [4].
>
> This can be reproduced simply by creating a server in Nova from an
> image in Glance, which will attempt to create the Glance client
> instance using the global_request_id [5]. Passing the
> "X-Openstack-Request-Id" header during the initial API call for the
> server creation, makes it visible in Nova (using a suitable
> "logging_context_format_string" setting) but it's not visible in
> Glance. Using a Python debugger shows Glance generating a new local ID
> instead.
>
>
> 2. Glance interprets global ID as local one for Oslo Context objects
>
> While observing the Glance log file, I observed Glance always logging
> the global_request_id instead of a local one if it is available.
>
> Using "%(global_request_id)s" within "logging_context_format_string"[6]
> in the glance-api.conf will always print "None" in the logs whereas
> "%(request_id)s" will either be an ID generated by Glance if no global
> ID is available or the received global ID.
>
> Culprit seems to be the context middleware of Glance where the global
> ID in form of the "X-Openstack-Request-Id" header is parsed from the
> request and passed as "request_id" instead of "global_request_id" to
> the "glance.context.RequestContext.from_environ()" call [7].
>
> This is in contrast to other services such as Nova or Neutron where
> the two variables actually print the values according to their name
> (request_id always being the local one, whereas global_request_id is
> the global one or None).
>
>
> [1]
>
> https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.html
> [2]
>
> https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L355
> [3]
>
> https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d3141ad4fb36a339/keystoneauth1/adapter.py#L166
> [4]
>
> https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L162
> [5]
>
> https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca69225/nova/image/glance.py#L78
> [6]
>
> https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-variables
> [7]
>
> https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f56451086850cd/glance/api/middleware/context.py#L201
>
>
> Thanks!
>
> Markus
>
> --
> Markus Hentsch
> Team Leader
>
> secustack GmbH - Digital Sovereignty in the Cloud
> https://www.secustack.com
> Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden
> District Court Dresden, Register Number: HRB 38890
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200706/32bf4794/attachment.html>


More information about the openstack-discuss mailing list