[glance] Global Request ID issues in Glance

Markus Hentsch markus.hentsch at secustack.com
Tue Jul 7 12:49:11 UTC 2020


Hi Abhishek,

thanks for having a look! I've filed corresponding bug reports:

Glance client:
https://bugs.launchpad.net/python-glanceclient/+bug/1886650

Glance API:
https://bugs.launchpad.net/glance/+bug/1886657


Best regards,

Markus


Abhishek Kekane wrote:
> Hi Markus,
>
> Thank you for detailed analysis.
> Both cases you pointed out are valid bugs. Could you please report
> this to launchpad?
>
> Thanks & Best Regards,
>
> Abhishek Kekane
>
>
> On Fri, Jun 26, 2020 at 6:33 PM Markus Hentsch
> <markus.hentsch at secustack.com <mailto:markus.hentsch at secustack.com>>
> wrote:
>
>     Hello everyone,
>
>     while I was experimenting with the Global Request ID functionality of
>     OpenStack [1], I identified two issues in Glance related to this
>     topic.
>     I have written my findings below and would appreciate it if you could
>     take a look and confirm whether those are intended behaviors or indeed
>     issues with the implementation.
>
>     In case of the latter please advice me which bug tracker to report
>     them
>     to.
>
>
>     1. The Glance client does not correctly forward the global ID
>
>     When the SessionClient class is used, the global_request_id is removed
>     from kwargs in the constructor using pop() [2]. Directly after this,
>     the parent constructor is called using super(), which in this case is
>     Adapter from the keystoneauth1 library. Therein the global_request_id
>     is set again [3] but since it has been removed from the kwargs, it
>     defaults to None as specified in the Adapter's __init__() header.
>     Thus,
>     the global_request_id passed to the SessionClient constructor never
>     actually makes it to the Glance API. This is in contrast to the
>     HTTPClient class, where get() is used instead of pop() [4].
>
>     This can be reproduced simply by creating a server in Nova from an
>     image in Glance, which will attempt to create the Glance client
>     instance using the global_request_id [5]. Passing the
>     "X-Openstack-Request-Id" header during the initial API call for the
>     server creation, makes it visible in Nova (using a suitable
>     "logging_context_format_string" setting) but it's not visible in
>     Glance. Using a Python debugger shows Glance generating a new local ID
>     instead.
>
>
>     2. Glance interprets global ID as local one for Oslo Context objects
>
>     While observing the Glance log file, I observed Glance always logging
>     the global_request_id instead of a local one if it is available.
>
>     Using "%(global_request_id)s" within
>     "logging_context_format_string"[6]
>     in the glance-api.conf will always print "None" in the logs whereas
>     "%(request_id)s" will either be an ID generated by Glance if no global
>     ID is available or the received global ID.
>
>     Culprit seems to be the context middleware of Glance where the global
>     ID in form of the "X-Openstack-Request-Id" header is parsed from the
>     request and passed as "request_id" instead of "global_request_id" to
>     the "glance.context.RequestContext.from_environ()" call [7].
>
>     This is in contrast to other services such as Nova or Neutron where
>     the two variables actually print the values according to their name
>     (request_id always being the local one, whereas global_request_id is
>     the global one or None).
>
>
>     [1]
>     https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.html
>     [2]
>     https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L355
>     [3]
>     https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d3141ad4fb36a339/keystoneauth1/adapter.py#L166
>     [4]
>     https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L162
>     [5]
>     https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca69225/nova/image/glance.py#L78
>     [6]
>     https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-variables
>     [7]
>     https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f56451086850cd/glance/api/middleware/context.py#L201
>
>
>     Thanks!
>
>     Markus
>
>     -- 
>     Markus Hentsch
>     Team Leader
>
>     secustack GmbH - Digital Sovereignty in the Cloud
>     https://www.secustack.com
>     Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden
>     District Court Dresden, Register Number: HRB 38890
>
>

-- 
Markus Hentsch
Team Leader

secustack GmbH - Digital Sovereignty in the Cloud
https://www.secustack.com
Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden
District Court Dresden, Register Number: HRB 38890

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200707/e1b629fd/attachment.html>


More information about the openstack-discuss mailing list