Open Stack

Hello everyone,

while I was experimenting with the Global Request ID functionality of
OpenStack [1], I identified two issues in Glance related to this topic.
I have written my findings below and would appreciate it if you could
take a look and confirm whether those are intended behaviors or indeed
issues with the implementation.

In case of the latter please advice me which bug tracker to report them
to.


1. The Glance client does not correctly forward the global ID

When the SessionClient class is used, the global_request_id is removed
from kwargs in the constructor using pop() [2]. Directly after this,
the parent constructor is called using super(), which in this case is
Adapter from the keystoneauth1 library. Therein the global_request_id
is set again [3] but since it has been removed from the kwargs, it
defaults to None as specified in the Adapter's __init__() header. Thus,
the global_request_id passed to the SessionClient constructor never
actually makes it to the Glance API. This is in contrast to the
HTTPClient class, where get() is used instead of pop() [4].

This can be reproduced simply by creating a server in Nova from an
image in Glance, which will attempt to create the Glance client
instance using the global_request_id [5]. Passing the
"X-Openstack-Request-Id" header during the initial API call for the
server creation, makes it visible in Nova (using a suitable
"logging_context_format_string" setting) but it's not visible in
Glance. Using a Python debugger shows Glance generating a new local ID
instead.


2. Glance interprets global ID as local one for Oslo Context objects

While observing the Glance log file, I observed Glance always logging
the global_request_id instead of a local one if it is available.

Using "%(global_request_id)s" within "logging_context_format_string"[6]
in the glance-api.conf will always print "None" in the logs whereas
"%(request_id)s" will either be an ID generated by Glance if no global
ID is available or the received global ID.

Culprit seems to be the context middleware of Glance where the global
ID in form of the "X-Openstack-Request-Id" header is parsed from the
request and passed as "request_id" instead of "global_request_id" to
the "glance.context.RequestContext.from_environ()" call [7].

This is in contrast to other services such as Nova or Neutron where
the two variables actually print the values according to their name
(request_id always being the local one, whereas global_request_id is
the global one or None).


[1]
https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.html
[2]
https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L355
[3]
https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d3141ad4fb36a339/keystoneauth1/adapter.py#L166
[4]
https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L162
[5]
https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca69225/nova/image/glance.py#L78
[6]
https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-variables
[7]
https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f56451086850cd/glance/api/middleware/context.py#L201


Thanks!

Markus

-- 
Markus Hentsch
Team Leader

secustack GmbH - Digital Sovereignty in the Cloud
https://www.secustack.com
Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden
District Court Dresden, Register Number: HRB 38890