<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Hi Abhishek,<br>
<br>
thanks for having a look! I've filed corresponding bug reports:<br>
<br>
Glance client:<br>
<a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/python-glanceclient/+bug/1886650">https://bugs.launchpad.net/python-glanceclient/+bug/1886650</a><br>
<br>
Glance API:<br>
<a class="moz-txt-link-freetext" href="https://bugs.launchpad.net/glance/+bug/1886657">https://bugs.launchpad.net/glance/+bug/1886657</a><br>
<br>
<br>
Best regards,<br>
<br>
Markus<br>
<br>
<br>
<div class="moz-cite-prefix">Abhishek Kekane wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CALOt+STD90KxMA_XK-=kVMhUfR+q-3A=YgFPrQPxziVKsLtYqA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_default"
style="font-family:verdana,sans-serif">Hi Markus,</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Thank you for detailed
analysis.<br>
</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif">Both cases you pointed
out are valid bugs. Could you please report this to launchpad?</div>
<div class="gmail_default"
style="font-family:verdana,sans-serif"><br clear="all">
</div>
<div>
<div dir="ltr" class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">
<div>Thanks & Best Regards,<br>
<br>
</div>
Abhishek Kekane<br>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Jun 26, 2020 at 6:33
PM Markus Hentsch <<a
href="mailto:markus.hentsch@secustack.com"
moz-do-not-send="true">markus.hentsch@secustack.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello
everyone,<br>
<br>
while I was experimenting with the Global Request ID
functionality of<br>
OpenStack [1], I identified two issues in Glance related to
this topic.<br>
I have written my findings below and would appreciate it if
you could<br>
take a look and confirm whether those are intended behaviors
or indeed<br>
issues with the implementation.<br>
<br>
In case of the latter please advice me which bug tracker to
report them<br>
to.<br>
<br>
<br>
1. The Glance client does not correctly forward the global ID<br>
<br>
When the SessionClient class is used, the global_request_id is
removed<br>
from kwargs in the constructor using pop() [2]. Directly after
this,<br>
the parent constructor is called using super(), which in this
case is<br>
Adapter from the keystoneauth1 library. Therein the
global_request_id<br>
is set again [3] but since it has been removed from the
kwargs, it<br>
defaults to None as specified in the Adapter's __init__()
header. Thus,<br>
the global_request_id passed to the SessionClient constructor
never<br>
actually makes it to the Glance API. This is in contrast to
the<br>
HTTPClient class, where get() is used instead of pop() [4].<br>
<br>
This can be reproduced simply by creating a server in Nova
from an<br>
image in Glance, which will attempt to create the Glance
client<br>
instance using the global_request_id [5]. Passing the<br>
"X-Openstack-Request-Id" header during the initial API call
for the<br>
server creation, makes it visible in Nova (using a suitable<br>
"logging_context_format_string" setting) but it's not visible
in<br>
Glance. Using a Python debugger shows Glance generating a new
local ID<br>
instead.<br>
<br>
<br>
2. Glance interprets global ID as local one for Oslo Context
objects<br>
<br>
While observing the Glance log file, I observed Glance always
logging<br>
the global_request_id instead of a local one if it is
available.<br>
<br>
Using "%(global_request_id)s" within
"logging_context_format_string"[6]<br>
in the glance-api.conf will always print "None" in the logs
whereas<br>
"%(request_id)s" will either be an ID generated by Glance if
no global<br>
ID is available or the received global ID.<br>
<br>
Culprit seems to be the context middleware of Glance where the
global<br>
ID in form of the "X-Openstack-Request-Id" header is parsed
from the<br>
request and passed as "request_id" instead of
"global_request_id" to<br>
the "glance.context.RequestContext.from_environ()" call [7].<br>
<br>
This is in contrast to other services such as Nova or Neutron
where<br>
the two variables actually print the values according to their
name<br>
(request_id always being the local one, whereas
global_request_id is<br>
the global one or None).<br>
<br>
<br>
[1]<br>
<a
href="https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.html"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://specs.openstack.org/openstack/oslo-specs/specs/pike/global-req-id.html</a><br>
[2]<br>
<a
href="https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L355"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L355</a><br>
[3]<br>
<a
href="https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d3141ad4fb36a339/keystoneauth1/adapter.py#L166"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/openstack/keystoneauth/blob/dab8e1057ae8bb9a0e778fb8d3141ad4fb36a339/keystoneauth1/adapter.py#L166</a><br>
[4]<br>
<a
href="https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L162"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/openstack/python-glanceclient/blob/de178ac4382716cc93022be06b93697936e816fc/glanceclient/common/http.py#L162</a><br>
[5]<br>
<a
href="https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca69225/nova/image/glance.py#L78"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/openstack/nova/blob/1cae0cd7229207478b70275509aecd778ca69225/nova/image/glance.py#L78</a><br>
[6]<br>
<a
href="https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-variables"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://docs.openstack.org/oslo.context/2.17.0/user/usage.html#context-variables</a><br>
[7]<br>
<a
href="https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f56451086850cd/glance/api/middleware/context.py#L201"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://github.com/openstack/glance/blob/e6db0b10a703037f754007bef6f56451086850cd/glance/api/middleware/context.py#L201</a><br>
<br>
<br>
Thanks!<br>
<br>
Markus<br>
<br>
-- <br>
Markus Hentsch<br>
Team Leader<br>
<br>
secustack GmbH - Digital Sovereignty in the Cloud<br>
<a href="https://www.secustack.com" rel="noreferrer"
target="_blank" moz-do-not-send="true">https://www.secustack.com</a><br>
Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden<br>
District Court Dresden, Register Number: HRB 38890<br>
<br>
<br>
</blockquote>
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Markus Hentsch
Team Leader
secustack GmbH - Digital Sovereignty in the Cloud
<a class="moz-txt-link-freetext" href="https://www.secustack.com">https://www.secustack.com</a>
Königsbrücker Straße 96 (Gebäude 30) | 01099 Dresden
District Court Dresden, Register Number: HRB 38890</pre>
</body>
</html>