[openstack-dev] [nova] [glance] [cinder] [neutron] [keystone] [heat] [telemetry] - RFC cross project request id tracking

Doug Hellmann doug at doughellmann.com
Mon May 15 14:35:03 UTC 2017


Excerpts from Sean Dague's message of 2017-05-15 10:01:20 -0400:
> On 05/15/2017 09:35 AM, Doug Hellmann wrote:
> > Excerpts from Sean Dague's message of 2017-05-14 07:04:03 -0400:
> >> One of the things that came up in a logging Forum session is how much 
> >> effort operators are having to put into reconstructing flows for things 
> >> like server boot when they go wrong, as every time we jump a service 
> >> barrier the request-id is reset to something new. The back and forth 
> >> between Nova / Neutron and Nova / Glance would be definitely well served 
> >> by this. Especially if this is something that's easy to query in elastic 
> >> search.
> >>
> >> The last time this came up, some people were concerned that trusting 
> >> request-id on the wire was concerning to them because it's coming from 
> >> random users. We're going to assume that's still a concern by some. 
> >> However, since the last time that came up, we've introduced the concept 
> >> of "service users", which are a set of higher priv services that we are 
> >> using to wrap user requests between services so that long running 
> >> request chains (like image snapshot). We trust these service users 
> >> enough to keep on trucking even after the user token has expired for 
> >> this long run operations. We could use this same trust path for 
> >> request-id chaining.
> >>
> >> So, the basic idea is, services will optionally take an inbound 
> >> X-OpenStack-Request-ID which will be strongly validated to the format 
> >> (req-$uuid). They will continue to always generate one as well. When the 
> > 
> > Do all of our services use that format for request ID? I thought Heat
> > used something added on to a UUID, or at least longer than a UUID?
> 
> Don't know, now is a good time to speak up.
> http://logs.openstack.org/85/464585/1/check/gate-heat-dsvm-functional-orig-mysql-lbaasv2-ubuntu-xenial/e1bca9e/logs/screen-h-eng.txt.gz#_2017-05-15_10_08_10_617
> seems to indicate that it's using the format everyone else is using.
> 
> Swift does things a bit differently with suffixes, but they aren't using
> the common middleware.
> 
> I've done code look throughs on nova/glance/cinder/neutron/keystone, but
> beyond that folks will need to speak up as to where this might break
> down. At worst failing validation just means you end up in the old
> (current) behavior.
> 
>     -Sean
> 

OK. I vaguely remembered something from the early days of ceilometer
trying to collect those notifications, but maybe I'm confusing it with
something else. I've added [heat] to the subject line to get that team's
attention for input.

Doug



More information about the OpenStack-dev mailing list