<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Dec 1, 2013 at 7:04 PM, John Dickinson <span dir="ltr"><<a href="mailto:me@not.mn" target="_blank">me@not.mn</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Just to add to the story, Swift uses "X-Trans-Id" and generates it in the outer-most "catch_errors" middleware.<br>
<br>
Swift's catch errors middleware is responsible for ensuring that the transaction id exists on each request, and that all errors previously uncaught, anywhere in the pipeline, are caught and logged. If there is not a common way to do this, yet, I submit it as a great template for solving this problem. It's simple, scalable, and well-tested (ie tests and running in prod for years).<br>
<br>
<a href="https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py" target="_blank">https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py</a><br>
<br>
Leaving aside error handling and only focusing on the transaction id (or request id) generation, since OpenStack services are exposed to untrusted clients, how would you propose communicating the appropriate transaction id to a different service? I can see great benefit to having a glance transaction ID carry through to Swift requests (and so on), but how should the transaction id be communicated? It's not sensitive info, but I can imagine a pretty big problem when trying to track down errors if a client application decides to set eg the X-Set-Transaction-Id header on every request to the same thing.<br>
</blockquote><div><br></div><div>-1 to cross service request IDs, for the reasons John mentions above.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
Thanks for bringing this up, and I'd welcome a patch in Swift that would use a common library to generate the transaction id, if it were installed. I can see that there would be huge advantage to operators to trace requests through multiple systems.<br>
<br>
Another option would be for each system that calls an another OpenStack system to expect and log the transaction ID for the request that was given. This would be looser coupling and be more forgiving for a heterogeneous cluster. Eg when Glance makes a call to Swift, Glance cloud log the transaction id that Swift used (from the Swift response). Likewise, when Swift makes a call to Keystone, Swift could log the Keystone transaction id. This wouldn't result in a single transaction id across all systems, but it would provide markers so an admin could trace the request.<br>
</blockquote><div><br></div><div>There was a session on this at the summit, and although the notes are a little scarce this was the conclusion we came up with. Every time a cross service call is made, we will log and send a notification for ceilometer to consume, with the request-ids of both request ids. One of the benefits of this approach is that we can easily generate a tree of all the API calls that are made (and clearly show when multiple calls are made to the same service), something that just a cross service request id would have trouble with.</div>
<div><br></div><div><a href="https://etherpad.openstack.org/p/icehouse-summit-qa-gate-debugability">https://etherpad.openstack.org/p/icehouse-summit-qa-gate-debugability</a> </div><div><br></div><div><br></div><div>With that in mind I think having a standard x-openstack-request-id makes things a little more uniform, and means that adding new services doesn't require new logic to handle new request ids.</div>
<div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span class=""><font color="#888888"><br>
--John<br>
</font></span><div class=""><div class="h5"><br>
<br>
<br>
<br>
On Dec 1, 2013, at 5:48 PM, Maru Newby <<a href="mailto:marun@redhat.com">marun@redhat.com</a>> wrote:<br>
<br>
><br>
> On Nov 30, 2013, at 1:00 AM, Sean Dague <<a href="mailto:sean@dague.net">sean@dague.net</a>> wrote:<br>
><br>
>> On 11/29/2013 10:33 AM, Jay Pipes wrote:<br>
>>> On 11/28/2013 07:45 AM, Akihiro Motoki wrote:<br>
>>>> Hi,<br>
>>>><br>
>>>> I am working on adding request-id to API response in Neutron.<br>
>>>> After I checked what header is used in other projects<br>
>>>> header name varies project by project.<br>
>>>> It seems there is no consensus what header is recommended<br>
>>>> and it is better to have some consensus.<br>
>>>><br>
>>>> nova: x-compute-request-id<br>
>>>> cinder: x-compute-request-id<br>
>>>> glance: x-openstack-request-id<br>
>>>> neutron: x-network-request-id (under review)<br>
>>>><br>
>>>> request-id is assigned and used inside of each project now,<br>
>>>> so x-<service>-request-id looks good. On the other hand,<br>
>>>> if we have a plan to enhance request-id across projects,<br>
>>>> x-openstack-request-id looks better.<br>
>>><br>
>>> My vote is for:<br>
>>><br>
>>> x-openstack-request-id<br>
>>><br>
>>> With an implementation of "create a request UUID if none exists yet" in<br>
>>> some standardized WSGI middleware...<br>
>><br>
>> Agreed. I don't think I see any value in having these have different<br>
>> service names, having just x-openstack-request-id across all the<br>
>> services seems a far better idea, and come back through and fix nova and<br>
>> cinder to be that as well.<br>
><br>
> +1<br>
><br>
> An openstack request id should be service agnostic to allow tracking of a request across many services (e.g. a call to nova to boot a VM should generate a request id that is provided to other services in requests to provision said VM). All services would ideally share a facility for generating new request ids and for securely accepting request ids from other services.<br>
><br>
><br>
> m.<br>
><br>
>><br>
>> -Sean<br>
>><br>
>> --<br>
>> Sean Dague<br>
>> <a href="http://dague.net" target="_blank">http://dague.net</a><br>
>><br>
>> _______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>