Open Stack

On Sun, Dec 1, 2013 at 7:04 PM, John Dickinson <me at not.mn> wrote:

> Just to add to the story, Swift uses "X-Trans-Id" and generates it in the
> outer-most "catch_errors" middleware.
>
> Swift's catch errors middleware is responsible for ensuring that the
> transaction id exists on each request, and that all errors previously
> uncaught, anywhere in the pipeline, are caught and logged. If there is not
> a common way to do this, yet, I submit it as a great template for solving
> this problem. It's simple, scalable, and well-tested (ie tests and running
> in prod for years).
>
>
> https://github.com/openstack/swift/blob/master/swift/common/middleware/catch_errors.py
>
> Leaving aside error handling and only focusing on the transaction id (or
> request id) generation, since OpenStack services are exposed to untrusted
> clients, how would you propose communicating the appropriate transaction id
> to a different service? I can see great benefit to having a glance
> transaction ID carry through to Swift requests (and so on), but how should
> the transaction id be communicated? It's not sensitive info, but I can
> imagine a pretty big problem when trying to track down errors if a client
> application decides to set eg the X-Set-Transaction-Id header on every
> request to the same thing.
>

-1 to cross service request IDs, for the reasons John mentions above.


>
> Thanks for bringing this up, and I'd welcome a patch in Swift that would
> use a common library to generate the transaction id, if it were installed.
> I can see that there would be huge advantage to operators to trace requests
> through multiple systems.
>
> Another option would be for each system that calls an another OpenStack
> system to expect and log the transaction ID for the request that was given.
> This would be looser coupling and be more forgiving for a heterogeneous
> cluster. Eg when Glance makes a call to Swift, Glance cloud log the
> transaction id that Swift used (from the Swift response). Likewise, when
> Swift makes a call to Keystone, Swift could log the Keystone transaction
> id. This wouldn't result in a single transaction id across all systems, but
> it would provide markers so an admin could trace the request.
>

There was a session on this at the summit, and although the notes are a
little scarce this was the conclusion we came up with.  Every time a cross
service call is made, we will log and send a notification for ceilometer to
consume, with the request-ids of both request ids.  One of the benefits of
this approach is that we can easily generate a tree of all the API calls
that are made (and clearly show when multiple calls are made to the same
service), something that just a cross service request id would have trouble
with.

https://etherpad.openstack.org/p/icehouse-summit-qa-gate-debugability


With that in mind I think having a standard x-openstack-request-id makes
things a little more uniform, and means that adding new services doesn't
require new logic to handle new request ids.



> --John
>
>
>
>
> On Dec 1, 2013, at 5:48 PM, Maru Newby <marun at redhat.com> wrote:
>
> >
> > On Nov 30, 2013, at 1:00 AM, Sean Dague <sean at dague.net> wrote:
> >
> >> On 11/29/2013 10:33 AM, Jay Pipes wrote:
> >>> On 11/28/2013 07:45 AM, Akihiro Motoki wrote:
> >>>> Hi,
> >>>>
> >>>> I am working on adding request-id to API response in Neutron.
> >>>> After I checked what header is used in other projects
> >>>> header name varies project by project.
> >>>> It seems there is no consensus what header is recommended
> >>>> and it is better to have some consensus.
> >>>>
> >>>> nova:     x-compute-request-id
> >>>> cinder:   x-compute-request-id
> >>>> glance:   x-openstack-request-id
> >>>> neutron:  x-network-request-id  (under review)
> >>>>
> >>>> request-id is assigned and used inside of each project now,
> >>>> so x-<service>-request-id looks good. On the other hand,
> >>>> if we have a plan to enhance request-id across projects,
> >>>> x-openstack-request-id looks better.
> >>>
> >>> My vote is for:
> >>>
> >>> x-openstack-request-id
> >>>
> >>> With an implementation of "create a request UUID if none exists yet" in
> >>> some standardized WSGI middleware...
> >>
> >> Agreed. I don't think I see any value in having these have different
> >> service names, having just x-openstack-request-id across all the
> >> services seems a far better idea, and come back through and fix nova and
> >> cinder to be that as well.
> >
> > +1
> >
> > An openstack request id should be service agnostic to allow tracking of
> a request across many services (e.g. a call to nova to boot a VM should
> generate a request id that is provided to other services in requests to
> provision said VM).  All services would ideally share a facility for
> generating new request ids and for securely accepting request ids from
> other services.
> >
> >
> > m.
> >
> >>
> >>      -Sean
> >>
> >> --
> >> Sean Dague
> >> http://dague.net
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131202/ea701246/attachment.html>