[openstack-dev] OS tracing??

Joshua Harlow harlowja at yahoo-inc.com
Tue Sep 4 19:29:59 UTC 2012


Thanks much,

Almost forgot about tach, it seems like it can be hooked into arbitrary
functions, which is great. It'd be cool if that type of functionality was
included with say nova, and it could be remotely enabled/disabled as
needed (say a weird production issue u want to find more info about, so u
send a special command that says start monitoring this function, or even
better, integrate it into eventlet so that it can start reporting
automatically on 'hot' functions).

Is tach monkey patching the functions that it is asked to instrument?

-Josh

On 9/4/12 10:46 AM, "Sandy Walsh" <sandy.walsh at rackspace.com> wrote:

>We've been using Tach to orchestrate Openstack services and report to
>statsd/graphite. https://github.com/ohthree/tach ... works great
>
>I've been trying to land this Inflight Service branch to measure RPC and
>greenlet overhead 
>https://review.openstack.org/#/c/11179/
>BP: https://blueprints.launchpad.net/nova/+spec/monitoring-service
>
>Hope it helps,
>-S
>
>
>
>
>From: Joshua Harlow [harlowja at yahoo-inc.com]
>Sent: Tuesday, September 04, 2012 2:35 PM
>To: OpenStack Development Mailing List
>Cc: openstack-dev
>Subject: [openstack-dev] OS tracing??
>
>Has anyone had any luck with trying out some tracing/coverage with the
>openstack projects to see where the bottlenecks are (outside of test
>coverage)?
>
>I was thinking about possible ways to do this (there seems to be a lot of
>different libraries that might help) but was wondering if anyone else has
>figured out the best one to use yet.
>
>Ideally it should have the following properties (in my mind):
>
>Non-intrusive (shouldn't require sprinkling of timing/trace logic all
>over)Works with eventlet/greenlet (eventlet is going to switch things in
>and out, so that has to be taken account of)Probably does this via
>sampling (?)Writes out some standard format (valgrind like?) for
>analysisŠCan be turned on and off remotely (nice to have, it'd be cool to
>have an API/entrypoint/Š that says enable tracing which can be used on a
>live system, that system will become slower but it'd be neat)
>
>This could be some special 'admin' entry point (restricted to certain
>users of course) that could also do stuff like 'reload-configs' or
>'enable-tracing' or 'adjust-log-level' or similar administrative actions
>that would be useful during those crazy debug
> sessions (think a simple admin telnet entrypoint to view stats, similar
>to what memcache/redis provide via there 'stats' commandsŠ)
>
>
>Anyone have any ideas on this :-)
>
>
>
>
>
>-Josh 
>
>
>
>




More information about the OpenStack-dev mailing list