[openstack-dev] [nova] Change from Mitaka: Expected UNIX signal to generate Guru Meditation (error) Reports

Sean Dague sean at dague.net
Thu Oct 29 22:22:55 UTC 2015


Right, the crux of the problem is the move by the library from SIGUSR1
-> SIGUSR2 with no overlap and deprecation period breaks the ability to
have any tooling use this without atomically updating that tooling, and
this library, in all environments, all at the same time.

We need the SIGUSR1 handler added back in, deprecated, and not removed
for a couple cycles.

	-Sean

On 10/30/2015 06:46 AM, Davanum Srinivas wrote:
> Matt,
> 
> Yes, Sean already opened a critical bug
> - https://bugs.launchpad.net/oslo.reports/+bug/1510740
> 
> Please note that this change was added to oslo.reports *after* the
> liberty release in support of Mitaka. So i am not sure why we need to
> add it to liberty release notes. Also this is a consequence of NOT
> having version caps which was a decision a while ago as well.
> 
> -- DIms
> 
> On Fri, Oct 30, 2015 at 4:47 AM, Matt Riedemann
> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
> 
> 
> 
>     On 10/21/2015 7:43 AM, Kashyap Chamarthy wrote:
> 
>         Background
>         ----------
> 
>         Oslo Guru Meditation (error) Reports (GMR)[*] are a useful debugging
>         mechanism that allows one to capture the current state of a Nova
>         process/executable (e.g. `nova-compute`, `nova-api`, etc).
> 
>         The way to generate the error report is to supply the 'User-defined
>         signal', SIGUSR1, when killing a Nova process.  E.g.
> 
>              $ kill -USR1 `pgrep nova-compute`
> 
>         which results in GMR being printed to your standard error ('stderr')
>         stream, wherever it ends up being redirected to (e.g. to a
>         corresponding
>         Nova process-specific log file, otherwise, on systemd-enabled
>         systems,
>         to its journal).
> 
> 
>         Change in Mitaka (and above)
>         ----------------------------
> 
>          From the upcoming Mitaka release onwards, the default expected UNIX
>         signal to generate GMR has been changed[1] from USR1 to USR2
>         (another
>         User-defined singal), because the USR1 is reserved by Apache
>         'mod_wsgi'
>         for its own purpose.
> 
>         So, to generate GMR, from Mitaka release:
> 
>              $ kill -USR2 `pgrep nova-compute`
> 
>         A corresponding Nova documentation change[2] has been submitted to
>         reflect this new reality.
> 
> 
>         [1] https://review.openstack.org/#/c/223133/ --
>         guru_meditation_report:
>              Use SIGUSR2 instead of SIGUSR1
>         [2] https://review.openstack.org/#/c/227779/ -- doc: gmr: Update
>              instructions to generate GMR error reports
> 
> 
>         [*] References
>         --------------
> 
>         Related reading:
> 
>         - http://docs.openstack.org/developer/nova/gmr.html
>         - http://docs.openstack.org/developer/oslo.reports/usage.html
>         - https://wiki.openstack.org/wiki/GuruMeditationReport
>         -
>         https://www.berrange.com/posts/2015/02/19/nova-and-its-use-of-olso-incubator-guru-meditation-reports/
> 
> 
>     Looks like this broke some tooling in the gate job runs where gmr's
>     are created at the end of the service logs when the services exit.
>     Here is a mitaka change with grenade on the liberty side with the
>     gmr at the end:
> 
>     http://logs.openstack.org/97/227897/13/check/gate-grenade-dsvm/27723a9/logs/old/screen-n-cpu.txt.gz
> 
>     And on the new side it's gone:
> 
>     http://logs.openstack.org/97/227897/13/check/gate-grenade-dsvm/27723a9/logs/new/screen-n-cpu.txt.gz
> 
>     So obviously an upgrade impact, I'm hoping we get this into the
>     liberty release notes as something to change when people move up to
>     oslo.reports 1.6.0.
> 
>     We should also get the gate tooling fixed around this, I'm not sure
>     where that was configured/triggered though, sdague probably knows.
> 
>     -- 
> 
>     Thanks,
> 
>     Matt Riedemann
> 
> 
> 
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> -- 
> Davanum Srinivas :: https://twitter.com/dims
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


-- 
Sean Dague
http://dague.net

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151030/bfbc1664/attachment.pgp>


More information about the OpenStack-dev mailing list