[openstack-dev] [nova] Change from Mitaka: Expected UNIX signal to generate Guru Meditation (error) Reports

Matt Riedemann mriedem at linux.vnet.ibm.com
Fri Oct 30 15:06:01 UTC 2015



On 10/29/2015 5:22 PM, Sean Dague wrote:
> Right, the crux of the problem is the move by the library from SIGUSR1
> -> SIGUSR2 with no overlap and deprecation period breaks the ability to
> have any tooling use this without atomically updating that tooling, and
> this library, in all environments, all at the same time.
>
> We need the SIGUSR1 handler added back in, deprecated, and not removed
> for a couple cycles.
>
> 	-Sean
>
> On 10/30/2015 06:46 AM, Davanum Srinivas wrote:
>> Matt,
>>
>> Yes, Sean already opened a critical bug
>> - https://bugs.launchpad.net/oslo.reports/+bug/1510740
>>
>> Please note that this change was added to oslo.reports *after* the
>> liberty release in support of Mitaka. So i am not sure why we need to
>> add it to liberty release notes. Also this is a consequence of NOT
>> having version caps which was a decision a while ago as well.
>>
>> -- DIms
>>
>> On Fri, Oct 30, 2015 at 4:47 AM, Matt Riedemann
>> <mriedem at linux.vnet.ibm.com <mailto:mriedem at linux.vnet.ibm.com>> wrote:
>>
>>
>>
>>      On 10/21/2015 7:43 AM, Kashyap Chamarthy wrote:
>>
>>          Background
>>          ----------
>>
>>          Oslo Guru Meditation (error) Reports (GMR)[*] are a useful debugging
>>          mechanism that allows one to capture the current state of a Nova
>>          process/executable (e.g. `nova-compute`, `nova-api`, etc).
>>
>>          The way to generate the error report is to supply the 'User-defined
>>          signal', SIGUSR1, when killing a Nova process.  E.g.
>>
>>               $ kill -USR1 `pgrep nova-compute`
>>
>>          which results in GMR being printed to your standard error ('stderr')
>>          stream, wherever it ends up being redirected to (e.g. to a
>>          corresponding
>>          Nova process-specific log file, otherwise, on systemd-enabled
>>          systems,
>>          to its journal).
>>
>>
>>          Change in Mitaka (and above)
>>          ----------------------------
>>
>>           From the upcoming Mitaka release onwards, the default expected UNIX
>>          signal to generate GMR has been changed[1] from USR1 to USR2
>>          (another
>>          User-defined singal), because the USR1 is reserved by Apache
>>          'mod_wsgi'
>>          for its own purpose.
>>
>>          So, to generate GMR, from Mitaka release:
>>
>>               $ kill -USR2 `pgrep nova-compute`
>>
>>          A corresponding Nova documentation change[2] has been submitted to
>>          reflect this new reality.
>>
>>
>>          [1] https://review.openstack.org/#/c/223133/ --
>>          guru_meditation_report:
>>               Use SIGUSR2 instead of SIGUSR1
>>          [2] https://review.openstack.org/#/c/227779/ -- doc: gmr: Update
>>               instructions to generate GMR error reports
>>
>>
>>          [*] References
>>          --------------
>>
>>          Related reading:
>>
>>          - http://docs.openstack.org/developer/nova/gmr.html
>>          - http://docs.openstack.org/developer/oslo.reports/usage.html
>>          - https://wiki.openstack.org/wiki/GuruMeditationReport
>>          -
>>          https://www.berrange.com/posts/2015/02/19/nova-and-its-use-of-olso-incubator-guru-meditation-reports/
>>
>>
>>      Looks like this broke some tooling in the gate job runs where gmr's
>>      are created at the end of the service logs when the services exit.
>>      Here is a mitaka change with grenade on the liberty side with the
>>      gmr at the end:
>>
>>      http://logs.openstack.org/97/227897/13/check/gate-grenade-dsvm/27723a9/logs/old/screen-n-cpu.txt.gz
>>
>>      And on the new side it's gone:
>>
>>      http://logs.openstack.org/97/227897/13/check/gate-grenade-dsvm/27723a9/logs/new/screen-n-cpu.txt.gz
>>
>>      So obviously an upgrade impact, I'm hoping we get this into the
>>      liberty release notes as something to change when people move up to
>>      oslo.reports 1.6.0.
>>
>>      We should also get the gate tooling fixed around this, I'm not sure
>>      where that was configured/triggered though, sdague probably knows.
>>
>>      --
>>
>>      Thanks,
>>
>>      Matt Riedemann
>>
>>
>>
>>      __________________________________________________________________________
>>      OpenStack Development Mailing List (not for usage questions)
>>      Unsubscribe:
>>      OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>      <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>>      http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> --
>> Davanum Srinivas :: https://twitter.com/dims
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Is there going to be a revert of 
https://review.openstack.org/#/c/223133/ ?  This has already baked into 
a couple of oslo.reports releases:

Tag Name
0.6.0
0.7.0

And we've updated docs in at least nova for it.

-- 

Thanks,

Matt Riedemann




More information about the OpenStack-dev mailing list