Open Stack

Hi,

If I'm interpreting this correctly, nova compute is calling get_diagnostics
on all instances, including ones currently in a shutdown state.  And then
it throws an exception, and adds an entry into the instance_faults table in
the database.

nova-compute logs this message:

2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher Traceback
(most recent call last):
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line
142, in _dispatch_and_reply
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
executor_callback))
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line
186, in _dispatch
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
executor_callback)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line
129, in _do_dispatch
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
result = func(ctxt, **new_args)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/nova/exception.py", line 89, in wrapped
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
payload)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in
__exit__
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/nova/exception.py", line 72, in wrapped
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
return f(self, context, *args, **kw)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 378, in
decorated_function
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
kwargs['instance'], e, sys.exc_info())
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 195, in
__exit__
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
six.reraise(self.type_, self.value, self.tb)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 366, in
decorated_function
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
return function(self, context, *args, **kwargs)
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher   File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4089, in
get_diagnostics
2017-07-07 16:29:46.184 23077 ERROR oslo_messaging.rpc.dispatcher
method='get_diagnostics')

2017-07-07 16:30:10.017 23077 ERROR oslo_messaging.rpc.dispatcher
InstanceInvalidState: Instance 6ab60005-ccbf-4bc2-95ac-7daf31716754 in
power_state 4. Cannot get_diagnostics while the instance is in this state.

I don't think it should be trying to gather diags on shutdown instances,
and if it did, it shouldn't just create a never-ending stream of errors.
If anyone has any info on if this might be a bug that is fixed in the
latest release, or if I can turn off this behavior, it would be appreciated.

Best,
Peter
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170707/0c7e9557/attachment.html>