[Openstack] [ceilometer] Troubleshooting Ceilometer
Tom Walsh
expresswebsys at gmail.com
Thu Mar 3 18:29:36 UTC 2016
In case anybody runs into this issue in the future, we have determined
what the issue is and how to correct it.
First the ceilometer-agent-notification process seems to be
suppressing the exceptions that were being thrown from the daemon.
This is why the logs didn't seem to shine any light on the issue. The
logs were missing a critical exception message. So I had to run the
agent manually with the --verbose switch:
/usr/bin/python /usr/bin/ceilometer-agent-notification --verbose
--config-file=/etc/ceilometer/ceilometer.conf
Which then yielded this exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py",
line 457, in fire_timers
timer()
File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py",
line 58, in __call__
cb(*args, **kw)
File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py",
line 214, in main
result = function(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py",
line 671, in run_service
service.start()
File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
line 174, in start
self.event_pipe_manager)
File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
line 223, in _configure_main_queue_listeners
for new_tar in handler.get_targets(cfg.CONF):
File "/usr/lib/python2.7/dist-packages/ceilometer/ipmi/notifications/ironic.py",
line 87, in get_targets
for topic in conf.notification_topics]
File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py",
line 2062, in __getattr__
raise NoSuchOptError(name)
NoSuchOptError: no such option in group DEFAULT: notification_topics
A little quick googling resulted in this patchset:
https://review.openstack.org/#/c/255296/
After applying this patchset to the current ceilometer installation we
are seeing the notifications from the message bus being processed by
the ceilometer agent and then passed to the ceilometer collector.
My guess is that the underlying changes to oslo.messaging have now
broken the config parsing in ceilometer.
It appears that this patchset is missing from the current Ubuntu
cloudarchive ceilometer install (1:5.0.1-0ubuntu1~cloud0)
I hope that helps somebody... I spent enough time on it.
Tom Walsh
ExpressHosting.net
More information about the Openstack
mailing list