[Openstack] [ceilometer] Troubleshooting Ceilometer
Tom Walsh
expresswebsys at gmail.com
Sat Mar 5 17:57:08 UTC 2016
Nadya,
We had it set verbose = True in the conf file but it did not reveal this error.
On Fri, Mar 4, 2016 at 4:50 AM, Nadya Shakhat <nprivalova at mirantis.com> wrote:
> Tom,
>
> Thanks you very much for this! ceilometer.conf file contains "verbose"
> option. Perhaps, switching it to True would have shown any errors too. I'll
> check this behaviour one more time.
>
> Thanks,
> Nadya
>
> On Thu, Mar 3, 2016 at 9:29 PM, Tom Walsh <expresswebsys at gmail.com> wrote:
>>
>> In case anybody runs into this issue in the future, we have determined
>> what the issue is and how to correct it.
>>
>> First the ceilometer-agent-notification process seems to be
>> suppressing the exceptions that were being thrown from the daemon.
>> This is why the logs didn't seem to shine any light on the issue. The
>> logs were missing a critical exception message. So I had to run the
>> agent manually with the --verbose switch:
>>
>> /usr/bin/python /usr/bin/ceilometer-agent-notification --verbose
>> --config-file=/etc/ceilometer/ceilometer.conf
>>
>> Which then yielded this exception:
>>
>> Traceback (most recent call last):
>> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py",
>> line 457, in fire_timers
>> timer()
>> File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py",
>> line 58, in __call__
>> cb(*args, **kw)
>> File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py",
>> line 214, in main
>> result = function(*args, **kwargs)
>> File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py",
>> line 671, in run_service
>> service.start()
>> File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
>> line 174, in start
>> self.event_pipe_manager)
>> File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
>> line 223, in _configure_main_queue_listeners
>> for new_tar in handler.get_targets(cfg.CONF):
>> File
>> "/usr/lib/python2.7/dist-packages/ceilometer/ipmi/notifications/ironic.py",
>> line 87, in get_targets
>> for topic in conf.notification_topics]
>> File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py",
>> line 2062, in __getattr__
>> raise NoSuchOptError(name)
>> NoSuchOptError: no such option in group DEFAULT: notification_topics
>>
>> A little quick googling resulted in this patchset:
>>
>> https://review.openstack.org/#/c/255296/
>>
>> After applying this patchset to the current ceilometer installation we
>> are seeing the notifications from the message bus being processed by
>> the ceilometer agent and then passed to the ceilometer collector.
>>
>> My guess is that the underlying changes to oslo.messaging have now
>> broken the config parsing in ceilometer.
>>
>> It appears that this patchset is missing from the current Ubuntu
>> cloudarchive ceilometer install (1:5.0.1-0ubuntu1~cloud0)
>>
>> I hope that helps somebody... I spent enough time on it.
>>
>> Tom Walsh
>> ExpressHosting.net
>
>
More information about the Openstack
mailing list