Open Stack

Nadya,

We had it set verbose = True in the conf file but it did not reveal this error.

On Fri, Mar 4, 2016 at 4:50 AM, Nadya Shakhat <nprivalova at mirantis.com> wrote:
> Tom,
>
> Thanks you very much for this! ceilometer.conf file contains "verbose"
> option. Perhaps, switching it to True would have shown any errors too. I'll
> check this behaviour one more time.
>
> Thanks,
> Nadya
>
> On Thu, Mar 3, 2016 at 9:29 PM, Tom Walsh <expresswebsys at gmail.com> wrote:
>>
>> In case anybody runs into this issue in the future, we have determined
>> what the issue is and how to correct it.
>>
>> First the ceilometer-agent-notification process seems to be
>> suppressing the exceptions that were being thrown from the daemon.
>> This is why the logs didn't seem to shine any light on the issue. The
>> logs were missing a critical exception message. So I had to run the
>> agent manually with the --verbose switch:
>>
>> /usr/bin/python /usr/bin/ceilometer-agent-notification --verbose
>> --config-file=/etc/ceilometer/ceilometer.conf
>>
>> Which then yielded this exception:
>>
>> Traceback (most recent call last):
>>   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/hub.py",
>> line 457, in fire_timers
>>     timer()
>>   File "/usr/local/lib/python2.7/dist-packages/eventlet/hubs/timer.py",
>> line 58, in __call__
>>     cb(*args, **kw)
>>   File "/usr/local/lib/python2.7/dist-packages/eventlet/greenthread.py",
>> line 214, in main
>>     result = function(*args, **kwargs)
>>   File "/usr/local/lib/python2.7/dist-packages/oslo_service/service.py",
>> line 671, in run_service
>>     service.start()
>>   File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
>> line 174, in start
>>     self.event_pipe_manager)
>>   File "/usr/lib/python2.7/dist-packages/ceilometer/notification.py",
>> line 223, in _configure_main_queue_listeners
>>     for new_tar in handler.get_targets(cfg.CONF):
>>   File
>> "/usr/lib/python2.7/dist-packages/ceilometer/ipmi/notifications/ironic.py",
>> line 87, in get_targets
>>     for topic in conf.notification_topics]
>>   File "/usr/local/lib/python2.7/dist-packages/oslo_config/cfg.py",
>> line 2062, in __getattr__
>>     raise NoSuchOptError(name)
>> NoSuchOptError: no such option in group DEFAULT: notification_topics
>>
>> A little quick googling resulted in this patchset:
>>
>> https://review.openstack.org/#/c/255296/
>>
>> After applying this patchset to the current ceilometer installation we
>> are seeing the notifications from the message bus being processed by
>> the ceilometer agent and then passed to the ceilometer collector.
>>
>> My guess is that the underlying changes to oslo.messaging have now
>> broken the config parsing in ceilometer.
>>
>> It appears that this patchset is missing from the current Ubuntu
>> cloudarchive ceilometer install (1:5.0.1-0ubuntu1~cloud0)
>>
>> I hope that helps somebody... I spent enough time on it.
>>
>> Tom Walsh
>> ExpressHosting.net
>
>