[Openstack] [openstack][ceilometer] Periodic checks error on Compute node with ceilometer

Eoghan Glynn eglynn at redhat.com
Wed Nov 20 09:34:53 UTC 2013


Hi Jean-Daniel,

That doesn't look like a ceilometer issue.

Rather, IIUC it's a case of the nova-compute agent reporting an issue
from the nova-conductor service when it calls out to the conductor over
RPC to record the beginning of a task in the task log.

Interestingly, the error message seems to suggest that the nova-conductor
is using sqlite:

  DBError SQLite DateTime type only accepts Python datetime and date objects as input.

whereas the nova.conf you quote suggests that mysql is in use:

  connection = mysql://nova:password@infra-eval-03/nova

Please check the config used by your nova-conductor service, as the
same DB should be configured for the nova services across the board.

Cheers,
Eoghan  

----- Original Message -----
> ​​ Hi stackers!
> 
> When integrating ceilometer I experience periodic task errors on a compute
> node.
> I paste below my configurations, if anyone has experienced this or a clue of
> what could be the cause info would be very helpful.
> 
> The error output is:
> nova-compute.log======================================
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> Traceback (most recent call last):
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/periodic_task.py",
> line 180, in run_periodic_tasks
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> task(self, context)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 4495, in
> _instance_usage_audit
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> self.host, num_instances)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/compute/utils.py", line 372, in
> start_instance_usage_audit
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> "Instance usage audit started...")
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 279, in
> task_log_begin_task
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> task_items, message)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/conductor/rpcapi.py", line 411, in
> task_log_begin_task
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> task_items=task_items, message=message)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 85, in call
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> return self._invoke(self.proxy.call, ctxt, method, **kwargs)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 63, in _invoke
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> return cast_or_call(ctxt, msg, **self.kwargs)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py", line
> 126, in call
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> result = rpc.call(context, real_topic, msg, timeout)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/__init__.py",
> line 139, in call
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> return _get_impl().call(CONF, context, topic, msg, timeout)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/impl_kombu.py",
> line 816, in call
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> rpc_amqp.get_connection_pool(conf, Connection))
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
> 574, in call
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task rv =
> list(rv)
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
> 539, in __iter__
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task raise
> result
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> RemoteError: Remote error: DBError SQLite DateTime type only accepts Python
> datetime and date objects as input. (original cause: TypeError: SQLite
> DateTime type only accepts Python datetime and date objects as input.)
> 'INSERT INTO task_log (created_at, updated_at, deleted_at, deleted,
> task_name, state, host, period_beginning, period_ending, message,
> task_items, errors) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)'
> [{'period_ending': u'2013-11-20T06:00:00.000000', 'state': 'RUNNING',
> 'updated_at': None, 'host': u'infra-eval-06', 'task_items': 2, 'task_name':
> u'instance_usage_audit', 'period_beginning': u'2013-11-20T05:00:00.000000',
> 'message': u'Instance usage audit started...', 'deleted_at': None}]
> 2013-11-20 15:01:46.068 22480 TRACE nova.openstack.common.periodic_task
> [u'Traceback (most recent call last):\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py", line
> 461, in _process_data\n **args)\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
> line 172, in dispatch\n result = getattr(proxyobj, method)(ctxt,
> **kwargs)\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 471, in
> task_log_begin_task\n message)\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 1891, in
> task_log_begin_task\n message)\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 108, in
> wrapper\n return f(*args, **kwargs)\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 5435, in
> task_log_begin_task\n task.save()\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/models.py",
> line 53, in save\n session.flush()\n', u' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
> line 543, in _wrap\n raise exception.DBError(e)\n', u"DBError: SQLite
> DateTime type only accepts Python datetime and date objects as input.
> (original cause: TypeError: SQLite DateTime type only accepts Python
> datetime and date objects as input.) 'INSERT INTO task_log (created_at,
> updated_at, deleted_at, deleted, task_name, state, host, period_beginning,
> period_ending, message, task_items, errors) VALUES (?, ?, ?, ?, ?, ?, ?, ?,
> ?, ?, ?, ?)' [{'period_ending': u'2013-11-20T06:00:00.000000', 'state':
> 'RUNNING', 'updated_at': None, 'host': u'infra-eval-06', 'task_items': 2,
> 'task_name': u'instance_usage_audit', 'period_beginning':
> u'2013-11-20T05:00:00.000000', 'message': u'Instance usage audit
> started...', 'deleted_at': None}]\n"].
> ​===========================================​
> 
> My nova.conf is:
> nova.conf​===========================================​
> [DEFAULT]
> dhcpbridge_flagfile=/etc/nova/nova.conf
> dhcpbridge=/usr/bin/nova-dhcpbridge
> logdir=/var/log/nova
> state_path=/var/lib/nova
> lock_path=/var/lock/nova
> force_dhcp_release=True
> iscsi_helper=tgtadm
> libvirt_use_virtio_for_bridges=True
> connection_type=libvirt
> root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
> verbose=False
> debug=False
> ec2_private_dns_show_ip=True
> api_paste_config=/etc/nova/api-paste.ini
> volumes_path=/var/lib/nova/volumes
> enabled_apis=ec2,osapi_compute,metadata
> connection = mysql://nova:password@infra-eval-03/nova
> my_ip=192.168.1.26
> vncserver_listen=0.0.0.0
> vncserver_proxyclient_address=172.16.1.16
> rpc_backend = nova.rpc.impl_kombu
> rabbit_host = infra-eval-03
> glance_host = infra-eval-03
> network_api_class=nova.network.neutronv2.api.API
> neutron_url= http://infra-eval-03:9696
> neutron_auth_strategy=keystone
> neutron_admin_tenant_name=service
> neutron_admin_username=neutron
> neutron_admin_password=password
> neutron_admin_auth_url= http://infra-eval-03:35357/v2.0
> firewall_driver=nova.virt.firewall.NoopFirewallDriver
> ​​===========================================​
> 
> My ceilometer conf file is:
> ceilometer.conf​===========================================​
> [DEFAULT]
> sqlite_db=ceilometer.sqlite
> log_dir=/var/log/ceilometer
> rabbit_host=infra-eval-03
> [publisher_rpc]
> metering_secret=password
> [ssl]
> [database]
> connection = mongodb://ceilometer:password@infra-eval-03:27017/ceilometer
> [alarm]
> [rpc_notifier2]
> [api]
> [service_credentials]
> [dispatcher_file]
> [keystone_authtoken]
> auth_host = infra-eval-03
> auth_port = 35357
> auth_protocol = http
> admin_tenant_name = service
> admin_user = ceilometer
> admin_password = password
> auth_uri = http://192.168.1.23:5000
> [collector]
> [matchmaker_ring]
> [matchmaker_redis]
> ​===========================================​​
> 
> regards
> 
> 
> BUSSY Jean-Daniel
> 
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>




More information about the Openstack mailing list