[Openstack] [ceilometer] Troubleshooting Ceilometer

Tom Walsh expresswebsys at gmail.com
Wed Mar 2 20:26:22 UTC 2016


Nadya,

The logs for agent-notification appear to be clear of errors. Here is
an an excerpt from it: (I turned on debug to see if there is any
additional information that might be suppressed, but there doesn't
seem to be much there)

 2016-03-02 13:43:23.194 31681 DEBUG ceilometer.pipeline [-] Pipeline
config file: /etc/ceilometer/pipeline.yaml _setup_pipeline_manager
/usr/lib/python2.7/dist-packages/ceilometer/pipeline.py:792
2016-03-02 13:43:23.209 31681 INFO ceilometer.pipeline [-] Pipeline
config: {'sources': [{'interval': 600, 'meters': ['*'], 'name':
'meter_source', 'sinks': ['meter_sink']}, {'interval': 600, 'meters':
['cpu'], 'name': 'cpu_source', 'sinks': ['cpu_sink',
'cpu_delta_sink']}, {'interval': 600, 'meters': ['disk.read.bytes',
'disk.read.requests', 'disk.write.bytes', 'disk.write.requests',
'disk.device.read.bytes', 'disk.device.read.requests',
'disk.device.write.bytes', 'disk.device.write.requests'], 'name':
'disk_source', 'sinks': ['disk_sink']}, {'interval': 600, 'meters':
['network.incoming.bytes', 'network.incoming.packets',
'network.outgoing.bytes', 'network.outgoing.packets'], 'name':
'network_source', 'sinks': ['network_sink']}], 'sinks':
[{'publishers': ['notifier://'], 'transformers': None, 'name':
'meter_sink'}, {'publishers': ['notifier://'], 'transformers':
[{'name': 'rate_of_change', 'parameters': {'target': {'scale': '100.0
/ (10**9 * (resource_metadata.cpu_number or 1))', 'type': 'gauge',
'name': 'cpu_util', 'unit': '%'}}}], 'name': 'cpu_sink'},
{'publishers': ['notifier://'], 'transformers': [{'name': 'delta',
'parameters': {'target': {'name': 'cpu.delta'}, 'growth_only':
True}}], 'name': 'cpu_delta_sink'}, {'publishers': ['notifier://'],
'transformers': [{'name': 'rate_of_change', 'parameters': {'source':
{'map_from': {'name':
'(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)', 'unit':
'(B|request)'}}, 'target': {'type': 'gauge', 'map_to': {'name':
'\\1.\\2.\\3.rate', 'unit': '\\1/s'}}}}], 'name': 'disk_sink'},
{'publishers': ['notifier://'], 'transformers': [{'name':
'rate_of_change', 'parameters': {'source': {'map_from': {'name':
'network\\.(incoming|outgoing)\\.(bytes|packets)', 'unit':
'(B|packet)'}}, 'target': {'type': 'gauge', 'map_to': {'name':
'network.\\1.\\2.rate', 'unit': '\\1/s'}}}}], 'name':
'network_sink'}]}
2016-03-02 13:43:23.262 31681 INFO ceilometer.pipeline [-] detected
decoupled pipeline config format
2016-03-02 13:43:23.263 31681 INFO ceilometer.publisher.messaging [-]
Publishing policy set to default
2016-03-02 13:43:23.283 31681 INFO ceilometer.publisher.messaging [-]
Publishing policy set to default
2016-03-02 13:43:23.283 31681 DEBUG ceilometer.transformer.conversions
[-] scaling conversion transformer with source: {} target: {'scale':
'100.0 / (10**9 * (resource_metadata.cpu_number or 1))', 'type':
'gauge', 'name': 'cpu_util', 'unit': '%'}: __init__
/usr/lib/python2.7/dist-packages/ceilometer/transformer/conversions.py:136
2016-03-02 13:43:23.292 31681 INFO ceilometer.pipeline [-] Pipeline
cpu_sink: Setup transformer instance rate_of_change with parameter
{'target': {'scale': '100.0 / (10**9 * (resource_metadata.cpu_number
or 1))', 'type': 'gauge', 'name': 'cpu_util', 'unit': '%'}}
2016-03-02 13:43:23.292 31681 INFO ceilometer.publisher.messaging [-]
Publishing policy set to default
2016-03-02 13:43:23.300 31681 INFO ceilometer.pipeline [-] Pipeline
cpu_delta_sink: Setup transformer instance delta with parameter
{'target': {'name': 'cpu.delta'}, 'growth_only': True}
2016-03-02 13:43:23.301 31681 INFO ceilometer.publisher.messaging [-]
Publishing policy set to default
2016-03-02 13:43:23.301 31681 DEBUG ceilometer.transformer.conversions
[-] scaling conversion transformer with source: {'map_from': {'name':
'(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)', 'unit':
'(B|request)'}} target: {'map_to': {'name': '\\1.\\2.\\3.rate',
'unit': '\\1/s'}, 'type': 'gauge'}: __init__
/usr/lib/python2.7/dist-packages/ceilometer/transformer/conversions.py:136
2016-03-02 13:43:23.310 31681 INFO ceilometer.pipeline [-] Pipeline
disk_sink: Setup transformer instance rate_of_change with parameter
{'source': {'map_from': {'name':
'(disk\\.device|disk)\\.(read|write)\\.(bytes|requests)', 'unit':
'(B|request)'}}, 'target': {'type': 'gauge', 'map_to': {'name':
'\\1.\\2.\\3.rate', 'unit': '\\1/s'}}}
2016-03-02 13:43:23.311 31681 INFO ceilometer.publisher.messaging [-]
Publishing policy set to default
2016-03-02 13:43:23.311 31681 DEBUG ceilometer.transformer.conversions
[-] scaling conversion transformer with source: {'map_from': {'name':
'network\\.(incoming|outgoing)\\.(bytes|packets)', 'unit':
'(B|packet)'}} target: {'map_to': {'name': 'network.\\1.\\2.rate',
'unit': '\\1/s'}, 'type': 'gauge'}: __init__
/usr/lib/python2.7/dist-packages/ceilometer/transformer/conversions.py:136
2016-03-02 13:43:23.319 31681 INFO ceilometer.pipeline [-] Pipeline
network_sink: Setup transformer instance rate_of_change with parameter
{'source': {'map_from': {'name':
'network\\.(incoming|outgoing)\\.(bytes|packets)', 'unit':
'(B|packet)'}}, 'target': {'type': 'gauge', 'map_to': {'name':
'network.\\1.\\2.rate', 'unit': '\\1/s'}}}
2016-03-02 13:43:23.329 31681 DEBUG ceilometer.meter.notifications [-]
Meter Definitions configuration file:
/usr/lib/python2.7/dist-packages/ceilometer/meter/data/meters.yaml
setup_meters_config
/usr/lib/python2.7/dist-packages/ceilometer/meter/notifications.py:150
2016-03-02 13:43:23.447 31681 INFO ceilometer.meter.notifications [-]
Meter Definitions: {'metric': [{'name': 'image.size', 'resource_id':
'$.payload.id', 'volume': '$.payload.size', 'project_id':
'$.payload.owner', 'type': 'gauge', 'unit': 'B', 'event_type':
['image.upload', 'image.delete', 'image.update']}, {'user_id':
'$.payload.receiver_user_id', 'name': 'image.download', 'resource_id':
'$.payload.image_id', 'volume': '$.payload.bytes_sent', 'project_id':
'$.payload.receiver_tenant_id', 'type': 'delta', 'unit': 'B',
'event_type': 'image.send'}, {'name': 'image.serve', 'resource_id':
'$.payload.image_id', 'volume': '$.payload.bytes_sent', 'project_id':
'$.payload.owner_id', 'type': 'delta', 'unit': 'B', 'event_type':
'image.send'}, {'user_id': '$._context_user', 'name':
'magnetodb.table.index.count', 'resource_id': '$.payload.table_uuid',
'volume': '$.payload.index_count', 'type': 'gauge', 'unit': 'index',
'event_type': 'magnetodb.table.create.end'}, {'user_id':
'$.payload.user_id', 'name': 'volume.size', 'resource_id':
'$.payload.volume_id', 'volume': '$.payload.size', 'project_id':
'$.payload.tenant_id', 'type': 'gauge', 'unit': 'GB', 'event_type':
['volume.exists', 'volume.create.*', 'volume.delete.*',
'volume.resize.*', 'volume.attach.*', 'volume.detach.*',
'volume.update.*']}, {'user_id': '$.payload.user_id', 'name':
'snapshot.size', 'resource_id': '$.payload.snapshot_id', 'volume':
'$.payload.volume_size', 'project_id': '$.payload.tenant_id', 'type':
'gauge', 'unit': 'GB', 'event_type': ['snapshot.exists',
'snapshot.create.*', 'snapshot.delete.*']}, {'user_id':
'$.payload.user_id', 'name': '$.payload.metrics.[*].name',
'resource_id': '$.payload.resource_id', 'volume':
'$.payload.metrics.[*].value', 'lookup': ['name', 'unit', 'volume'],
'project_id': '$.payload.project_id', 'type': 'gauge', 'unit':
'$.payload.metrics.[*].unit', 'event_type': 'magnum.bay.metrics.*'},
{'user_id': '$.payload.initiator.id', 'name':
'$.payload.measurements.[*].metric.[*].name', 'resource_id':
'$.payload.target.id', 'volume': '$.payload.measurements.[*].result',
'lookup': ['name', 'unit', 'volume'], 'project_id':
'$.payload.initiator.project_id', 'type': 'delta', 'unit':
'$.payload.measurements.[*].metric.[*].unit', 'event_type':
'objectstore.http.request'}, {'user_id': '$.payload.user_id', 'name':
'memory', 'resource_id': '$.payload.instance_id', 'volume':
'$.payload.memory_mb', 'project_id': '$.payload.tenant_id', 'type':
'gauge', 'unit': 'MB', 'event_type': 'compute.instance.*'},
{'user_id': '$.payload.user_id', 'name': 'vcpus', 'resource_id':
'$.payload.instance_id', 'volume': '$.payload.vcpus', 'project_id':
'$.payload.tenant_id', 'type': 'gauge', 'unit': 'vcpu', 'event_type':
'compute.instance.*'}, {'user_id': '$.payload.user_id', 'name':
'disk.root.size', 'resource_id': '$.payload.instance_id', 'volume':
'$.payload.root_gb', 'project_id': '$.payload.tenant_id', 'type':
'gauge', 'unit': 'GB', 'event_type': 'compute.instance.*'},
{'user_id': '$.payload.user_id', 'name': 'disk.ephemeral.size',
'resource_id': '$.payload.instance_id', 'volume':
'$.payload.ephemeral_gb', 'project_id': '$.payload.tenant_id', 'type':
'gauge', 'unit': 'GB', 'event_type': 'compute.instance.*'}, {'name':
'bandwidth', 'resource_id': '$.payload.label_id', 'volume':
'$.payload.bytes', 'project_id': '$.payload.tenant_id', 'type':
'delta', 'unit': 'B', 'event_type': 'l3.meter'}, {'name':
'compute.node.cpu.frequency', 'resource_id': '$.payload.host + "_" +
$.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.frequency')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.frequency')].value", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.frequency')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'gauge', 'unit': 'MHz', 'event_type': 'compute.metrics.update'},
{'name': 'compute.node.cpu.user.time', 'resource_id': '$.payload.host
+ "_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.user.time')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.user.time')].value", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.user.time')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'cumulative', 'unit': 'ns', 'event_type': 'compute.metrics.update'},
{'name': 'compute.node.cpu.kernel.time', 'resource_id':
'$.payload.host + "_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.kernel.time')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.kernel.time')].value", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.kernel.time')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'cumulative', 'unit': 'ns', 'event_type': 'compute.metrics.update'},
{'name': 'compute.node.cpu.idle.time', 'resource_id': '$.payload.host
+ "_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.idle.time')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.idle.time')].value", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.idle.time')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'cumulative', 'unit': 'ns', 'event_type': 'compute.metrics.update'},
{'name': 'compute.node.cpu.iowait.time', 'resource_id':
'$.payload.host + "_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.iowait.time')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.iowait.time')].value", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.iowait.time')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'cumulative', 'unit': 'ns', 'event_type': 'compute.metrics.update'},
{'name': 'compute.node.cpu.kernel.percent', 'resource_id':
'$.payload.host + "_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.kernel.percent')].timestamp",
'volume': "$.payload.metrics[?(@.name='cpu.kernel.percent')].value *
100", 'metadata': {'source':
"$.payload.metrics[?(@.name='cpu.kernel.percent')].source", 'host':
'$.publisher_id', 'event_type': '$.event_type'}, 'type': 'gauge',
'unit': 'percent', 'event_type': 'compute.metrics.update'}, {'name':
'compute.node.cpu.idle.percent', 'resource_id': '$.payload.host + "_"
+ $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.idle.percent')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.idle.percent')].value * 100",
'metadata': {'source':
"$.payload.metrics[?(@.name='cpu.idle.percent')].source", 'host':
'$.publisher_id', 'event_type': '$.event_type'}, 'type': 'gauge',
'unit': 'percent', 'event_type': 'compute.metrics.update'}, {'name':
'compute.node.cpu.user.percent', 'resource_id': '$.payload.host + "_"
+ $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.user.percent')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.user.percent')].value * 100",
'metadata': {'source':
"$.payload.metrics[?(@.name='cpu.user.percent')].source", 'host':
'$.publisher_id', 'event_type': '$.event_type'}, 'type': 'gauge',
'unit': 'percent', 'event_type': 'compute.metrics.update'}, {'name':
'compute.node.cpu.iowait.percent', 'resource_id': '$.payload.host +
"_" + $.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.iowait.percent')].timestamp",
'volume': "$.payload.metrics[?(@.name='cpu.iowait.percent')].value *
100", 'metadata': {'source':
"$.payload.metrics[?(@.name='cpu.iowait.percent')].source", 'host':
'$.publisher_id', 'event_type': '$.event_type'}, 'type': 'gauge',
'unit': 'percent', 'event_type': 'compute.metrics.update'}, {'name':
'compute.node.cpu.percent', 'resource_id': '$.payload.host + "_" +
$.payload.nodename', 'timestamp':
"$.payload.metrics[?(@.name='cpu.percent')].timestamp", 'volume':
"$.payload.metrics[?(@.name='cpu.percent')].value * 100", 'metadata':
{'source': "$.payload.metrics[?(@.name='cpu.percent')].source",
'host': '$.publisher_id', 'event_type': '$.event_type'}, 'type':
'gauge', 'unit': 'percent', 'event_type': 'compute.metrics.update'},
{'name': 'image', 'resource_id': '$.payload.id', 'volume': 1,
'project_id': '$.payload.owner', 'type': 'gauge', 'unit': 'image',
'event_type': ['image.upload', 'image.update', 'image.delete']},
{'name': 'image.upload', 'resource_id': '$.payload.id', 'volume': 1,
'project_id': '$.payload.owner', 'type': 'gauge', 'unit': 'image',
'event_type': ['image.upload']}, {'name': 'image.delete',
'resource_id': '$.payload.id', 'volume': 1, 'project_id':
'$.payload.owner', 'type': 'gauge', 'unit': 'image', 'event_type':
['image.delete']}, {'name': 'image.update', 'resource_id':
'$.payload.id', 'volume': 1, 'project_id': '$.payload.owner', 'type':
'gauge', 'unit': 'image', 'event_type': ['image.update']}, {'user_id':
'_context_trustor_user_id', 'name': 'stack.create', 'resource_id':
'$.payload.stack_identity', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'stack', 'event_type':
['orchestration.stack.create.end']}, {'user_id':
'_context_trustor_user_id', 'name': 'stack.update', 'resource_id':
'$.payload.stack_identity', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'stack', 'event_type':
['orchestration.stack.update.end']}, {'user_id':
'_context_trustor_user_id', 'name': 'stack.delete', 'resource_id':
'$.payload.stack_identity', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'stack', 'event_type':
['orchestration.stack.delete.end']}, {'user_id':
'_context_trustor_user_id', 'name': 'stack.resume', 'resource_id':
'$.payload.stack_identity', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'stack', 'event_type':
['orchestration.stack.resume.end']}, {'user_id':
'_context_trustor_user_id', 'name': 'stack.suspend', 'resource_id':
'$.payload.stack_identity', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'stack', 'event_type':
['orchestration.stack.suspend.end']}, {'user_id': '_context_user',
'name': 'magnetodb.table.create', 'resource_id':
'$.payload.table_uuid', 'volume': 1, 'project_id': '_context_tenant',
'type': 'gauge', 'unit': 'table', 'event_type':
'magnetodb.table.create.end'}, {'user_id': '_context_user', 'name':
'magnetodb.table.delete', 'resource_id': '$.payload.table_uuid',
'volume': 1, 'project_id': '_context_tenant', 'type': 'gauge', 'unit':
'table', 'event_type': 'magnetodb.table.delete.end'}, {'user_id':
'$.payload.user_id', 'name': 'volume', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'gauge', 'unit': 'volume',
'event_type': ['volume.exists', 'volume.create.*', 'volume.delete.*',
'volume.resize.*', 'volume.attach.*', 'volume.detach.*',
'volume.update.*']}, {'user_id': '$.payload.user_id', 'name':
'volume.exists', 'resource_id': '$.payload.volume_id', 'volume': 1,
'project_id': '$.payload.tenant_id', 'type': 'delta', 'unit':
'volume', 'event_type': ['volume.exists']}, {'user_id':
'$.payload.user_id', 'name': 'volume.create.start', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.create.start']}, {'user_id':
'$.payload.user_id', 'name': 'volume.create.end', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.create.end']}, {'user_id': '$.payload.user_id',
'name': 'volume.delete.start', 'resource_id': '$.payload.volume_id',
'volume': 1, 'project_id': '$.payload.tenant_id', 'type': 'delta',
'unit': 'volume', 'event_type': ['volume.delete.start']}, {'user_id':
'$.payload.user_id', 'name': 'volume.delete.end', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.delete.end']}, {'user_id': '$.payload.user_id',
'name': 'volume.update.end', 'resource_id': '$.payload.volume_id',
'volume': 1, 'project_id': '$.payload.tenant_id', 'type': 'delta',
'unit': 'volume', 'event_type': ['volume.update.end']}, {'user_id':
'$.payload.user_id', 'name': 'volume.update.start', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.update.start']}, {'user_id':
'$.payload.user_id', 'name': 'volume.resize.end', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.resize.end']}, {'user_id': '$.payload.user_id',
'name': 'volume.resize.start', 'resource_id': '$.payload.volume_id',
'volume': 1, 'project_id': '$.payload.tenant_id', 'type': 'delta',
'unit': 'volume', 'event_type': ['volume.resize.start']}, {'user_id':
'$.payload.user_id', 'name': 'volume.attach.end', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.attach.end']}, {'user_id': '$.payload.user_id',
'name': 'volume.attach.start', 'resource_id': '$.payload.volume_id',
'volume': 1, 'project_id': '$.payload.tenant_id', 'type': 'delta',
'unit': 'volume', 'event_type': ['volume.attach.start']}, {'user_id':
'$.payload.user_id', 'name': 'volume.detach.end', 'resource_id':
'$.payload.volume_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'volume',
'event_type': ['volume.detach.end']}, {'user_id': '$.payload.user_id',
'name': 'volume.detach.start', 'resource_id': '$.payload.volume_id',
'volume': 1, 'project_id': '$.payload.tenant_id', 'type': 'delta',
'unit': 'volume', 'event_type': ['volume.detach.start']}, {'user_id':
'$.payload.user_id', 'name': 'snapshot', 'resource_id':
'$.payload.snapshot_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'gauge', 'unit': 'snapshot',
'event_type': ['snapshot.exists', 'snapshot.create.*',
'snapshot.delete.*']}, {'user_id': '$.payload.user_id', 'name':
'snapshot.exists', 'resource_id': '$.payload.snapshot_id', 'volume':
1, 'project_id': '$.payload.tenant_id', 'type': 'delta', 'unit':
'snapshot', 'event_type': ['snapshot.exists']}, {'user_id':
'$.payload.user_id', 'name': 'snapshot.create.start', 'resource_id':
'$.payload.snapshot_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'snapshot',
'event_type': ['snapshot.create.start']}, {'user_id':
'$.payload.user_id', 'name': 'snapshot.create.end', 'resource_id':
'$.payload.snapshot_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'snapshot',
'event_type': ['snapshot.create.end']}, {'user_id':
'$.payload.user_id', 'name': 'snapshot.delete.start', 'resource_id':
'$.payload.snapshot_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'snapshot',
'event_type': ['snapshot.delete.start']}, {'user_id':
'$.payload.user_id', 'name': 'snapshot.delete.end', 'resource_id':
'$.payload.snapshot_id', 'volume': 1, 'project_id':
'$.payload.tenant_id', 'type': 'delta', 'unit': 'snapshot',
'event_type': ['snapshot.delete.end']}, {'name': 'cluster.create',
'resource_id': '$.payload.cluster_id', 'volume': 1, 'project_id':
'$.payload.project_id', 'type': 'delta', 'unit': 'cluster',
'event_type': ['sahara.cluster.create']}, {'name': 'cluster.update',
'resource_id': '$.payload.cluster_id', 'volume': 1, 'project_id':
'$.payload.project_id', 'type': 'delta', 'unit': 'cluster',
'event_type': ['sahara.cluster.update']}, {'name': 'cluster.delete',
'resource_id': '$.payload.cluster_id', 'volume': 1, 'project_id':
'$.payload.project_id', 'type': 'delta', 'unit': 'cluster',
'event_type': ['sahara.cluster.delete']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.user.created',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'user', 'event_type': ['identity.user.created']},
{'user_id': '$.payload.initiator.id', 'name': 'identity.user.updated',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'user', 'event_type': ['identity.user.updated']},
{'user_id': '$.payload.initiator.id', 'name': 'identity.user.deleted',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'user', 'event_type': ['identity.user.deleted']},
{'user_id': '$.payload.initiator.id', 'name':
'identity.group.created', 'resource_id': '$.payload.resource_info',
'volume': 1, 'type': 'delta', 'unit': 'group', 'event_type':
['identity.group.created']}, {'user_id': '$.payload.initiator.id',
'name': 'identity.group.updated', 'resource_id':
'$.payload.resource_info', 'volume': 1, 'type': 'delta', 'unit':
'group', 'event_type': ['identity.group.updated']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.group.deleted',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'group', 'event_type': ['identity.group.deleted']},
{'user_id': '$.payload.initiator.id', 'name':
'identity.project.created', 'resource_id': '$.payload.resource_info',
'volume': 1, 'type': 'delta', 'unit': 'project', 'event_type':
['identity.project.created']}, {'user_id': '$.payload.initiator.id',
'name': 'identity.project.updated', 'resource_id':
'$.payload.resource_info', 'volume': 1, 'type': 'delta', 'unit':
'project', 'event_type': ['identity.project.updated']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.project.deleted',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'project', 'event_type':
['identity.project.deleted']}, {'user_id': '$.payload.initiator.id',
'name': 'identity.role.created', 'resource_id':
'$.payload.resource_info', 'volume': 1, 'type': 'delta', 'unit':
'role', 'event_type': ['identity.role.created']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.role.updated',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'role', 'event_type': ['identity.role.updated']},
{'user_id': '$.payload.initiator.id', 'name': 'identity.role.deleted',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'role', 'event_type': ['identity.role.deleted']},
{'user_id': '$.payload.initiator.id', 'name':
'identity.role_assignment.created', 'resource_id': '$.payload.role',
'volume': 1, 'type': 'delta', 'unit': 'role_assignment', 'event_type':
['identity.role_assignment.created']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.role_assignment.deleted',
'resource_id': '$.payload.role', 'volume': 1, 'type': 'delta', 'unit':
'role_assignment', 'event_type':
['identity.role_assignment.deleted']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.authenticate.success',
'resource_id': '$.payload.initiator.id', 'volume': 1, 'type': 'delta',
'unit': 'user', 'event_type': ['identity.authenticate']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.authenticate.pending',
'resource_id': '$.payload.initiator.id', 'volume': 1, 'type': 'delta',
'unit': 'user', 'event_type': ['identity.authenticate']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.authenticate.failure',
'resource_id': '$.payload.initiator.id', 'volume': 1, 'type': 'delta',
'unit': 'user', 'event_type': ['identity.authenticate']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.trust.created',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'trust', 'event_type':
['identity.OS-TRUST:trust.created']}, {'user_id':
'$.payload.initiator.id', 'name': 'identity.trust.deleted',
'resource_id': '$.payload.resource_info', 'volume': 1, 'type':
'delta', 'unit': 'trust', 'event_type':
['identity.OS-TRUST:trust.deleted']}, {'user_id':
'$.payload.initiator.id', 'name': 'storage.api.request',
'resource_id': '$.payload.target.id', 'volume': 1, 'project_id':
'$.payload.initiator.project_id', 'type': 'delta', 'unit': 'request',
'event_type': ['objectstore.http.request']}, {'user_id':
'$.payload.user_id', 'name': '$.payload.name', 'resource_id':
'"profiler-" + $.payload.base_id', 'volume': 1, 'project_id':
'$.payload.project_id', 'type': 'gauge', 'unit': 'trace',
'event_type': 'profiler.*'}]}

So there are no exceptions being thrown... the process seems to be
running correctly.

Tom Walsh
ExpressHosting.net

On Wed, Mar 2, 2016 at 12:19 PM, Nadya Shakhat <nprivalova at mirantis.com> wrote:
> Tom,
>
> From results you provided is seen that neither polling nor notification data
> have consumers. So, first of all, check the status of agent-notification
> service, something is wrong with it. Before it is fixed, no sense to check
> Mongo. It just cannot contain any data.
>
> Thanks,
> Nadya
>
> On Wed, Mar 2, 2016 at 6:48 PM, Tom Walsh <expresswebsys at gmail.com> wrote:
>>
>> Nadya,
>>
>> Okay. So looking at these two here is what I am seeing:
>>
>> metering.sample 0       1
>>
>> notifications.error     5       0
>> notifications.info      1139    0
>> notifications.sample    18488   0
>>
>> So I am guessing that means we have a consumer for metering, but not a
>> consumer on notifications (and there is a bit of a backlog on those
>> queues). Is that a correct statement?
>>
>> We are utilizing the mongodb backend for storage, but we aren't overly
>> familiar with it or how to inspect data within it. Any pointers for
>> checking that side of things?
>>
>>
>> On Wed, Mar 2, 2016 at 5:10 AM, Nadya Shakhat <nprivalova at mirantis.com>
>> wrote:
>> > Hi Tom,
>> >
>> > You may check several things:
>> > 1. Do you have messages in queues and do the queues have listeners? For
>> > example, check collector-related queue 'metering.sample' by default.
>> >
>> > rabbitmqctl list_queues name messages consumers | grep metering
>> >
>> > rabbitmqctl list_queues name messages consumers | grep notification
>> >
>> > The both queues should have consumers
>> >
>> >
>> > 2. Check entries directly in db you use.
>> >
>> >
>> > Thanks,
>> >
>> > Nadya
>> >
>> >
>> >
>> > On Wed, Mar 2, 2016 at 6:57 AM, Tom Walsh <expresswebsys at gmail.com>
>> > wrote:
>> >>
>> >> We currently have a working Liberty based Openstack deployment based
>> >> on Ubuntu LT 14.04 (2 controllers, 4 compute).
>> >>
>> >> We have followed the instructions at
>> >>
>> >>
>> >> http://docs.openstack.org/liberty/install-guide-ubuntu/ceilometer-install.html
>> >> and everything seems to have installed correctly. No errors in the
>> >> logs, and nothing really obviously wrong.
>> >>
>> >> However in the instructions there is this page
>> >>
>> >>
>> >> http://docs.openstack.org/liberty/install-guide-ubuntu/ceilometer-verify.html
>> >> that shows the output of meter-list, however our meter-list output is
>> >> blank.
>> >>
>> >> Example output:
>> >>
>> >> $ ceilometer meter-list
>> >>
>> >>
>> >> +--------------+-------+-------+--------------------------------------+---------+------------+
>> >> | Name         | Type  | Unit  | Resource ID
>> >> | User ID | Project ID |
>> >>
>> >>
>> >> +--------------+-------+-------+--------------------------------------+---------+------------+
>> >> | image        | gauge | image | acafc7c0-40aa-4026-9673-b879898e1fc2
>> >> | None    | cf12a15... |
>> >> | image.size   | gauge | B     | acafc7c0-40aa-4026-9673-b879898e1fc2
>> >> | None    | cf12a15... |
>> >>
>> >>
>> >> +--------------+-------+-------+--------------------------------------+---------+------------+
>> >>
>> >> root at controller01:/var/log/ceilometer# ceilometer meter-list
>> >> +------+------+------+-------------+---------+------------+
>> >> | Name | Type | Unit | Resource ID | User ID | Project ID |
>> >> +------+------+------+-------------+---------+------------+
>> >> +------+------+------+-------------+---------+------------+
>> >>
>> >> There are no errors output, and it seems the command completes
>> >> correctly.
>> >>
>> >> So where am I going wrong here? Is there a step I am missing? Or is
>> >> there something obvious I have missed?
>> >>
>> >> We have enabled glance, nova-compute, and cinder from the
>> >> instructions, but we aren't seeing any data (at least according to the
>> >> instructions).
>> >>
>> >>
>> >> Tom Walsh
>> >> Expresshosting.net
>> >>
>> >> _______________________________________________
>> >> Mailing list:
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >> Post to     : openstack at lists.openstack.org
>> >> Unsubscribe :
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >
>> >
>
>




More information about the Openstack mailing list