What do you or what would you expect to see in Gnocchi being pushed by Ceilometer? What are the resources (volumes, VMs, and so) that you created in the OpenStack environment in DevStack?
On Fri, Mar 4, 2022 at 1:26 PM Molka Gharbaoui < Molka.Gharbaoui@santannapisa.it> wrote:
Unfortunately, the local.conf file shared by Matthias gives the same result: no metrics are pushed to gnocchi.
Gnocchi database is empty: I can create metrics and resources manually but nothing is pushed from ceilometer to the database. So I guess the problem is at the ceilometer level.
Here is an extract of the ceilometer-polling logs:
2022-03-04 16:13:07.904 679128 INFO ceilometer.polling.manager [-] Polling pollster disk.device.usage in the context of all_pollsters 2022-03-04 16:13:07.904 679128 DEBUG ceilometer.compute.pollsters [-] afcf32c4-e7c2-40fd-8f75-fdf0bfdb5ef9/disk.device.usage volume: 5368709120 _stats_to_sample /opt/stack/ceilometer/ceilometer/compute/pollsters/_ _init__.py:108 2022-03-04 16:13:07.907 679128 INFO ceilometer.polling.manager [-] Polling pollster disk.device.read.requests in the context of all_pollsters 2022-03-04 16:13:07.908 679128 DEBUG ceilometer.compute.pollsters [-] afcf32c4-e7c2-40fd-8f75-fdf0bfdb5ef9/disk.device.read.requests volume: 6138 _stats_to_sample /opt/stack/ceilometer/ceilometer/compute/pollsters/_ _init__.py:108 2022-03-04 16:13:08.038 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found. 2022-03-04 16:13:08.038 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.pool, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177 2022-03-04 16:13:08.162 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found. 2022-03-04 16:13:08.162 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.vip, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177 2022-03-04 16:13:08.163 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.outgoing.bytes, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py: 177 2022-03-04 16:13:08.164 679128 ERROR ceilometer.polling.manager [-] Skipping tenant, keystone issue: admin endpoint for identity service in RegionOne region not found: keystoneauth1.exceptions.catalog.EndpointNotFound: admin endpoint for identity service in RegionOne region not found 2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster radosgw.containers.objects.size, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177 2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster port.receive.packets, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177 2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster hardware.network.ip.outgoing.datagrams, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py: 177 2022-03-04 16:13:08.355 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found. 2022-03-04 16:13:08.355 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.listener, no resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
And here I report the content of the ceilometer.conf file: [DEFAULT]
debug = True transport_url = rabbit://stackrabbit:admin@10.30.2.194:5672/
[oslo_messaging_notifications] topics = notifications
[coordination] backend_url = redis://localhost:6379
[notification] workers = 2
[cache] backend_argument = url:redis://localhost:6379 backend_argument = distributed_lock:True backend_argument = db:0 backend_argument = redis_expiration_time:600 backend = dogpile.cache.redis enabled = True
[service_credentials] auth_url = http://10.30.2.194/identity http://10.30.2.194/identity/v3/ region_name = RegionOne password = admin username = ceilometer project_name = service project_domain_name = default user_domain_id = default auth_type = password
[keystone_authtoken] memcached_servers = localhost:11211 cafile = /opt/stack/data/ca-bundle.pem project_domain_name = default project_name = admin user_domain_name = default password = admin username = ceilometer auth_url = http://10.30.2.194/identity http://10.30.2.194/identity/v3/ interface = public auth_type = password
I believe nothing should be changed in the configuration files since Devstack should configure every components automatically. I also checked pipeline.yaml and polling.yaml files and they seem correct.
Do you have any idea on how to make ceilometer push metrics to the gnocchi database??
Thanks.
*Da:* Matthias Runge mrunge@matthias-runge.de *Inviato:* giovedì 3 marzo 2022 13:58 *A:* Molka Gharbaoui Molka.Gharbaoui@santannapisa.it *Oggetto:* Re: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi
On 3/3/22 12:31, Molka Gharbaoui wrote:
Hi Matthias,
Sorry for the confusion. Gnocchi is running and I am able to execute commands like "gnocchi metric list" and "gnocchi resource list" without errors even though the response is empty. Also checking mysql, the gnocchi database is correctly created.
By the link I previously shared I wanted just to highlight that there are some differences between what was written there and between the environemnt I installed. I was searching there because I would have expected that Devstack installs and configures every component in automatic with no or minimum external intervention (in fact if I don't consider the Telemetry aspect, the rest works perfectly fine).
I will follow Rafael guidelines to better undrstand where is the issue.
Thanks.
Something I have used in the past is a local.conf like this:
[[local|localrc]] ADMIN_PASSWORD=secret DATABASE_PASSWORD=$ADMIN_PASSWORD RABBIT_PASSWORD=$ADMIN_PASSWORD SERVICE_PASSWORD=$ADMIN_PASSWORD
IMAGE_URL_SITE="https://download.fedoraproject.org" IMAGE_URL_PATH="/pub/fedora/linux/releases/33/Cloud/x86_64/images/" IMAGE_URL_FILE="Fedora-Cloud-Base-33-1.2.x86_64.qcow2" IMAGE_URLS+=","$IMAGE_URL_SITE$IMAGE_URL_PATH$IMAGE_URL_FILE
CEILOMETER_BACKEND=gnocchi USE_PYTHON3="True"
VERBOSE="True"
VERBOSE_NO_TIMESTAMP="True"
#
LIBS_FROM_GIT=aodh,ceilometer,cinder,devstack,devstack-gate,dib-utils,diskimage-builder,glance,heat,keystone,neutron,nova,placement,requirements,swift
enable_plugin aodh https://opendev.org/openstack/aodh enable_plugin ceilometer https://opendev.org/openstack/ceilometer enable_plugin heat https://opendev.org/openstack/heat
INSTALL_TEMPEST=False
Hope that helps. The IMAGE_URL_FILE needs probably adjustment, fedora 33 is not supported anymore.
Matthias
*Da:* Matthias Runge mrunge@matthias-runge.de *Inviato:* giovedì 3 marzo 2022 11:59 *A:* openstack-discuss@lists.openstack.org openstack-discuss@lists.openstack.org *Oggetto:* Re: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi On 3/3/22 10:29, Molka Gharbaoui wrote:
Thank you Matthias for your reply.
Unfortunately, from the logs I still cannot understand what is the
issue.
Could someone kindly indicate me how ceilometer and gnocchi should be configured to get the data?
For example in this link:
https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.htm...
<
https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.htm...
<
https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.htm...
<
https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.htm... the
endpoints should use the following URL: http://controller:8041 <
<http://controller:8041 http://controller:8041> while in my case the
8041 port is not even
listening and the API indicated in the dashboard for Gnocchi is http:// <http://controller:8041 http://controller:8041>x.x.x.x/metric.
Thank you in advance.
OpenStack configuration is complex. The guide does not remotely touch on production environments. Please do not confuse devstack with production envionments. Also the doc you linked is not for devstack itself.
The flow in general is: data is generated by the services like nova or glance. Ceilometer listens on the bus and also polls for information. The info is sent then over (as configured in publishers.yaml) to gnocchi. Gnocchi has an api, ceilometer has not. Reading your response, this makes me suspicious if there is gnocchi running. From the initial question it suggests it is, from your most recent response, it is not.
Matthias