Open Stack

Yes, all ceilometer services are up and running: ceilometer-agent, ceilometer-notification. Actually, all devstack services are up and running.
________________________________
Da: Rafael Weingärtner <rafaelweingartner at gmail.com>
Inviato: venerdì 4 marzo 2022 17:56
A: Molka Gharbaoui <molka.gharbaoui at santannapisa.it>
Cc: Openstack Discuss List <openstack-discuss at lists.openstack.org>
Oggetto: Re: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi

I see. Is the Ceilometer compute agent up and running?

On Fri, Mar 4, 2022 at 1:54 PM Molka Gharbaoui <Molka.Gharbaoui at santannapisa.it<mailto:Molka.Gharbaoui at santannapisa.it>> wrote:
I have a running instance and I would like to retrieve some statistics on it like cpu usage, memory usage, packets received and sent.
________________________________
Da: Rafael Weingärtner <rafaelweingartner at gmail.com<mailto:rafaelweingartner at gmail.com>>
Inviato: venerdì 4 marzo 2022 17:28
A: Molka Gharbaoui <molka.gharbaoui at santannapisa.it<mailto:molka.gharbaoui at santannapisa.it>>
Cc: Openstack Discuss List <openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org>>
Oggetto: Re: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi

What do you or what would you expect to see in Gnocchi being pushed by Ceilometer?
What are the resources (volumes, VMs, and so) that you created in the OpenStack environment in DevStack?


On Fri, Mar 4, 2022 at 1:26 PM Molka Gharbaoui <Molka.Gharbaoui at santannapisa.it<mailto:Molka.Gharbaoui at santannapisa.it>> wrote:
Unfortunately, the local.conf file shared by Matthias gives the same result: no metrics are pushed to gnocchi.

Gnocchi database is empty: I can create metrics and resources manually but nothing is pushed from ceilometer to the database. So I guess the problem is at the ceilometer level.

Here is an extract of the ceilometer-polling logs:

2022-03-04 16:13:07.904 679128 INFO ceilometer.polling.manager [-] Polling pollster disk.device.usage in the context of all_pollsters
2022-03-04 16:13:07.904 679128 DEBUG ceilometer.compute.pollsters [-] afcf32c4-e7c2-40fd-8f75-fdf0bfdb5ef9/disk.device.usage volume: 5368709120 _stats_to_sample /opt/stack/ceilometer/ceilometer/compute/pollsters/__init__.py:108
2022-03-04 16:13:07.907 679128 INFO ceilometer.polling.manager [-] Polling pollster disk.device.read.requests in the context of all_pollsters
2022-03-04 16:13:07.908 679128 DEBUG ceilometer.compute.pollsters [-] afcf32c4-e7c2-40fd-8f75-fdf0bfdb5ef9/disk.device.read.requests volume: 6138 _stats_to_sample /opt/stack/ceilometer/ceilometer/compute/pollsters/__init__.py:108
2022-03-04 16:13:08.038 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-03-04 16:13:08.038 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.pool, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.162 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-03-04 16:13:08.162 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.vip, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.163 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.outgoing.bytes, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.164 679128 ERROR ceilometer.polling.manager [-] Skipping tenant, keystone issue: admin endpoint for identity service in RegionOne region not found: keystoneauth1.exceptions.catalog.EndpointNotFound: admin endpoint for identity service in RegionOne region not found
2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster radosgw.containers.objects.size, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster port.receive.packets, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.164 679128 DEBUG ceilometer.polling.manager [-] Skip pollster hardware.network.ip.outgoing.datagrams, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177
2022-03-04 16:13:08.355 679128 WARNING ceilometer.neutron_client [-] The resource could not be found.: neutronclient.common.exceptions.NotFound: The resource could not be found.
2022-03-04 16:13:08.355 679128 DEBUG ceilometer.polling.manager [-] Skip pollster network.services.lb.listener, no  resources found this cycle poll_and_notify /opt/stack/ceilometer/ceilometer/polling/manager.py:177

And here I report the content of the ceilometer.conf file:
[DEFAULT]
debug = True
transport_url = rabbit://stackrabbit:admin@10.30.2.194:5672/<http://stackrabbit:admin@10.30.2.194:5672/>

[oslo_messaging_notifications]
topics = notifications

[coordination]
backend_url = redis://localhost:6379

[notification]
workers = 2

[cache]
backend_argument = url:redis://localhost:6379
backend_argument = distributed_lock:True
backend_argument = db:0
backend_argument = redis_expiration_time:600
backend = dogpile.cache.redis
enabled = True

[service_credentials]
auth_url = http://10.30.2.194/identity<http://10.30.2.194/identity/v3/>
region_name = RegionOne
password = admin
username = ceilometer
project_name = service
project_domain_name = default
user_domain_id = default
auth_type = password

[keystone_authtoken]
memcached_servers = localhost:11211
cafile = /opt/stack/data/ca-bundle.pem
project_domain_name = default
project_name = admin
user_domain_name = default
password = admin
username = ceilometer
auth_url = http://10.30.2.194/identity<http://10.30.2.194/identity/v3/>
interface = public
auth_type = password

I believe nothing should be changed in the configuration files since Devstack should configure every components automatically. I also checked pipeline.yaml and polling.yaml files and they seem correct.

Do you have any idea on how to make ceilometer push metrics to the gnocchi database??

Thanks.


________________________________
Da: Matthias Runge <mrunge at matthias-runge.de<mailto:mrunge at matthias-runge.de>>
Inviato: giovedì 3 marzo 2022 13:58
A: Molka Gharbaoui <Molka.Gharbaoui at santannapisa.it<mailto:Molka.Gharbaoui at santannapisa.it>>
Oggetto: Re: R: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve metrics using ceilometer and gnocchi

On 3/3/22 12:31, Molka Gharbaoui wrote:
> Hi Matthias,
>
> Sorry for the confusion. Gnocchi is running and I am able to execute
> commands like "gnocchi metric list" and "gnocchi resource list" without
> errors even though the response is empty. Also checking mysql, the
> gnocchi database is correctly created.
>
> By the link I previously shared I wanted just to highlight that there
> are some differences between what was written there and between the
> environemnt I installed. I was searching there because I would have
> expected that Devstack installs and configures every component in
> automatic with no or minimum external intervention (in fact if I don't
> consider the Telemetry aspect, the rest works perfectly fine).
>
> I will follow Rafael guidelines to better undrstand where is the issue.
>
> Thanks.


Something I have used in the past is a local.conf like this:

[[local|localrc]]
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD

IMAGE_URL_SITE="https://download.fedoraproject.org"
IMAGE_URL_PATH="/pub/fedora/linux/releases/33/Cloud/x86_64/images/"
IMAGE_URL_FILE="Fedora-Cloud-Base-33-1.2.x86_64.qcow2"
IMAGE_URLS+=","$IMAGE_URL_SITE$IMAGE_URL_PATH$IMAGE_URL_FILE

CEILOMETER_BACKEND=gnocchi
USE_PYTHON3="True"

VERBOSE="True"

VERBOSE_NO_TIMESTAMP="True"

#
LIBS_FROM_GIT=aodh,ceilometer,cinder,devstack,devstack-gate,dib-utils,diskimage-builder,glance,heat,keystone,neutron,nova,placement,requirements,swift

enable_plugin aodh https://opendev.org/openstack/aodh
enable_plugin ceilometer https://opendev.org/openstack/ceilometer
enable_plugin heat https://opendev.org/openstack/heat


INSTALL_TEMPEST=False


Hope that helps. The IMAGE_URL_FILE needs probably adjustment, fedora 33
is not supported anymore.

Matthias



>
>
> ------------------------------------------------------------------------
> *Da:* Matthias Runge <mrunge at matthias-runge.de<mailto:mrunge at matthias-runge.de>>
> *Inviato:* giovedì 3 marzo 2022 11:59
> *A:* openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org>
> <openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org>>
> *Oggetto:* Re: R: [Devstack][Gnocchi][Ceilometer] Not able to retrieve
> metrics using ceilometer and gnocchi
> On 3/3/22 10:29, Molka Gharbaoui wrote:
>> Thank you Matthias for your reply.
>>
>> Unfortunately, from the logs I still cannot understand what is the issue.
>>
>> Could someone kindly indicate me how ceilometer and gnocchi should be
>> configured to get the data?
>>
>> For example in this link:
>> https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.html
> <https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.html>
>
>> <https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.html
> <https://docs.openstack.org/ceilometer/latest/install/install-base-ubuntu.html>> the
>
>> endpoints should use the following URL: http://controller:8041 <http://controller:8041>
>> <http://controller:8041 <http://controller:8041>> while in my case the
> 8041 port is not even
>> listening and the API indicated in the dashboard for Gnocchi is http://
>> <http://controller:8041 <http://controller:8041>>x.x.x.x/metric.
>>
>> Thank you in advance.
>>
>
> OpenStack configuration is complex. The guide does not remotely touch on
> production environments. Please do not confuse devstack with production
> envionments. Also the doc you linked is not for devstack itself.
>
> The flow in general is: data is generated by the services like nova or
> glance. Ceilometer listens on the bus and also polls for information.
> The info is sent then over (as configured in publishers.yaml) to gnocchi.
> Gnocchi has an api, ceilometer has not. Reading your response, this
> makes me suspicious if there is gnocchi running. From the initial
> question it suggests it is, from your most recent response, it is not.
>
>
> Matthias
>



--
Rafael Weingärtner


--
Rafael Weingärtner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20220304/b156a6f0/attachment-0001.htm>