Hello Greg, I don't think there is one approach to reach the one telemetry solution for OpenStack. It depends on your needs, the ecosystem around or present and so on. The services of OpenStack (ceilometer,aodh) and also gnocchi are very divers in its possibilitys to configure and use. I'm not so deep in all of these services, but in my opinion there are multiple solution. Look for example to the respective deployment tools around OpenStack and what they will do. There are working many capable people with also the feedback from multiple deployments in different environments and use cases. So possibly there are thoughts for you. We use kolla-ansible approach [1] but as an example there are others out there [2] [1] https://docs.openstack.org/kolla-ansible/latest/reference/logging-and-monito... [2]https://docs.scs.community/docs/category/monitoring Mit freundlichen Grüßen / Best Regards i.A. Jörn Kaster Cloud Platform Engineer Note: My work hours may not be your work hours. Please do not feel the need to respond during a time that is not convenient for you. EPG – Ehrhardt Partner Group [EPG_Logo_withClaim_rgb_transparent-background.png]<https://www.epg.com/de/> __________________________________________ EPX Ehrhardt + Partner Xtended GmbH Alte Römerstraße 3 56154 Boppard-Buchholz Germany Phone: (+49) 67 42 / 87 27 0 Fax: (+49) 67 42 / 87 27 50 E-Mail: info@epg.com Internet: www.epg.com __________________________________________ CEO: Marco Ehrhardt, Markus Derksen Commercial register Koblenz HRB 22546 Registered office: Boppard ________________________________ Von: Waines, Greg <Greg.Waines@windriver.com> Gesendet: Mittwoch, 25. September 2024 17:42 An: openstack-discuss <openstack-discuss@lists.openstack.org> Betreff: [ceilometer] What's the general status / recommendation for Telemetry in OpenStack [Sie erhalten nicht h?ufig E-Mails von greg.waines@windriver.com. Weitere Informationen, warum dies wichtig ist, finden Sie unter https://aka.ms/LearnAboutSenderIdentification ] OUTSIDE-EPG! Hey ... I am working in OpenStack StarlingX which supports a Kubernetes cluster and a containerized OpenStack cluster (via openstack-helm). Our OpenStack offering is fairly simple ... only running Nova, Cinder, Neutron, Keystone, Glance, Horizon and Heat ... @ Antelope. In a predecessor of StarlingX, maybe in the Queens(?) timeframe ... we had a Telemetry solution that consisted of: - Ceilometer (for basic data polling, notifications and collection) - Panko (for event DB) - Gnocchi (for time-series DB) - Aodh (for alarming based on metric thresholds, etc.) HOWEVER ... today (Caracal) ... - I think Gnocchi is still alive and well ... https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgnocchixyz%2Fgnocchi&data=05%7C02%7Cjoern.kaster%40epg.com%7C6daa096897bf43ad2ef208dcdd78d81a%7Cd560862d1a634bec968adb98247481f5%7C0%7C0%7C638628758323428757%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=Z5hJKcjoo%2FHp17tbmlZtw3H2od3prSWa%2FTS3ErC1m4k%3D&reserved=0<https://github.com/gnocchixyz/gnocchi> * but maintained outside of OpenStack - I think OpenStack Panko was deprecated in ~ Wallaby ? * and don't think(?) it is maintained outside of OpenStack ? - The OpenStack Landscape at https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.openstack.org%2Fsoftware%2F&data=05%7C02%7Cjoern.kaster%40epg.com%7C6daa096897bf43ad2ef208dcdd78d81a%7Cd560862d1a634bec968adb98247481f5%7C0%7C0%7C638628758323440855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=dP7MHE2AfXkZ03B522v0vNV%2Ba%2Bff1aYqhEnrh7Z18QY%3D&reserved=0<https://www.openstack.org/software/> * shows only Ceilometer and Aodh MY QUESTIONs : ----------------------- - so what are the current recommendations for OpenStack Telemetry ? - is it just Ceilometer and Aodh ? * Does Ceilometer provide DBs ? (like it used to a long time ago) - is Gnocchi (although maintained outside of OpenStack) still supported in the OpenStack environment as a DB option ? - what is used for event DB ? ... now that panko is gone ... Gnocchi ? Ceilometer ? - are there actually multiple options/solutions for OpenStack Telemetry ? e.g. * Ceilometer / Gnocchi / Aodh * Ceilometer / External Elastic Solution / Aodh * Others ? Greg.

Hi Greg! On 9/25/24 17:42, Waines, Greg wrote:

HOWEVER ... today (Caracal) ... - I think Gnocchi is still alive and well ... https://github.com/gnocchixyz/gnocchi * but maintained outside of OpenStack

Yeah, but it's still well maintained. The fact it's outside of OpenStack isn't a problem.

- I think OpenStack Panko was deprecated in ~ Wallaby ? * and don't think(?) it is maintained outside of OpenStack ?

Panko is dead, forget about it.

- The OpenStack Landscape at https://www.openstack.org/software/ * shows only Ceilometer and Aodh

Indeed, since Gnocchi isn't OpenStack maintained. There's also cloudkitty in the mix though.

MY QUESTIONs : ----------------------- - so what are the current recommendations for OpenStack Telemetry ? - is it just Ceilometer and Aodh ? * Does Ceilometer provide DBs ? (like it used to a long time ago)

Ceilometer used to have a mongodb backend, but his was a long time ago. This didn't scale and it was removed. So for Ceilometer, there's no client, and no API, and no db either. Ceilometer just plugs on Gnocchi. Gnocchi itself has a few backends. Best is influxdb as a timeseries backend for Gnocchi, though this doesn't support clustering in the free-software version (one has to pay for supporting clustering). So best is to use either galera or postgress for its indexing, and ceph for its timeseries blobs. That's what we use in production, with a dedicated Ceph (so it is always not too busy, and doesn't interfere with the production cluster).

- is Gnocchi (although maintained outside of OpenStack) still supported in the OpenStack environment as a DB option ?

Yes.

- what is used for event DB ? ... now that panko is gone ... Gnocchi ? Ceilometer ?

aodh can do what you want whenever an event on the metrics happen. You have no choice but to use aodh only, and forget about Panko.

- are there actually multiple options/solutions for OpenStack Telemetry ? e.g. * Ceilometer / Gnocchi / Aodh * Ceilometer / External Elastic Solution / Aodh * Others ?

Our setup for our public cloud consist of: - 3 nodes dedicated to rabbitmq for the notification bus (separated from the rest of the messages), and also running gnocchi-api (which can become very busy). - 3 nodes running galera (also separated from the "main" services dbs) - a ceph cluster with 3 mons, and 6 cephosd nodes - an elastic cluster for Cloudkitty to send billing data Note that we used to use the storage v1 for cloudkitty, and this doesn't scale. When you reach 5k+ VMs on your cluster, cloudkitty-processor can't cope with the load, and goes slower than recording metrics. BTW, I was wondering: since windriver is moving toward using Debian everywhere with its derivative, is StarlingX also moving to Debian? Will that use Debian packages for OpenStack? I hope the above helps, Cheers, Thomas Goirand (zigo)

I'm dumping what I know, but some people actually working in the Telemetry project may give more detailed (and probably correct) info.

- so what are the current recommendations for OpenStack Telemetry ? - is it just Ceilometer and Aodh ? * Does Ceilometer provide DBs ? (like it used to a long time ago)

Aodh and Ceilometer are the two projects currently maintained in OpenStack namespace. Ceilometer itself has no db and you need external data store for metrics.

- is Gnocchi (although maintained outside of OpenStack) still supported in the OpenStack environment as a DB option ?

Gnocchi is still supported as the metric store used by ceilometer and aodh. On the other hand, I've seen some work to replace it completely by prometheus these days. # but I've not actually tried it so have limited knowledge about it.

- what is used for event DB ? ... now that panko is gone ... Gnocchi ? Ceilometer ?

I'm not aware of any alternatives. However ceilometer provides the mechansim to implement own publisher so if you need store event data then you can send event data to external services.

- are there actually multiple options/solutions for OpenStack Telemetry ? e.g. * Ceilometer / Gnocchi / Aodh * Ceilometer / External Elastic Solution / Aodh * Others ?

The following two options are ones I'm aware of now, but Ceilometer + Gnocchi + Aodh Ceilometer + Prometheus + Aodh On 9/26/24 00:42, Waines, Greg wrote:

Hey ...

I am working in OpenStack StarlingX which supports a Kubernetes cluster and a containerized OpenStack cluster (via openstack-helm).

Our OpenStack offering is fairly simple ... only running Nova, Cinder, Neutron, Keystone, Glance, Horizon and Heat ... @ Antelope.

In a predecessor of StarlingX, maybe in the Queens(?) timeframe ... we had a Telemetry solution that consisted of: - Ceilometer (for basic data polling, notifications and collection) - Panko (for event DB) - Gnocchi (for time-series DB) - Aodh (for alarming based on metric thresholds, etc.)

HOWEVER ... today (Caracal) ... - I think Gnocchi is still alive and well ... https://github.com/gnocchixyz/gnocchi * but maintained outside of OpenStack - I think OpenStack Panko was deprecated in ~ Wallaby ? * and don't think(?) it is maintained outside of OpenStack ? - The OpenStack Landscape at https://www.openstack.org/software/ * shows only Ceilometer and Aodh

MY QUESTIONs : ----------------------- - so what are the current recommendations for OpenStack Telemetry ? - is it just Ceilometer and Aodh ? * Does Ceilometer provide DBs ? (like it used to a long time ago) - is Gnocchi (although maintained outside of OpenStack) still supported in the OpenStack environment as a DB option ? - what is used for event DB ? ... now that panko is gone ... Gnocchi ? Ceilometer ? - are there actually multiple options/solutions for OpenStack Telemetry ? e.g. * Ceilometer / Gnocchi / Aodh * Ceilometer / External Elastic Solution / Aodh * Others ?

Greg.