> watcher, aodh, and cloud kitty i believe all  have some level if support for
> Prometheus but they can also use other backends. in not sure what level of enablement they have in osa.

So I think we will test them both with Gnocchi as a backend as of today. I believe the operator is able to flip the backend, but must have a pre-configured prometheus, as Prometheus deployment is out of OSA scope as of today.

Having a scrape frontend for Prometheus might be a solution, though I kinda wonder how it's gonna work and where data is gonna be stored in the meanwhile (or in case of networking issues).
As still, I believe the timestamp of metrics will be at the moment of scrape from the proxy. So in case Prometheus is not able to reach the proxy - no "billable" data is received. Thus it does not conceptually solve the issue of Prometheus not being the best choice when it comes to some "billing" implementation.  As matching of data "issue" timestamp with data "stored" timestamp is crucial there, IMO.

чт, 17 апр. 2025 г. в 17:04, <thomas@goirand.fr>:

On Apr 17, 2025 16:49, Sean Mooney <smooney@redhat.com> wrote:
>
>
> On 17/04/2025 13:17, Dmitriy Rabotyagov wrote:
> >> well gnocchi is also not a native OpenStack telemetry datastore, it left
> >> our community to pursue its own goals and is now a third party datastore
> >> just like Grafana or Prometheus.
> > Yeah, well, true. Is still somehow treated as the "default" thing with
> > Telemetry, likely due to existing integration with Keystone and
> > multi-tenancy support. And beyond it - all other options become
> > opinionated too fast - ie, some do OpenTelemetry, some do Zabbix,
> > VictoriaMetrics, etc. As pretty much from what I got as well, is that
> > still relies on Ceilometer metrics?
> > And then Prometheus is obviously not the best storage for them, as it
> > requires to have pushgatgeway, and afaik prometheus maintainers are
> > strictly against "push" concept to it and treat it as conceptually
> > wrong (on contrary to OpenTelemetry).
>
> i dont know the detail but i know there is work planned for native
> supprot of
> Prometheus scrpe endpoint in ceilometer

That is what the telemetry folks discussed during the last online PTG yes. If I understand correctly, the major issue is that prometheus does not support multi-tenancy. Telemetry team hope to add the feature by writing a proxy in front of it.

This would be nice, as Prometheus performs a way better than Gnocchi (it can eat 100 times more metrics per seconds).

Cheers,

Thomas Goirand (zigo)