[aodh][ceilometer] how create alarms with prometheus ceilometer backend
Hello. I try to set up alarm such as: openstack alarm create --type prometheus --type prometheus --name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"}' --threshold 10 I use ceilometer with prometheus backend but when it shows: +---------------------------+------------------------------------------------------------------+ | Field | Value | +---------------------------+------------------------------------------------------------------+ | alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1 | | name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query | memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} | +---------------------------+------------------------------------------------------------------+ and in aodh logs: 2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus I am using openstack 2025.1 I would like to get some guides on this, Many thanks. Nguyen Huu Khoi
Hi, From https://opendev.org/openstack/python-observabilityclient/src/branch/master/o... We need nano /etc/openstack/prometheus.yaml with host: your-prometheus-host.example.com port: your prometheus port on where aodh_evaluator service is running. Nguyen Huu Khoi On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus --name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+---------------------------+------------------------------------------------------------------+ | Field | Value |
+---------------------------+------------------------------------------------------------------+ | alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1 | | name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query | memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} |
+---------------------------+------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
Hello, I would like to know if aodh supports evaluation-periods and period with prometheus query type? Any help would be appreciated. Thank you. Nguyen Huu Khoi On Fri, Sep 19, 2025 at 7:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hi,
From https://opendev.org/openstack/python-observabilityclient/src/branch/master/o...
We need nano /etc/openstack/prometheus.yaml with
host: your-prometheus-host.example.com port: your prometheus port
on where aodh_evaluator service is running.
Nguyen Huu Khoi
On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus --name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+---------------------------+------------------------------------------------------------------+ | Field | Value |
+---------------------------+------------------------------------------------------------------+ | alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1 | | name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query | memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} |
+---------------------------+------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
Hi there, yes it does and it is also tested in CI. Unfortunately I found the documentation for aodh missing https://docs.openstack.org/aodh/latest/admin/telemetry-alarms.html Reading your error message, I would think your version of aodh is not new enough and much older than 2025.1, the code was added 2 years ago https://opendev.org/openstack/aodh/commits/branch/master/aodh/evaluator/prom... Matthias Am 19.09.25 um 16:49 schrieb Nguyễn Hữu Khôi:
Hello,
I would like to know if aodh supports evaluation-periods and period with prometheus query type?
Any help would be appreciated.
Thank you.
Nguyen Huu Khoi
On Fri, Sep 19, 2025 at 7:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hi,
From https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27 <https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27>
We need nano /etc/openstack/prometheus.yaml with
host: your-prometheus-host.example.com <http://your-prometheus- host.example.com> port: your prometheus port
on where aodh_evaluator service is running.
Nguyen Huu Khoi
On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus -- name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a- d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+--------------------------- +------------------------------------------------------------------+ | Field | Value | +--------------------------- +------------------------------------------------------------------+ | alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1 | | name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query | memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} | +--------------------------- +------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
Hi, I am using aodh latest and it is still not working for evaluation-periods and period option. Aodh still evaluates but it follows evaluation_interval in aodh.conf and just evaluator fist time then fire alarm, Nguyen Huu Khoi On Sun, Sep 21, 2025 at 8:05 PM Matthias Runge <mrunge@matthias-runge.de> wrote:
Hi there,
yes it does and it is also tested in CI.
Unfortunately I found the documentation for aodh missing https://docs.openstack.org/aodh/latest/admin/telemetry-alarms.html
Reading your error message, I would think your version of aodh is not new enough and much older than 2025.1, the code was added 2 years ago
https://opendev.org/openstack/aodh/commits/branch/master/aodh/evaluator/prom...
Matthias
Am 19.09.25 um 16:49 schrieb Nguyễn Hữu Khôi:
Hello,
I would like to know if aodh supports evaluation-periods and period with prometheus query type?
Any help would be appreciated.
Thank you.
Nguyen Huu Khoi
On Fri, Sep 19, 2025 at 7:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hi,
From https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27 <https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27>
We need nano /etc/openstack/prometheus.yaml with
host: your-prometheus-host.example.com <http://your-prometheus- host.example.com> port: your prometheus port
on where aodh_evaluator service is running.
Nguyen Huu Khoi
On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus -- name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a- d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+---------------------------
+------------------------------------------------------------------+
| Field | Value | +---------------------------
+------------------------------------------------------------------+
| alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1
|
| name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query | memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"}
|
+---------------------------
+------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
Hello, I see that evaluation-periods and period option just works with "--type gnocchi_resources_threshold" aodh alarm create --name cpu_hi --type gnocchi_resources_threshold --description 'instance running hot' --metric cpu_util --threshold 70.0 --comparison-operator gt --aggregation-method mean --period 60 --evaluation-periods 3 --alarm-action 'log://' --resource-id INSTANCE_ID --resource-type instance +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | alarm_id | b30d83cd-2548-4964-a8fc-6c826bef5233 | | name | cpu_hi | | description | instance running hot | | enabled | True | | ok_actions | [] | | alarm_actions | ['log:'] | | insufficient_data_actions | [] | | repeat_actions | False | | type | gnocchi_resources_threshold | | time_constraints | [] | | project_id | 7fafdc0ed8f84cec9e3248966985533f | | user_id | c9062ecef45646958894c50d95c02ce2 | | timestamp | 2025-09-21T14:14:50.287176 | | state | insufficient data | | state_timestamp | 2025-09-21T14:14:50.287176 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-21T14:14:50.314318 | | metric | cpu_util | | resource_id | INSTANCE_ID | | resource_type | instance | | comparison_operator | gt | | threshold | 70.0 | | aggregation_method | mean | | evaluation_periods | 3 | | granularity | 60 | +---------------------------+--------------------------------------+ It will be right if type prometheus can use two of these options too. Please correct me if it is our design for prometheus. Nguyen Huu Khoi On Sun, Sep 21, 2025 at 8:20 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hi, I am using aodh latest and it is still not working for evaluation-periods and period option. Aodh still evaluates but it follows evaluation_interval in aodh.conf and just evaluator fist time then fire alarm, Nguyen Huu Khoi
On Sun, Sep 21, 2025 at 8:05 PM Matthias Runge <mrunge@matthias-runge.de> wrote:
Hi there,
yes it does and it is also tested in CI.
Unfortunately I found the documentation for aodh missing https://docs.openstack.org/aodh/latest/admin/telemetry-alarms.html
Reading your error message, I would think your version of aodh is not new enough and much older than 2025.1, the code was added 2 years ago
https://opendev.org/openstack/aodh/commits/branch/master/aodh/evaluator/prom...
Matthias
Am 19.09.25 um 16:49 schrieb Nguyễn Hữu Khôi:
Hello,
I would like to know if aodh supports evaluation-periods and period with prometheus query type?
Any help would be appreciated.
Thank you.
Nguyen Huu Khoi
On Fri, Sep 19, 2025 at 7:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hi,
From https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27 <https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27>
We need nano /etc/openstack/prometheus.yaml with
host: your-prometheus-host.example.com <http://your-prometheus- host.example.com> port: your prometheus port
on where aodh_evaluator service is running.
Nguyen Huu Khoi
On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus -- name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a- d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+---------------------------
+------------------------------------------------------------------+
| Field | Value | +---------------------------
+------------------------------------------------------------------+
| alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1
|
| name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True | | ok_actions | [] | | alarm_actions | [] | | insufficient_data_actions | [] | | repeat_actions | False | | type | prometheus | | time_constraints | [] | | project_id | 31ef81adc8ad406e9e91e3eb8776c959 | | user_id | 2e7a252719434d93ae0d270c8753294d | | timestamp | 2025-09-18T14:10:50.467568 | | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384 | | comparison_operator | eq | | threshold | 10.0 | | query |
memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} |
+---------------------------
+------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-] Skipping alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
Hi, after get your explanation, I changed my PromQL to avg_over_time(memory_usage{resource_id="04803e3e-853f-4e9e-b064-59f13242b07a"}[5m:60s]) to apply --period 60 --evaluation-periods 3. Nguyen Huu Khoi On Sun, Sep 21, 2025 at 9:20 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hello,
I see that evaluation-periods and period option just works with "--type gnocchi_resources_threshold"
aodh alarm create --name cpu_hi --type gnocchi_resources_threshold --description 'instance running hot' --metric cpu_util --threshold 70.0 --comparison-operator gt --aggregation-method mean --period 60 --evaluation-periods 3 --alarm-action 'log://' --resource-id INSTANCE_ID --resource-type instance +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | alarm_id | b30d83cd-2548-4964-a8fc-6c826bef5233 | | name | cpu_hi | | description | instance running hot | | enabled | True | | ok_actions | [] | | alarm_actions | ['log:'] | | insufficient_data_actions | [] | | repeat_actions | False | | type | gnocchi_resources_threshold | | time_constraints | [] | | project_id | 7fafdc0ed8f84cec9e3248966985533f | | user_id | c9062ecef45646958894c50d95c02ce2 | | timestamp | 2025-09-21T14:14:50.287176 | | state | insufficient data | | state_timestamp | 2025-09-21T14:14:50.287176 | | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-21T14:14:50.314318 | | metric | cpu_util | | resource_id | INSTANCE_ID | | resource_type | instance | | comparison_operator | gt | | threshold | 70.0 | | aggregation_method | mean | | evaluation_periods | 3 | | granularity | 60 | +---------------------------+--------------------------------------+
It will be right if type prometheus can use two of these options too. Please correct me if it is our design for prometheus.
Nguyen Huu Khoi
On Sun, Sep 21, 2025 at 8:20 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Hi, I am using aodh latest and it is still not working for evaluation-periods and period option. Aodh still evaluates but it follows evaluation_interval in aodh.conf and just evaluator fist time then fire alarm, Nguyen Huu Khoi
On Sun, Sep 21, 2025 at 8:05 PM Matthias Runge <mrunge@matthias-runge.de> wrote:
Hi there,
yes it does and it is also tested in CI.
Unfortunately I found the documentation for aodh missing https://docs.openstack.org/aodh/latest/admin/telemetry-alarms.html
Reading your error message, I would think your version of aodh is not new enough and much older than 2025.1, the code was added 2 years ago
https://opendev.org/openstack/aodh/commits/branch/master/aodh/evaluator/prom...
Matthias
Am 19.09.25 um 16:49 schrieb Nguyễn Hữu Khôi:
Hello,
I would like to know if aodh supports evaluation-periods and period with prometheus query type?
Any help would be appreciated.
Thank you.
Nguyen Huu Khoi
On Fri, Sep 19, 2025 at 7:09 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hi,
From https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27 <https://opendev.org/openstack/python-observabilityclient/src/ branch/master/observabilityclient/utils/metric_utils.py#L27>
We need nano /etc/openstack/prometheus.yaml with
host: your-prometheus-host.example.com <http://your-prometheus- host.example.com> port: your prometheus port
on where aodh_evaluator service is running.
Nguyen Huu Khoi
On Thu, Sep 18, 2025 at 9:25 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com <mailto:nguyenhuukhoinw@gmail.com>> wrote:
Hello.
I try to set up alarm such as:
openstack alarm create --type prometheus --type prometheus -- name memory_high_alarm --query 'memory_usage{resource_id="21d0792e-2d01-4df9-958a- d9018d13207f"}' --threshold 10
I use ceilometer with prometheus backend but when it shows:
+---------------------------
+------------------------------------------------------------------+
| Field | Value | +---------------------------
+------------------------------------------------------------------+
| alarm_id | cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1
|
| name | memory_high_alarm | | description | prometheus alarm rule | | enabled | True
| | ok_actions | []
| | alarm_actions | []
| | insufficient_data_actions | []
| | repeat_actions | False | | type | prometheus
| | time_constraints | []
| | project_id |
31ef81adc8ad406e9e91e3eb8776c959
| | user_id |
2e7a252719434d93ae0d270c8753294d
| | timestamp | 2025-09-18T14:10:50.467568
| | state | *insufficient data * | | state_timestamp | 2025-09-18T14:10:50.467568
| | state_reason | Not evaluated yet | | severity | low | | evaluate_timestamp | 2025-09-18T14:10:50.485384
| | comparison_operator | eq
| | threshold | 10.0
| | query |
memory_usage{resource_id="21d0792e-2d01-4df9-958a-d9018d13207f"} |
+---------------------------
+------------------------------------------------------------------+
and in aodh logs:
2025-09-18 21:20:02.442 1079 INFO aodh.evaluator [-] initiating evaluation cycle on 1 alarms 2025-09-18 21:20:02.442 1079 WARNING aodh.evaluator [-]
Skipping
alarm cb7b8d07-0af1-4771-92c8-1a4c2ce4d1d1, unsupported type: prometheus
I am using openstack 2025.1
I would like to get some guides on this, Many thanks.
Nguyen Huu Khoi
participants (2)
-
Matthias Runge
-
Nguyễn Hữu Khôi