Re: [vitrage][monasca] Monasca datasource for Vitrage
I left Bartosz some comments on his review, and I'll add a few things below. joseph On 12/13/18 12:52 AM, openstack-discuss-request@lists.openstack.org wrote
From: Ifat Afek <ifatafekn@gmail.com> Message-ID: <CALxdkcxxrtYYv+f4-enZ2uuA88qJdPuze732Ufg6MLMADBcOtg@mail.gmail.com>
Hi Witek,
On Wed, Dec 12, 2018 at 5:04 PM Bedyk, Witold <witold.bedyk@est.fujitsu.com> wrote:
There is one issue that we need to close though – how to connect the Monasca alarms to the resources in Vitrage. In order to connect an alarm to a resource, Vitrage should be able to identify that resource in its entity graph using the dimensions coming from Monasca. I understood that these dimensions are configurable and not pre-defined. Correct. To make the picture complete: every alarm contains an associated `metric name` and a set of dimensions which identify the monitored resource. Both metric name and dimensions could potentially be used in Vitrage.
1. For OpenStack services we typically use dimension keys `hostname` and `service`. Examples would be:
`name`: `http_status`, `dimensions`: {`hostname`: `node1`, `service`: `keystone`, `url`: `http://node1/identity` <http://node1/identity>}
Libvirt metrics all have the dimension keys `hostname`, `service`=`compute`, `resource_id`=instance_id
By instance_id do you mean the libvirt id or the OpenStack UUID? In order to correlate the id with the vm that exists in Vitrage, we need the Nova UUID. Another question: are these dimensions names always the same? Is it possible to change their names?
I believe it is the OpenStack id in each case. Dimension names are always the same per type of metric, but different metrics can define different dimensions. For example, the http_status metric has a 'url' but disk.space_used_perc has a 'device' and cpu.idle_perc has neither. All metrics have 'hostname', 'service', and a few other dimensions in common.
2. For other resources I’m not sure if we can define a general rule for assignments. As you have said, the unique information may be included in more the one dimension.
For Prometheus, we use labels on each metric as dimensions. Additional dimensions can be configured in agent per Prometheus endpoint.
We also have a monasca-ceilometer component (aka. Ceilosca) that can send metrics from Ceilometer to Monasca. These metrics include the OpenStack resource_id of the source of the metric. Any alarms created from Ceilometer data can carry through that resource_id. From what I've seen we don't currently include a resource_type, but that should be something we can configure in the metric. If nothing else, we may be able to statically define the resource_type as part of the metric definition (each alarm will be specific to the type of resource anyway). But don't hold me to that - I have some more reading to do to understand the nuances. :) The dimensions from the metrics are carried over to the alarms. <snip> Can you give us some more examples of Monasca dimensions?
As Witek said, the metric dimensions are pretty flexible. You can see some of the dimensions mentioned in my answer above. I can try to pull a list of some standard alarm definitions and associated metrics from a SUSE install, or if we have a particular few alarms we want to start with then we can focus on those.
Hi Joseph, Thanks for your comments. What I understand is that we can always assume that Vitrage gets from Monasca enough dimensions to identify the resource in Vitrage, and the challenge is to define a mechanism for correlating the Monasca dimensions with Vitrage resource properties. I thought of a possible solution. We can write in Vitrage a configuration file, which defines for every metric which dimension(s) from Monsaca should be mapped to which property in Vitrage in order to uniquely identify the resource. For example, for libvirt metrics, ‘resource_id’ dimension should be mapped to ‘id’ in Vitrage. A draft of the file structure: * metric_name: the name of the metric in Monasca * vitrage_property: the name of a property to match in the Vitrage resource * vitrage_property_value: the value to match based on the dimensions coming from Monasca For example: config: - metric_name: low_disk_space vitrage_property: id vitrage_property_value: ${resource_id} # resource_id is the dimension of libvirt metrics - metric_name: low_disk_space vitrage_property: id vitrage_property_value: ${hosname} # in case the disk_space is monitored for a host (?) - metric_name: http_status vitrage_property: service_id vitrage_property_value: ${hostname}/${service} - metric_name: nic_status vitrage_property: id vitrage_property_value: ${hostname}/${nic_name} - … What do you think? Can a metric like low_disk_space be set both for an instance (with a resource_id that is uuid) and for a host (with hostname)? How can we tell which dimension to use? Do you think that this configuration can solve the POC use case? Ifat On Sat, Dec 15, 2018 at 5:50 AM Joseph Davis <joseph.davis@suse.com> wrote:
I left Bartosz some comments on his review, and I'll add a few things below.
joseph
On 12/13/18 12:52 AM, openstack-discuss-request@lists.openstack.org wrote
From: Ifat Afek <ifatafekn@gmail.com> Message-ID: < CALxdkcxxrtYYv+f4-enZ2uuA88qJdPuze732Ufg6MLMADBcOtg@mail.gmail.com>
Hi Witek,
On Wed, Dec 12, 2018 at 5:04 PM Bedyk, Witold < witold.bedyk@est.fujitsu.com> wrote:
There is one issue that we need to close though – how to connect the Monasca alarms to the resources in Vitrage. In order to connect an alarm to a resource, Vitrage should be able to identify that resource in its entity graph using the dimensions coming from Monasca. I understood that these dimensions are configurable and not pre-defined. Correct. To make the picture complete: every alarm contains an associated `metric name` and a set of dimensions which identify the monitored resource. Both metric name and dimensions could potentially be used in Vitrage.
1. For OpenStack services we typically use dimension keys `hostname` and `service`. Examples would be:
`name`: `http_status`, `dimensions`: {`hostname`: `node1`, `service`: `keystone`, `url`: `http://node1/identity` <http://node1/identity> <http://node1/identity>}
Libvirt metrics all have the dimension keys `hostname`, `service`=`compute`, `resource_id`=instance_id
By instance_id do you mean the libvirt id or the OpenStack UUID? In order to correlate the id with the vm that exists in Vitrage, we need the Nova UUID. Another question: are these dimensions names always the same? Is it possible to change their names?
I believe it is the OpenStack id in each case.
Dimension names are always the same per type of metric, but different metrics can define different dimensions.
For example, the http_status metric has a 'url' but disk.space_used_perc has a 'device' and cpu.idle_perc has neither. All metrics have 'hostname', 'service', and a few other dimensions in common.
2. For other resources I’m not sure if we can define a general rule for assignments. As you have said, the unique information may be included in more the one dimension.
For Prometheus, we use labels on each metric as dimensions. Additional dimensions can be configured in agent per Prometheus endpoint.
We also have a monasca-ceilometer component (aka. Ceilosca) that can send metrics from Ceilometer to Monasca. These metrics include the OpenStack resource_id of the source of the metric. Any alarms created from Ceilometer data can carry through that resource_id. From what I've seen we don't currently include a resource_type, but that should be something we can configure in the metric. If nothing else, we may be able to statically define the resource_type as part of the metric definition (each alarm will be specific to the type of resource anyway). But don't hold me to that - I have some more reading to do to understand the nuances. :) The dimensions from the metrics are carried over to the alarms. <snip> Can you give us some more examples of Monasca dimensions?
As Witek said, the metric dimensions are pretty flexible. You can see some of the dimensions mentioned in my answer above. I can try to pull a list of some standard alarm definitions and associated metrics from a SUSE install, or if we have a particular few alarms we want to start with then we can focus on those.
participants (2)
-
Ifat Afek
-
Joseph Davis