Hello CloudKitters!

It was a great opportunity to be able to present this talk for you all guys. It was a pleasure to see that so many people are interested in CloudKitty, Ceilometer, and Gnocchi. I really hope that we can all collaborate together to keep improving CloudKitty, and other related systems.

Attached is the presentation we used during the presentation, so you all can consult this material. Also, as always, if you need some guidance, let us know.

Furthermore, as I mentioned in the presentation, I will provide a step-by-step process on how to create a new metric using Ceilometer Dynamic pollster that can collect/generate/derive (however you want to call it) IOPs information for volumes. The new metric name is going to be called "dynamic_pollster.volume.services.volume.iops". Also, the idea is that we collect some attributes, and we use the attribute called "bootable", which indicates if the volume is bootable or not. Then, if the volume is bootable, we will charge for its IOPS; otherwise, we do not charge it. As I mentioned, this is just a very simple use case. It is not targeting anyone's need in production, but it can be used to get you going quite easily with the pipeline.

Requirements:

• Ceilometer up and running. Version 15.0.0 (or higher is recommended);
• Gnocchi up and running.  Version 4.3.0 (or higher). If you want to run this in production, this patch might be very important (https://github.com/gnocchixyz/gnocchi/pull/1059); it has not been merged yet;
• CloudKitty up and running. Version 10.0.0 (or higher).

With all systems up and running, you will need to do the following:

• Ceilometer

• Add the new metric collection in the "polling.yaml" file. You can add it in any of the sections you already have there. Or, you can create its own, as follows:

• After declaring the metrics there, you can create the pollster in the "pollster.d" folder in Ceilometer. The name of the file can be anything, as long as it ends with ".yaml"; for instance, "pollsters.d/volume-iops.yaml". Then, you will need to add the following content. Remember to replace the contents of "<>" with the values of your environment. 
  
  ```
  - name: "dynamic_pollster.volume.services.volume.iops"
    response_entries_key: "volumes"
    next_sample_url_attribute: "volumes_links | filter(lambda v: v.get('rel') == 'next', value) | list(value) | value[0] | value.get('href')"
    sample_type: "gauge"
    unit: "GB"
    value_attribute: "volume_type"
    endpoint_type: "volume"
    url_path: "/v3/<SERVICE_UUID_FOR_CEILOMETER_USER>/volumes/detail?all_tenants=true&limit=10"
    headers:
      "Openstack-API-Version": "latest"
    project_id_attribute: "os-vol-tenant-attr:tenant_id"
    metadata_fields:
      - "status"
      - "name"
      - "volume_type"
      - "availability_zone"
      - "user_id"
      - "bootable | str(value)"
      - "attachments | value or [ { 'server_id': '' } ] | value[0] | value['server_id']"
    metadata_mapping:
      "availability_zone": "dynamic_availability_zone"
      "name": "dynamic_display_name"
      "volume_type": "dynamic_volume_type"
      "attachments | value or [{ 'server_id': '' }] | value[0] | value['server_id']": "dynamic_instance_id"
      "bootable | str(value)": "bootable"
    value_mapping:
      "<VOLUME_TYPE_UUID_OR_VOLUME_TYPE_NAME>": "<THE IOPS THAT THIS VOLUME TYPE REPRESENTS>"
  
  ```
  
  
  
• After creating that, you will need to customize the file "gnocchi_resources.yaml", which is the file that maps metrics to resources, and attributes that are pushed to the backend. You can also create your own "gnocchi_resources.yaml" file, and then use it in the pipeline configuration in the "pipeline.yaml" file, in the configuration of "publishers". Then, you would have something like "gnocchi://?resources_definition_file=%2Fetc%2Fceilometer%2Fcustom_gnocchi_resources.yaml", assuming the file is called "custom_gnocchi_resources.yaml". In this file, you will need to add the new metric and attribute to be pushed to the metrics backend. The following is an example of "volume" resource declaration in the "gnocchi_resource.yaml" file with the new metric and attribute that we want to capture.
  
  ```
    - resource_type: volume
      metrics:
  <MANY OTHER METRICS HERE>
        dynamic_pollster.volume.services.volume.iops:
      attributes:
        display_name: resource_metadata.(dynamic_display_name|notification_display_name)
        bootable: resource_metadata.bootable
  <MANY OTHER ENTRIES HERE>
     
  ```
  
  
• After the above configuration, you will need to create the new attribute in the Gnocchi resource type. To achieve that, you can issue the following command.
  ```
  gnocchi resource-type update -a bootable:string:false volume
  ```
• Then, you will need to restart the Ceilometer.
• Now, you can issue "gnocchi resource show <VOLUME ID>", and you should see the new metric in the resource, and the new attribute as well.
  
  

With Ceilometer and Gnocchi configured for the new metric, we can move on to CloudKitty.