[openstack-dev] [Ceilometer]:Subscribe and Publish Notification frame work in Ceilometer !

Srikanth Vavilapalli srikanth.vavilapalli at ericsson.com
Mon Nov 23 03:04:08 UTC 2015


Thanks very much Gord for the pointers.

I have looked at this file reload feature and it addresses some part of my problem description (avoiding the restart of the ceilometer services to reflect the pipeline changes). However, this still requires users to directly access the internal ceilometer file (pipeline.yaml) to make any updates to pipeline. There are no APIs introduced as part of this feature to allow user to configure/unconfigure these pipeline. In one of the deployment that we are currently looking for, we wanted to expose only ceilometer API service to the users and block access to any internal modules of ceilometer. In the same deployment, we are also thinking of allowing users to configure/unconfigure a pipeline for a set of meters for only a subset of resources instead of getting notified about events/meters of all the resources as shown below.

sources:
    - name: user1_meter_source
      meters:
          - "cpu_util"
      query/filter:
          - {{field:"project_id",op:"eq",value:"111111"}}
      sinks:
          - user1_meter_sink
sinks:
    - name: user1_meter_sink
      transformers:
      publishers:
          - udp://<IP>:<port>


We have found a related blueprint [1], which is exactly addressing this dynamic provisioning of pipelines via an API and making use of a new data store to store this configuration. But seems like that blueprint was abandoned due its limited usage. We agree that it may not be that frequent operation to change the pipeline configuration in all deployments, but in certain deployments, where applications desire to turn ON/OFF receiving these notifications based on some other trigger (like anomaly detection, where application might have sensed an anomaly based on some set of meters and wanted to turn ON few additional set of event/meters to confirm that and turn OFF once the anomaly is confirmed).

Thanks
Srikanth

[1] https://wiki.openstack.org/wiki/Ceilometer/blueprints/Configuration-via-data-store



From: gord chung [mailto:gord at live.ca]
Sent: Thursday, November 19, 2015 9:08 PM
To: openstack-dev at lists.openstack.org
Subject: Re: [openstack-dev] [Ceilometer]:Subscribe and Publish Notification frame work in Ceilometer !


On 19/11/15 10:26 PM, Srikanth Vavilapalli wrote:
Hi Gord

On your second point, Yes, Ceilometer does provide a framework to capture a notification and republish to multiple "publish targets" in addition to the collector service using udp/kafka/notification as the transport mechanisms... We believe this is how "Event Alarm Evaluator" module in Aodh project get notified directly from Notification Agents.

However seems like the configuration of these additional "publish targets" is supported only through updating the pipeline_cfg_file and restarting the corresponding ceilometer services. i.e. the users need to manually update the pipeline config files to insert their "publish targets" in the sink-publisher configuration for a set of event filters of their interest. This type of provisioning is very static.

As per our understanding, ceilometer currently does not provide means for users to dynamically register/unregister their "publish targets" with ceilometer framework for a subset events of their interest? i.e User invokes a ceilometer API with a set of event filters and associated publish targets, that can be stored in a data store, which will eventually be used by the ceilometer Publisher to dispatch the notification to those configured destinations in addition to the statically configured "publish targets". Plz let us know if our understanding is wrong or if there are any other means to achieve the above functionality. We believe this as a very key functionality needed to build latency sensitive (sub-second) analytics application on-top of ceilometer framework. We are seeking the feedback from community on having this kind of functionality inside ceilometer before proceeding with blueprint submission.
actually, in Liberty you can configure a reload[1][2] to refresh pipeline when a change happens. based on survey results, most operators don't need this functionality so it's off by default but it seems you do. :)

[1] https://specs.openstack.org/openstack/ceilometer-specs/specs/liberty/reload-file-based-pipeline-configuration.html
[2] https://github.com/openstack/ceilometer/blob/master/ceilometer/pipeline.py#L50-L58

cheers,



--

gord
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151123/ff5cd9a5/attachment.html>


More information about the OpenStack-dev mailing list