Open Stack

Juha,

Your problem is the embedded period in the metadata key: "metering.server_group"

If the metric were gathered by ceilometer itself in the usual way, then the
compute agent would transform that problematic payload as follows, from:

{ ..., "resource_metadata" : { "AutoScalingGroupName": "tykyauto-Group_B-hmknsgn35efz", "metering.server_group": "Group_B" }, ... }

to:

{ ..., "resource_metadata" : { "AutoScalingGroupName": "tykyauto-Group_B-hmknsgn35efz", "user_metadata": {"server_group": "Group_B"} }, ... }

You should follow the same pattern.

Cheers,
Eoghan

----- Original Message -----
> Maybe this is because I didn't fill in instance related metadata (scaling
> group name and such) to the REST call I made when adding custom metric data
> to Ceilometer. I tried to create metric data again, now with metadata
> filled:
> 
> $ curl -X POST -H 'X-Auth-Token: 0722fcd0f403425bb8564808c37e8dc8' -H
> 'Content-Type: application/json' -d '[ { "counter_name": "vm_cpu_load",
> "resource_id": "e7eaf484-38b6-4689-8490-40aa8f0df8ae", "resource_metadata" :
> { "AutoScalingGroupName": "tykyauto-Group_B-hmknsgn35efz",
> "metering.server_group": "Group_B" }, "counter_unit": "%", "counter_volume":
> 11, "counter_type": "gauge" } ]'
> http://192.168.100.5:8777/v2/meters/vm_cpu_load
> 
> ...but as a result I can see the following error in ceilometer log:
> 
> <43>Apr 3 14:24:01 node-6 ceilometer-ceilometer.collector.dispatcher.database
> ERROR: Failed to record metering data: not okForStorage
> Traceback (most recent call last):
> File
> "/usr/lib/python2.7/dist-packages/ceilometer/collector/dispatcher/database.py",
> line 65, in record_metering_data
> self.storage_conn.record_metering_data(meter)
> File "/usr/lib/python2.7/dist-packages/ceilometer/storage/impl_mongodb.py",
> line 451, in record_metering_data
> upsert=True,
> File "/usr/lib/python2.7/dist-packages/pymongo/collection.py", line 487, in
> update
> check_keys, self.__uuid_subtype), safe)
> File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 969, in
> _send_message
> rv = self.__check_response_to_last_error(response)
> File "/usr/lib/python2.7/dist-packages/pymongo/mongo_client.py", line 911, in
> __check_response_to_last_error
> raise OperationFailure(details["err"], details["code"])
> OperationFailure: not okForStorage
> 
> Hmm. What am I doing wrong here?
> 
> Thanks,
> -Juha
> 
> 
> 
> 
> On 2 April 2014 14:04, Juha Tynninen < juha.tynninen at tieto.com > wrote:
> 
> 
> 
> Hi,
> 
> I'm sending custom Ceilometer metrics from inside VM instance with REST call
> to
> http://192.168.100.5:8777/v2/meters/vm_cpu_load .
> 
> This is successful and I can see the entered metric data with Ceilometer:
> 
> # ceilometer sample-list -m vm_cpu_load
> -q="resource_id=91951d0a-9a43-4894-99fb-ac67a1098771" | tail -n +4 | head -n
> -1 | sort -k 12
> ...
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 2.6 | % |
> | 2014-03-30T19:20:38.080000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 3.3 | % |
> | 2014-03-30T19:20:58.223000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 2.6 | % |
> | 2014-03-30T19:21:18.078000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 28.6 | % |
> | 2014-03-30T19:21:38.894000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 1.0 | % |
> | 2014-03-30T19:21:59.370000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 2.3 | % |
> | 2014-03-30T19:22:20.255000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 0.3 | % |
> | 2014-03-30T19:22:40.351000 |
> | 91951d0a-9a43-4894-99fb-ac67a1098771 | vm_cpu_load | gauge | 1.9 | % |
> | 2014-03-30T19:23:00.317000 |
> 
> # ceilometer meter-list | grep vm_cpu_load | grep
> 91951d0a-9a43-4894-99fb-ac67a1098771
> | vm_cpu_load | gauge | % | 91951d0a-9a43-4894-99fb-ac67a1098771 |
> | 2884e2f624224227bb63d77a040126e6 | a12aee6f0da04d8d976eb4b761a73e14 |
> 
> I've started the instance with a Heat template having AutoScaling defined and
> I'm trying to base the scaling actions to this custom metric.
> The problem is that the autoscaling does not occur.
> 
> "Resources" : {
> 
> "Group_B" : {
> "Type" : "AWS::AutoScaling::AutoScalingGroup",
> "Properties" : {
> "AvailabilityZones" : { "Fn::GetAZs" : ""},
> "LaunchConfigurationName" : { "Ref" : "Group_B_Config" },
> "MinSize" : "1",
> "MaxSize" : "3",
> "Tags" : [
> { "Key" : "metering.server_group", "Value" : "Group_B" }
> ],
> "VPCZoneIdentifier" : [ { "Ref" : "Private Application Subnet ID" } ]
> }
> },
> ...
> "Group_B_Config" : {
> "Type" : "AWS::AutoScaling::LaunchConfiguration",
> "Properties": {
> "ImageId" : { "Ref" : "Image Id" },
> "InstanceType" : { "Ref" : "Instance Type" },
> "KeyName" : { "Ref" : "Key Name" }
> }
> },
> ...
> "CPUAlarmHigh": {
> "Type": "OS::Ceilometer::Alarm",
> "Properties": {
> "description": "Scale-up if CPU is greater than 80% for 60 seconds",
> "meter_name": "vm_cpu_load",
> "statistic": "avg",
> "period": "60",
> "evaluation_periods": "1",
> "threshold": "80",
> "alarm_actions":
> [ {"Fn::GetAtt": ["ScaleUpPolicy", "AlarmUrl"]} ],
> "matching_metadata":
> {"metadata.user_metadata.server_group": "Group_B" },
> "comparison_operator": "gt",
> "repeat_actions" : true
> }
> },
> ...
> nova show 91951d0a-9a43-4894-99fb-ac67a1098771
> ...
> | metadata | { u'AutoScalingGroupName': u'tykyauto-Group_B-76nubm24bnf6',
> | u'metering.server_group': u'Group_B'} |
> 
> For some reason the statistics query does not return anything when queried
> with the scaling group name, this probably explains why auto scaling actions
> are not triggered...? Without query parameter data is returned. Data is
> returned also ok for some other counter e.g. for cpu_util.
> 
> # ceilometer statistics -m vm_cpu_load -q
> metadata.user_metadata.server_group=Group_B -p 60
> 
> # ceilometer statistics -m vm_cpu_load
> +--------+----------------------------+----------------------------+-------+-----+-------+--------+---------------+------------+----------------------------+----------------------------+
> | Period | Period Start | Period End | Count | Min | Max | Sum | Avg |
> | Duration | Duration Start | Duration End |
> +--------+----------------------------+----------------------------+-------+-----+-------+--------+---------------+------------+----------------------------+----------------------------+
> | 0 | 2014-03-28T21:14:34.370000 | 2014-03-28T21:14:34.370000 | 520 | 0.3 |
> | 100.0 | 5865.5 | 11.2798076923 | 170135.609 | 2014-03-28T21:14:34.370000 |
> | 2014-03-30T20:30:09.979000 |
> +--------+----------------------------+----------------------------+-------+-----+-------+--------+---------------+------------+----------------------------+----------------------------+
> 
> Any ideas what might be the cause for this behaviour...?
> 
> Many thanks,
> -Juha
> 
> 
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>