I am not understanding. Your expression to obtain the value is ". | value['usage'] | value.get('rgw.main',{'size':0}) | value['size']". That assumes a response with a "value" entry in the JSON; then, you get the 'rgw.main' attribute, and then, you get the size.



So, you are working with samples that are similar to the following:
```
{
   "bucket":"huge",
   "num_shards":11,
   "tenant":"08bb8ee9c5bd41248025268ee1aea481",
   "zonegroup":"d28c435f-57a5-49ca-91e8-481a2ced1f18",
   "placement_rule":"default-placement",
   "explicit_placement":{
      "data_pool":"",
      "data_extra_pool":"",
      "index_pool":""
   },
   "id":"ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3",
   "marker":"ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3",
   "index_type":"Normal",
   "owner":"08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481",
   "ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#2,8#1,9#1,10#1",
   "master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0",
   "mtime":"2022-10-26T11:32:05.185527Z",
   "creation_time":"2022-10-26T11:32:05.181022Z",
   "max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#",
   "usage":{
      "rgw.main":{
         "size":8461984,
         "size_actual":8462336,
         "size_utilized":8461984,
         "size_kb":8264,
         "size_kb_actual":8264,
         "size_kb_utilized":8264,
         "num_objects":1
      }
   },
   "bucket_quota":{
      "enabled":false,
      "check_on_raw":true,
      "max_size":-1,
      "max_size_kb":0,
      "max_objects":-1
   }
}
```

These samples are probably coming from an API with "usage" JSON attribute, and that is why it works with your expression.

To answer your initial question, Ceilometer dynamic pollster will work with whatever you have in the response. If data comes in a bucket fashion, each sample is going to represent a bucket. If you want to group/aggregate that data in a project/tenant fashion you might need to do some working. Either using a different API, or doing some groupby in Gnocchi with the aggregates API.

Furthermore, what about the "admin/usage" instead of the "admin/bucket?stats=True" . The admin API will bring data grouped in a user fashion.

On Fri, Oct 28, 2022 at 9:46 AM Taltavull Jean-François <jean-francois.taltavull@elca.ch> wrote:

See below. Hope this will help !

 

[[{'bucket': 'huge', 'num_shards': 11, 'tenant': '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup': 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3', 'marker': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3', 'index_type': 'Normal', 'owner': '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#2,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2022-10-26T11:32:05.185527Z', 'creation_time': '2022-10-26T11:32:05.181022Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 8461984, 'size_actual': 8462336, 'size_utilized': 8461984, 'size_kb': 8264, 'size_kb_actual': 8264, 'size_kb_utilized': 8264, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}, {'bucket': 'empty', 'num_shards': 11, 'tenant': '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup': 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27142035.4', 'marker': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27142035.4', 'index_type': 'Normal', 'owner': '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2022-10-26T11:31:40.229337Z', 'creation_time': '2022-10-26T11:31:40.224401Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}, {'bucket': 'photos', 'num_shards': 11, 'tenant': '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup': 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.2', 'marker': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.2', 'index_type': 'Normal', 'owner': '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver': '0#2,1#1,2#1,3#3,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2022-10-24T11:54:18.320141Z', 'creation_time': '2022-10-24T11:54:18.315194Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 14, 'size_actual': 4096, 'size_utilized': 14, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}, {'bucket': 'big', 'num_shards': 11, 'tenant': '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup': 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27100595.1', 'marker': 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27100595.1', 'index_type': 'Normal', 'owner': '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver': '0#2,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2022-10-24T13:28:45.864925Z', 'creation_time': '2022-10-24T13:28:45.860346Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 249, 'size_actual': 4096, 'size_utilized': 249, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}]

 

 

From: Rafael Weingärtner <rafaelweingartner@gmail.com>
Sent: vendredi, 28 octobre 2022 12:27
To: Taltavull Jean-François <jean-francois.taltavull@elca.ch>
Cc: openstack-discuss <openstack-discuss@lists.openstack.org>
Subject: Re: [Ceilometer] RADOS GW metrics : cannot get radosgw.objects.size metric

 

 

EXTERNAL MESSAGE - This email comes from outside ELCA companies.

Can you show us the json you are trying to process with Ceilometer? Then,we can move on from there. You can post here a minimalistic version of the json output.

 

On Fri, Oct 28, 2022 at 4:32 AM Taltavull Jean-François <jean-francois.taltavull@elca.ch> wrote:

Hello,

I can ask the question another way: what's the difference between 'radosgw.containers.objects.size' and 'radosgw.objects.size' metrics ?

Thanks,

JF

> -----Original Message-----
> From: Taltavull Jean-François
> Sent: lundi, 24 octobre 2022 16:26
> To: openstack-discuss <openstack-discuss@lists.openstack.org>
> Subject: [Ceilometer] RADOS GW metrics : cannot get radosgw.objects.size
> metric
>
> Hello,
>
> I'm trying to get the 'radosgw.objects.size' metric, that is the total bucket
> objects size per tenant. I expected to get one sample per tenant but I get one
> sample per bucket instead, as with the 'rados.containers.objects.size' metric.
>
> Here is my pollster definition:
> '''
> - name: "radosgw.objects.size"
>   sample_type: "gauge"
>   unit: "B"
>   value_attribute: ". | value['usage'] | value.get('rgw.main',{'size':0}) |
> value['size']"
>   url_path: "FQDN/admin/bucket?stats=True"
>   module: "awsauth"
>   authentication_object: "S3Auth"
>   authentication_parameters: my_access_key,my_secret_key,FQDN
>   user_id_attribute: "owner | value.split('$') | value[0]"
>   project_id_attribute: "tenant"
>   resource_id_attribute: "id"
> '''
>
> I tried with "resource_id_attribute: "tenant" but it does not work better.
>
> Any idea ? Is there something wrong in the pollster definition ?
>
> Regards,
> Jean-Francois



--

Rafael Weingärtner



--
Rafael Weingärtner