Open Stack

I am not understanding. Your expression to obtain the value is ". |
value['usage'] | value.get('rgw.main',{'size':0}) | value['size']". That
assumes a response with a "value" entry in the JSON; then, you get the
'rgw.main' attribute, and then, you get the size.



So, you are working with samples that are similar to the following:
```
{
   "bucket":"huge",
   "num_shards":11,
   "tenant":"08bb8ee9c5bd41248025268ee1aea481",
   "zonegroup":"d28c435f-57a5-49ca-91e8-481a2ced1f18",
   "placement_rule":"default-placement",
   "explicit_placement":{
      "data_pool":"",
      "data_extra_pool":"",
      "index_pool":""
   },
   "id":"ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3",
   "marker":"ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3",
   "index_type":"Normal",

 "owner":"08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481",
   "ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#2,8#1,9#1,10#1",
   "master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0",
   "mtime":"2022-10-26T11:32:05.185527Z",
   "creation_time":"2022-10-26T11:32:05.181022Z",
   "max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#",
   "usage":{
      "rgw.main":{
         "size":8461984,
         "size_actual":8462336,
         "size_utilized":8461984,
         "size_kb":8264,
         "size_kb_actual":8264,
         "size_kb_utilized":8264,
         "num_objects":1
      }
   },
   "bucket_quota":{
      "enabled":false,
      "check_on_raw":true,
      "max_size":-1,
      "max_size_kb":0,
      "max_objects":-1
   }
}
```

These samples are probably coming from an API with "usage" JSON attribute,
and that is why it works with your expression.

To answer your initial question, Ceilometer dynamic pollster will work with
whatever you have in the response. If data comes in a bucket fashion, each
sample is going to represent a bucket. If you want to group/aggregate that
data in a project/tenant fashion you might need to do some working. Either
using a different API, or doing some groupby in Gnocchi with the aggregates
API.

Furthermore, what about the "admin/usage" instead of the
"admin/bucket?stats=True" . The admin API will bring data grouped in a user
fashion.

On Fri, Oct 28, 2022 at 9:46 AM Taltavull Jean-François <
jean-francois.taltavull at elca.ch> wrote:

> See below. Hope this will help !
>
>
>
> [[{'bucket': 'huge', 'num_shards': 11, 'tenant':
> '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup':
> 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule':
> 'default-placement', 'explicit_placement': {'data_pool': '',
> 'data_extra_pool': '', 'index_pool': ''}, 'id':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3', 'marker':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.3', 'index_type': 'Normal',
> 'owner':
> '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver':
> '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#2,8#1,9#1,10#1', 'master_ver':
> '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime':
> '2022-10-26T11:32:05.185527Z', 'creation_time':
> '2022-10-26T11:32:05.181022Z', 'max_marker':
> '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size':
> 8461984, 'size_actual': 8462336, 'size_utilized': 8461984, 'size_kb': 8264,
> 'size_kb_actual': 8264, 'size_kb_utilized': 8264, 'num_objects': 1}},
> 'bucket_quota': {'enabled': False, 'check_on_raw': True, 'max_size': -1,
> 'max_size_kb': 0, 'max_objects': -1}}, {'bucket': 'empty', 'num_shards':
> 11, 'tenant': '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup':
> 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule':
> 'default-placement', 'explicit_placement': {'data_pool': '',
> 'data_extra_pool': '', 'index_pool': ''}, 'id':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27142035.4', 'marker':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27142035.4', 'index_type': 'Normal',
> 'owner':
> '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver':
> '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver':
> '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime':
> '2022-10-26T11:31:40.229337Z', 'creation_time':
> '2022-10-26T11:31:40.224401Z', 'max_marker':
> '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota':
> {'enabled': False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0,
> 'max_objects': -1}}, {'bucket': 'photos', 'num_shards': 11, 'tenant':
> '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup':
> 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule':
> 'default-placement', 'explicit_placement': {'data_pool': '',
> 'data_extra_pool': '', 'index_pool': ''}, 'id':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.2', 'marker':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27108481.2', 'index_type': 'Normal',
> 'owner':
> '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver':
> '0#2,1#1,2#1,3#3,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver':
> '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime':
> '2022-10-24T11:54:18.320141Z', 'creation_time':
> '2022-10-24T11:54:18.315194Z', 'max_marker':
> '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 14,
> 'size_actual': 4096, 'size_utilized': 14, 'size_kb': 1, 'size_kb_actual':
> 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled':
> False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0,
> 'max_objects': -1}}, {'bucket': 'big', 'num_shards': 11, 'tenant':
> '08bb8ee9c5bd41248025268ee1aea481', 'zonegroup':
> 'd28c435f-57a5-49ca-91e8-481a2ced1f18', 'placement_rule':
> 'default-placement', 'explicit_placement': {'data_pool': '',
> 'data_extra_pool': '', 'index_pool': ''}, 'id':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27100595.1', 'marker':
> 'ba604862-46ad-4cf1-a554-7da4e7168ac3.27100595.1', 'index_type': 'Normal',
> 'owner':
> '08bb8ee9c5bd41248025268ee1aea481$08bb8ee9c5bd41248025268ee1aea481', 'ver':
> '0#2,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver':
> '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime':
> '2022-10-24T13:28:45.864925Z', 'creation_time':
> '2022-10-24T13:28:45.860346Z', 'max_marker':
> '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 249,
> 'size_actual': 4096, 'size_utilized': 249, 'size_kb': 1, 'size_kb_actual':
> 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled':
> False, 'check_on_raw': True, 'max_size': -1, 'max_size_kb': 0,
> 'max_objects': -1}}]
>
>
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* vendredi, 28 octobre 2022 12:27
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] RADOS GW metrics : cannot get
> radosgw.objects.size metric
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> Can you show us the json you are trying to process with Ceilometer?
> Then,we can move on from there. You can post here a minimalistic version of
> the json output.
>
>
>
> On Fri, Oct 28, 2022 at 4:32 AM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Hello,
>
> I can ask the question another way: what's the difference between
> 'radosgw.containers.objects.size' and 'radosgw.objects.size' metrics ?
>
> Thanks,
>
> JF
>
> > -----Original Message-----
> > From: Taltavull Jean-François
> > Sent: lundi, 24 octobre 2022 16:26
> > To: openstack-discuss <openstack-discuss at lists.openstack.org>
> > Subject: [Ceilometer] RADOS GW metrics : cannot get radosgw.objects.size
> > metric
> >
> > Hello,
> >
> > I'm trying to get the 'radosgw.objects.size' metric, that is the total
> bucket
> > objects size per tenant. I expected to get one sample per tenant but I
> get one
> > sample per bucket instead, as with the 'rados.containers.objects.size'
> metric.
> >
> > Here is my pollster definition:
> > '''
> > - name: "radosgw.objects.size"
> >   sample_type: "gauge"
> >   unit: "B"
> >   value_attribute: ". | value['usage'] |
> value.get('rgw.main',{'size':0}) |
> > value['size']"
> >   url_path: "FQDN/admin/bucket?stats=True"
> >   module: "awsauth"
> >   authentication_object: "S3Auth"
> >   authentication_parameters: my_access_key,my_secret_key,FQDN
> >   user_id_attribute: "owner | value.split('$') | value[0]"
> >   project_id_attribute: "tenant"
> >   resource_id_attribute: "id"
> > '''
> >
> > I tried with "resource_id_attribute: "tenant" but it does not work
> better.
> >
> > Any idea ? Is there something wrong in the pollster definition ?
> >
> > Regards,
> > Jean-Francois
>
>
>
> --
>
> Rafael Weingärtner
>


-- 
Rafael Weingärtner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221028/9cd6df4a/attachment.htm>