Open Stack

Jean, there are two problems with the Ceilometer. I just opened the patches
to resolve it:
- https://review.opendev.org/c/openstack/ceilometer/+/856305
- https://review.opendev.org/c/openstack/ceilometer/+/856304

Without these patches, you might have problems to use Ceilometer with
Non-OpenStack dynamic pollsters and barbican credentials.

On Wed, Aug 31, 2022 at 3:55 PM Rafael Weingärtner <
rafaelweingartner at gmail.com> wrote:

> It is the RGW user that you have. This user must have the role that is
> needed to access the usage feature in RGW. If I am not mistaken, it
> required an admin user.
>
> On Wed, Aug 31, 2022 at 1:54 PM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
>> Thanks to your help, I am close to the goal. Dynamic pollster is loaded
>> and triggered.
>>
>>
>>
>> But I get a “Status[403] and reason [Forbidden]” in ceilometer logs while
>> requesting admin/usage.
>>
>>
>>
>> I’m not sure to understand well the auth mechanism. Are we talking about
>> keystone credentials, ec2 credentials, Rados GW user ?...
>>
>>
>>
>> For now, in testing phase, I use “authentication_parameters”, not
>> barbican.
>>
>>
>>
>> -JF
>>
>>
>>
>> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
>> *Sent:* mardi, 30 août 2022 14:17
>> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
>> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
>> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
>> endpoints are based on URL instead of port number
>>
>>
>>
>>
>>
>> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>>
>> Yes, you will need to enable the metric/pollster to be processed. That is
>> done via "polling.yml" file. Also, do not forget that you will need to
>> configure Ceilometer to push this new metric. If you use Gnocchi as the
>> backend, you will need to change/update the gnocchi resource YML file. That
>> file maps resources and metrics in the Gnocchi backend. The configuration
>> resides in Ceilometer. You can create/define new resource types and map
>> them to specific metrics. It depends on how you structure your solution.
>>
>> P.S. You do not need to use "authentication_parameters". You can use the
>> barbican integration to avoid setting your credentials in a file.
>>
>>
>>
>> On Tue, Aug 30, 2022 at 9:11 AM Taltavull Jean-François <
>> jean-francois.taltavull at elca.ch> wrote:
>>
>> Hello,
>>
>>
>>
>> I tried to define a Rados GW dynamic pollster and I can see, in
>> Ceilometer logs, that it’s actually loaded. But it looks like it was not
>> triggered, I see no trace of ceilometer connection in Rados GW logs.
>>
>>
>>
>> My definition:
>>
>>
>>
>> - name: "dynamic.radosgw.usage"
>>
>>   sample_type: "gauge"
>>
>>   unit: "B"
>>
>>   value_attribute: "total.size"
>>
>>   url_path: http://<FQDN>/object-store/swift/v1/admin/usage
>>
>>   module: "awsauth"
>>
>>   authentication_object: "S3Auth"
>>
>>   authentication_parameters: xxxxxxxxxxxxx,yyyyyyyyyyyyy,<FQDN>
>>
>>   user_id_attribute: "admin"
>>
>>   project_id_attribute: "admin"
>>
>>   resource_id_attribute: "admin"
>>
>>   response_entries_key: "summary"
>>
>>
>>
>> Do I have to set an option in ceilometer.conf, or elsewhere, to get my
>> Rados GW dynamic pollster triggered ?
>>
>>
>>
>> -JF
>>
>>
>>
>> *From:* Taltavull Jean-François
>> *Sent:* lundi, 29 août 2022 18:41
>> *To:* 'Rafael Weingärtner' <rafaelweingartner at gmail.com>
>> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
>> *Subject:* RE: [Ceilometer] Pollster cannot get RadosGW metrics when API
>> endpoints are based on URL instead of port number
>>
>>
>>
>> Thanks a lot for your quick answer, Rafael !
>>
>> I will explore this approach.
>>
>>
>>
>> Jean-Francois
>>
>>
>>
>> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
>> *Sent:* lundi, 29 août 2022 17:54
>> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
>> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
>> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
>> endpoints are based on URL instead of port number
>>
>>
>>
>>
>>
>> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>>
>> You could use a different approach. You can use Dynamic pollster [1], and
>> create your own mechanism to collect data, without needing to change
>> Ceilometer code. Basically all hard-coded pollsters can be converted to a
>> dynamic pollster that is defined in YML.
>>
>>
>>
>> [1]
>> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html#the-dynamic-pollsters-system-configuration-for-non-openstack-apis
>>
>>
>>
>>
>>
>> On Mon, Aug 29, 2022 at 12:51 PM Taltavull Jean-François <
>> jean-francois.taltavull at elca.ch> wrote:
>>
>> Hi All,
>>
>> In our OpenStack deployment, API endpoints are defined by using URLs
>> instead of port numbers and HAProxy  forwards requests to the right bakend
>> after having ACLed the URL.
>>
>> In the case of our object-store service, based on RadosGW, the internal
>> API endpoint is "https://<FQDN>/object-store/swift/v1/AUTH_<tenant_id>"
>>
>> When Ceilometer RadosGW pollster tries to connect to the RadosGW admin
>> API with the object-store internal endpoint, the URL becomes
>> https://<FQDN>/admin, as shown by HAProxy logs. This URL does not match
>> any API endpoint from HAProxy point of view. The line of code that rewrites
>> the URL is this one:
>> https://opendev.org/openstack/ceilometer/src/branch/stable/wallaby/ceilometer/objectstore/rgw.py#L81
>>
>> What would you think of adding a mechanism based on new Ceilometer
>> configuration option(s) to control the URL rewriting ?
>>
>> Our deployment characteristics:
>> - OpenStack release: Wallaby
>> - Ceph and RadosGW version: 15.2.16
>> - deployment tool: OSA 23.2.1 and ceph-ansible
>>
>>
>> Best regards,
>> Jean-Francois
>>
>>
>>
>> --
>>
>> Rafael Weingärtner
>>
>>
>>
>> --
>>
>> Rafael Weingärtner
>>
>
>
> --
> Rafael Weingärtner
>


-- 
Rafael Weingärtner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220907/dbe6edf3/attachment-0001.htm>