Open Stack

Can you also execute the following:
```
python

import awsauth

awsauth
```

That will output a path, and then you can `cat <path>`, example: `cat
/var/lib/kolla/venv/lib/python3.8/site-packages/awsauth.py`

On Wed, Sep 28, 2022 at 1:21 PM Taltavull Jean-François <
jean-francois.taltavull at elca.ch> wrote:

> I removed trailing ‘/object-store/’ from the last value of
> authentication_parameters
>
>
>
> I also:
>
> - disabled s3 keystone auth in RGW
>
> - created a RGW “admin” user with the right privileges to allow admin API
> calls
>
> - put RGW in debug mode
>
>
>
> And here is what I get in RGW logs:
>
>
>
> get_usage
> string_to_sign=GET
>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      Wed,
> 28 Sep 2022 16:15:45
> GMT
>                                                                                                                     /admin/usage
>
> get_usage server signature=BlaBlaBlaBla
>
> get_usage client signature=BloBloBlo
>
> get_usage compare=-75
>
> get_usage rgw::auth::s3::LocalEngine denied with reason=-2027
>
> get_usage rgw::auth::s3::AWSAuthStrategy denied with reason=-2027
>
> get_usage rgw::auth::StrategyRegistry::s3_main_strategy_t: trying
> rgw::auth::s3::AWSAuthStrategy
>
> get_usage rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::LocalEngine
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* mercredi, 28 septembre 2022 13:15
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> I think that the last parameter "<FQDN>/object-store/", should be only "
> <FQDN>". Can you test it?
>
>
>
>
>
> You are using EC2 credentials to authenticate in RGW. Did you enable the
> Keystone integration in RGW?
>
> Also, as far as I know, this admin endpoint needs a RGW admin. I am not
> sure if the Keystone and RGW integration would enable/make it possible for
> someone to authenticate as an admin in RGW. Can you check it? To see if you
> can call that endpoint with these credentials.
>
>
>
> On Wed, Sep 28, 2022 at 6:01 AM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Pollster YML configuration :
>
>
>
> ---
>
> - name: "dynamic.radosgw.usage"
>
>   sample_type: "gauge"
>
>   unit: "B"
>
>   value_attribute: "total.size"
>
>   url_path: http://<FQDN>/object-store/admin/usage
>
>   module: "awsauth"
>
>   authentication_object: "S3Auth"
>
>   authentication_parameters: <ACCESS_KEY>,<SECRET_KEY>,<FQDN>/object-store/
>
>   user_id_attribute: "user"
>
>   project_id_attribute: "user"
>
>   resource_id_attribute: "user"
>
>   response_entries_key: "summary"
>
>
>
> ACCESS_KEY and SECRET_KEY have been created with “openstack ec2
> credentials create”.
>
>
>
> Ceilometer central is deployed with OSA and it uses awsauth.py module.
>
>
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* mercredi, 28 septembre 2022 02:01
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> Can you show your YML configuration? Also, did you install the AWS
> authentication module in the container/host where Ceilometer central is
> running?
>
>
>
> On Mon, Sep 26, 2022 at 12:58 PM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Hello Rafael,
>
>
>
> Thanks for the information about ceilometer patches but for now I’m
> testing with the credentials in the dynamic pollster config file. I will
> use barbican when I push all this to production.
>
>
>
> The keystone authentication performed by the rados gw with the credentials
> provided by ceilometer still does not work. I wonder if this could be a S3
> signature version issue on ceilometer side, that is on S3 client side. This
> kind of issue exists with the s3 client “s3cmd” and you have to add
> “—signature-v2” so that “s3cmd” works well.
>
>
>
> What do you think ? Do you know which version of S3 signature ceilometer
> uses while authenticating ?
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* mercredi, 7 septembre 2022 19:23
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> Jean, there are two problems with the Ceilometer. I just opened the
> patches to resolve it:
> - https://review.opendev.org/c/openstack/ceilometer/+/856305
>
> - https://review.opendev.org/c/openstack/ceilometer/+/856304
>
>
>
> Without these patches, you might have problems to use Ceilometer with
> Non-OpenStack dynamic pollsters and barbican credentials.
>
>
>
> On Wed, Aug 31, 2022 at 3:55 PM Rafael Weingärtner <
> rafaelweingartner at gmail.com> wrote:
>
> It is the RGW user that you have. This user must have the role that is
> needed to access the usage feature in RGW. If I am not mistaken, it
> required an admin user.
>
>
>
> On Wed, Aug 31, 2022 at 1:54 PM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Thanks to your help, I am close to the goal. Dynamic pollster is loaded
> and triggered.
>
>
>
> But I get a “Status[403] and reason [Forbidden]” in ceilometer logs while
> requesting admin/usage.
>
>
>
> I’m not sure to understand well the auth mechanism. Are we talking about
> keystone credentials, ec2 credentials, Rados GW user ?...
>
>
>
> For now, in testing phase, I use “authentication_parameters”, not barbican.
>
>
>
> -JF
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* mardi, 30 août 2022 14:17
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> Yes, you will need to enable the metric/pollster to be processed. That is
> done via "polling.yml" file. Also, do not forget that you will need to
> configure Ceilometer to push this new metric. If you use Gnocchi as the
> backend, you will need to change/update the gnocchi resource YML file. That
> file maps resources and metrics in the Gnocchi backend. The configuration
> resides in Ceilometer. You can create/define new resource types and map
> them to specific metrics. It depends on how you structure your solution.
>
> P.S. You do not need to use "authentication_parameters". You can use the
> barbican integration to avoid setting your credentials in a file.
>
>
>
> On Tue, Aug 30, 2022 at 9:11 AM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Hello,
>
>
>
> I tried to define a Rados GW dynamic pollster and I can see, in Ceilometer
> logs, that it’s actually loaded. But it looks like it was not triggered, I
> see no trace of ceilometer connection in Rados GW logs.
>
>
>
> My definition:
>
>
>
> - name: "dynamic.radosgw.usage"
>
>   sample_type: "gauge"
>
>   unit: "B"
>
>   value_attribute: "total.size"
>
>   url_path: http://<FQDN>/object-store/swift/v1/admin/usage
>
>   module: "awsauth"
>
>   authentication_object: "S3Auth"
>
>   authentication_parameters: xxxxxxxxxxxxx,yyyyyyyyyyyyy,<FQDN>
>
>   user_id_attribute: "admin"
>
>   project_id_attribute: "admin"
>
>   resource_id_attribute: "admin"
>
>   response_entries_key: "summary"
>
>
>
> Do I have to set an option in ceilometer.conf, or elsewhere, to get my
> Rados GW dynamic pollster triggered ?
>
>
>
> -JF
>
>
>
> *From:* Taltavull Jean-François
> *Sent:* lundi, 29 août 2022 18:41
> *To:* 'Rafael Weingärtner' <rafaelweingartner at gmail.com>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* RE: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
> Thanks a lot for your quick answer, Rafael !
>
> I will explore this approach.
>
>
>
> Jean-Francois
>
>
>
> *From:* Rafael Weingärtner <rafaelweingartner at gmail.com>
> *Sent:* lundi, 29 août 2022 17:54
> *To:* Taltavull Jean-François <jean-francois.taltavull at elca.ch>
> *Cc:* openstack-discuss <openstack-discuss at lists.openstack.org>
> *Subject:* Re: [Ceilometer] Pollster cannot get RadosGW metrics when API
> endpoints are based on URL instead of port number
>
>
>
>
>
> *EXTERNAL MESSAGE *- This email comes from *outside ELCA companies*.
>
> You could use a different approach. You can use Dynamic pollster [1], and
> create your own mechanism to collect data, without needing to change
> Ceilometer code. Basically all hard-coded pollsters can be converted to a
> dynamic pollster that is defined in YML.
>
>
>
> [1]
> https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html#the-dynamic-pollsters-system-configuration-for-non-openstack-apis
>
>
>
>
>
> On Mon, Aug 29, 2022 at 12:51 PM Taltavull Jean-François <
> jean-francois.taltavull at elca.ch> wrote:
>
> Hi All,
>
> In our OpenStack deployment, API endpoints are defined by using URLs
> instead of port numbers and HAProxy  forwards requests to the right bakend
> after having ACLed the URL.
>
> In the case of our object-store service, based on RadosGW, the internal
> API endpoint is "https://<FQDN>/object-store/swift/v1/AUTH_<tenant_id>"
>
> When Ceilometer RadosGW pollster tries to connect to the RadosGW admin API
> with the object-store internal endpoint, the URL becomes
> https://<FQDN>/admin, as shown by HAProxy logs. This URL does not match
> any API endpoint from HAProxy point of view. The line of code that rewrites
> the URL is this one:
> https://opendev.org/openstack/ceilometer/src/branch/stable/wallaby/ceilometer/objectstore/rgw.py#L81
>
> What would you think of adding a mechanism based on new Ceilometer
> configuration option(s) to control the URL rewriting ?
>
> Our deployment characteristics:
> - OpenStack release: Wallaby
> - Ceph and RadosGW version: 15.2.16
> - deployment tool: OSA 23.2.1 and ceph-ansible
>
>
> Best regards,
> Jean-Francois
>
>
>
> --
>
> Rafael Weingärtner
>
>
>
> --
>
> Rafael Weingärtner
>
>
>
> --
>
> Rafael Weingärtner
>
>
>
> --
>
> Rafael Weingärtner
>
>
>
> --
>
> Rafael Weingärtner
>
>
>
> --
>
> Rafael Weingärtner
>


-- 
Rafael Weingärtner
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220928/5a7c0644/attachment-0001.htm>