<div dir="ltr">No, I just showed you the code, so you can see how the authentication is being executed, and where/how the parameters are set in the headers. It is a bit odd, I have used this so many times, and it always works. What is your RGW instance version?<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Sep 30, 2022 at 4:09 AM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch">jean-francois.taltavull@elca.ch</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg4143276329858614983">
<div style="overflow-wrap: break-word;" lang="FR">
<div class="m_4143276329858614983WordSection1">
<p class="MsoNormal"><span lang="EN-US">Do you mean the issue comes from how the `awsauth` module handles the signature ?<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"><u></u> <u></u></span></p>
<div style="border-color:currentcolor currentcolor currentcolor blue;border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt">
<div>
<div style="border-color:rgb(225,225,225) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> jeudi, 29 septembre 2022 17:23<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number<u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:10pt">This is the signature used by the `awsauth` library:<br>
```<u></u><u></u></span></p>
<pre style="background:rgb(43,43,43)"><span style="color:rgb(204,120,50)">def </span><span style="color:rgb(255,198,109)">get_signature</span><span style="color:rgb(169,183,198)">(</span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(169,183,198)">r):<br> canonical_string = </span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(169,183,198)">.get_canonical_string(<br> r.url</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(169,183,198)">r.headers</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(169,183,198)">r.method)<br> </span><span style="color:rgb(204,120,50)">if </span><span style="color:rgb(169,183,198)">py3k:<br> key = </span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(169,183,198)">.secret_key.encode(</span><span style="color:rgb(106,135,89)">'utf-8'</span><span style="color:rgb(169,183,198)">)<br> msg = canonical_string.encode(</span><span style="color:rgb(106,135,89)">'utf-8'</span><span style="color:rgb(169,183,198)">)<br> </span><span style="color:rgb(204,120,50)">else</span><span style="color:rgb(169,183,198)">:<br> key = </span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(169,183,198)">.secret_key<br> msg = canonical_string<br> h = hmac.new(key</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(169,183,198)">msg</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(170,73,38)">digestmod</span><span style="color:rgb(169,183,198)">=sha)<br> </span><span style="color:rgb(204,120,50)">return </span><span style="color:rgb(169,183,198)">encodestring(h.digest()).strip()<u></u><u></u></span></pre>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10pt">```<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt"><span style="font-size:10pt">After that is generated, it is added in the headers:<u></u><u></u></span></p>
<pre style="background:rgb(43,43,43)"><span style="color:gray"># Create date header if it is not created yet.<br></span><span style="color:rgb(204,120,50)">if </span><span style="color:rgb(106,135,89)">'date' </span><span style="color:rgb(204,120,50)">not in </span><span style="color:rgb(169,183,198)">r.headers </span><span style="color:rgb(204,120,50)">and </span><span style="color:rgb(106,135,89)">'x-amz-date' </span><span style="color:rgb(204,120,50)">not in </span><span style="color:rgb(169,183,198)">r.headers:<br> r.headers[</span><span style="color:rgb(106,135,89)">'date'</span><span style="color:rgb(169,183,198)">] = formatdate(<br> </span><span style="color:rgb(170,73,38)">timeval</span><span style="color:rgb(169,183,198)">=</span><span style="color:rgb(204,120,50)">None,<br> </span><span style="color:rgb(170,73,38)">localtime</span><span style="color:rgb(169,183,198)">=</span><span style="color:rgb(204,120,50)">False,<br> </span><span style="color:rgb(170,73,38)">usegmt</span><span style="color:rgb(169,183,198)">=</span><span style="color:rgb(204,120,50)">True</span><span style="color:rgb(169,183,198)">)<br>signature = </span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(169,183,198)">.get_signature(r)<br></span><span style="color:rgb(204,120,50)">if </span><span style="color:rgb(169,183,198)">py3k:<br> signature = signature.decode(</span><span style="color:rgb(106,135,89)">'utf-8'</span><span style="color:rgb(169,183,198)">)<br>r.headers[</span><span style="color:rgb(106,135,89)">'Authorization'</span><span style="color:rgb(169,183,198)">] = </span><span style="color:rgb(106,135,89)">'AWS %s:%s' </span><span style="color:rgb(169,183,198)">% (</span><span style="color:rgb(148,85,141)">self</span><span style="color:rgb(169,183,198)">.access_key</span><span style="color:rgb(204,120,50)">, </span><span style="color:rgb(169,183,198)">signature)<u></u><u></u></span></pre>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><u></u> <u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Thu, Sep 29, 2022 at 9:15 AM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>> wrote:<u></u><u></u></span></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<div>
<p class="MsoNormal">```<u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">$ python test_creds.py</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Executing test on: [FQDN/object-store/].</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Rados GW admin context [/admin] and path [/usage?stats=True] used.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Rados GW request URL [<a href="http://FQDN/object-store/admin/bucket?stats=True" target="_blank">http://FQDN/object-store/admin/bucket?stats=True</a>].</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Rados GW host: FQDN</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Traceback (most recent call last):</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> File "test_creds.py", line 45, in <module></span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> raise RGWAdminAPIFailed(</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">__main__.RGWAdminAPIFailed: RGW AdminOps API returned 403 Forbidden</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">```</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">So the same as with ceilometer. Auth is done by RGW, not by keystone, and the ceph “admin” user exists and owns the right privileges:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">```</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">$ sudo radosgw-admin user info --uid admin [22/296]{</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "user_id": "admin",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "display_name": "admin user",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "email": "",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "suspended": 0,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "max_buckets": 1000,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "subusers": [],</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "keys": [</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> {</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "user": "admin",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "access_key": “admin_access_key",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "secret_key": "admin_secret_key"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> }</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> ],</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "swift_keys": [],</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "caps": [</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> {</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "type": "buckets",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "perm": "*"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> },</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> {</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "type": "metadata",</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "perm": "*"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> },</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> { "type":
"usage", "perm":
"*" },
{</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> "type": "users", "perm":
"*" }
],
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">```</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> jeudi, 29 septembre 2022 12:32<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Can you test you credentials with the following code?<br>
<br>
```<br>
<br>
import json<br>
import requests<br>
import os<br>
<br>
import six.moves.urllib.parse as urlparse<br>
<br>
<br>
class RGWAdminAPIFailed(Exception):<br>
pass<br>
<br>
<br>
if __name__ == '__main__':<br>
<br>
rados_gw_base_url = "put your RGW URL here. E.g. <a href="http://server.com:port/something" target="_blank">http://server.com:port/something</a>"<br>
print("Executing test on: [%s]." % rados_gw_base_url)<br>
<br>
rados_gw_admin_context = "/admin"<br>
<br>
rados_gw_path = "/usage?stats=True"<br>
<br>
print("Rados GW admin context [%s] and path [%s] used." % (rados_gw_admin_context, rados_gw_path))<br>
<br>
rados_gw_request_url = urlparse.urljoin(rados_gw_base_url, '/admin') + '/bucket?stats=True'<br>
print("Rados GW request URL [%s]." % rados_gw_request_url)<br>
<br>
rados_gw_access_key_to_use = "put your access key here"<br>
rados_gw_secret_key_to_use = "put your secret key here"<br>
<br>
rados_gw_host_name = urlparse.urlparse(rados_gw_request_url).netloc<br>
print("Rados GW host: %s" % rados_gw_host_name)<br>
module_name = "awsauth"<br>
class_name = "S3Auth"<br>
arguments = [rados_gw_access_key_to_use, rados_gw_secret_key_to_use, rados_gw_host_name]<br>
module = __import__(module_name)<br>
class_ = getattr(module, class_name)<br>
instance = class_(*arguments)<br>
<br>
r = requests.get(<br>
rados_gw_request_url,<br>
auth=instance, timeout=30)<br>
#auth=awsauth.S3Auth(*arguments))<br>
<br>
<br>
if r.status_code != 200:<br>
raise RGWAdminAPIFailed(<br>
('RGW AdminOps API returned %(status)s %(reason)s') %<br>
{'status': r.status_code, 'reason': r.reason})<br>
<br>
response_body = r.text<br>
parsed_json = json.loads(response_body)<br>
<br>
print("Response cookies: [%s]." % r.cookies)<br>
<br>
radosGw_output_file = "/home/<user_here>/Downloads/radosGw-usage.json"<br>
<br>
if os.path.exists(radosGw_output_file):<br>
os.remove(radosGw_output_file)<br>
<br>
with open(radosGw_output_file, "w") as file1:<br>
file1.writelines(json.dumps(parsed_json, indent=4, sort_keys=True))<br>
file1.flush()<br>
<br>
exit(0)<br>
<br>
```</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Thu, Sep 29, 2022 at 4:09 AM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">python</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Python 3.8.10 (default, Sep 28 2021, 16:10:42)</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">[GCC 9.3.0] on linux</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Type "help", "copyright", "credits" or "license" for more information.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">>>> import awsauth</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">>>> awsauth</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"><module 'awsauth' from '/openstack/venvs/ceilometer-23.2.0/lib/python3.8/site-packages/awsauth.py'></span><u></u><u></u></p>
<p class="MsoNormal">>>> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> mercredi, 28 septembre 2022 18:40<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Can you also execute the following:</span><u></u><u></u></p>
<div>
<p class="MsoNormal"><span style="font-size:10pt">```</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">python</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">import awsauth</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">awsauth</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12pt"><span style="font-size:10pt">```</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">That will output a path, and then you can `cat <path>`, example: `cat /var/lib/kolla/venv/lib/python3.8/site-packages/awsauth.py`</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Wed, Sep 28, 2022 at 1:21 PM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">I removed trailing ‘/object-store/’ from the last value of authentication_parameters</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I also:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">- disabled s3 keystone auth in RGW</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">- created a RGW “admin” user with the right privileges to allow admin API calls</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">- put RGW in debug mode</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">And here is what I get in RGW logs:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage string_to_sign=GET Wed,
28 Sep 2022 16:15:45 GMT /admin/usage</span><u></u><u></u></p>
<p class="MsoNormal">get_usage server signature=BlaBlaBlaBla<u></u><u></u></p>
<p class="MsoNormal">get_usage client signature=BloBloBlo<u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage compare=-75</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage rgw::auth::s3::LocalEngine denied with reason=-2027</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage rgw::auth::s3::AWSAuthStrategy denied with reason=-2027</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage rgw::auth::StrategyRegistry::s3_main_strategy_t: trying rgw::auth::s3::AWSAuthStrategy</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">get_usage rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::LocalEngine</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> mercredi, 28 septembre 2022 13:15<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red" lang="EN-US"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">I think that the last parameter "</span><span style="font-size:10pt" lang="EN-US"><FQDN>/object-store/</span><span style="font-size:10pt">", should
be only "</span><span style="font-size:10pt" lang="EN-US"><FQDN></span><span style="font-size:10pt">". Can you test it?</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">You are using EC2 credentials to authenticate in RGW. Did you enable the Keystone integration in RGW?
</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Also, as far as I know, this admin endpoint needs a RGW admin. I am not sure if the Keystone and RGW integration would enable/make it possible for
someone to authenticate as an admin in RGW. Can you check it? To see if you can call that endpoint with these credentials.</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Wed, Sep 28, 2022 at 6:01 AM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal">Pollster YML configuration :<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">---<u></u><u></u></p>
<p class="MsoNormal">- name: "dynamic.radosgw.usage"<u></u><u></u></p>
<p class="MsoNormal"> sample_type: "gauge"<u></u><u></u></p>
<p class="MsoNormal"> unit: "B"<u></u><u></u></p>
<p class="MsoNormal">
<span lang="EN-US">value_attribute: "total.size"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> url_path:
<a href="http://%3cFQDN%3e/object-store/admin/usage" target="_blank">http://<FQDN>/object-store/admin/usage</a></span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> module: "awsauth"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> authentication_object: "S3Auth"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> authentication_parameters: <ACCESS_KEY>,<SECRET_KEY>,<FQDN>/object-store/</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> user_id_attribute: "user"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> project_id_attribute: "user"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> resource_id_attribute: "user"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> response_entries_key: "summary"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">ACCESS_KEY and SECRET_KEY have been created with “openstack ec2 credentials create”.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Ceilometer central is deployed with OSA and it uses awsauth.py module.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> mercredi, 28 septembre 2022 02:01<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red" lang="EN-US"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:10pt;color:white">Can you show your YML configuration? Also, did you install the AWS authentication module in the container/host where Ceilometer central
is running?</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Mon, Sep 26, 2022 at 12:58 PM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal">Hello Rafael,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Thanks for the information about ceilometer patches but for now I’m testing with the credentials in the dynamic pollster config file. I will use barbican when
I push all this to production.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">The keystone authentication performed by the rados gw with the credentials provided by ceilometer still does not work. I wonder if this could be a S3 signature
version issue on ceilometer side, that is on S3 client side. This kind of issue exists with the s3 client “s3cmd” and you have to add “—signature-v2” so that “s3cmd” works well.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">What do you think ? Do you know which version of S3 signature ceilometer uses while authenticating ?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> mercredi, 7 septembre 2022 19:23<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><strong><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></strong><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<strong><span style="font-family:"Arial",sans-serif">outside ELCA companies</span></strong>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;color:white">Jean, there are two problems with the Ceilometer. I just opened the patches to resolve it:</span><span style="font-size:10pt"><br>
- <a href="https://review.opendev.org/c/openstack/ceilometer/+/856305" target="_blank">
https://review.opendev.org/c/openstack/ceilometer/+/856305</a></span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">-
<a href="https://review.opendev.org/c/openstack/ceilometer/+/856304" target="_blank">
https://review.opendev.org/c/openstack/ceilometer/+/856304</a></span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Without these patches, you might have problems to use Ceilometer with Non-OpenStack dynamic pollsters and barbican credentials.</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Wed, Aug 31, 2022 at 3:55 PM Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<p class="MsoNormal"><span style="font-size:10pt">It is the RGW user that you have. This user must have the role that is needed to access the usage feature in RGW. If I am not mistaken, it required
an admin user. </span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Wed, Aug 31, 2022 at 1:54 PM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Thanks to your help, I am close to the goal. Dynamic pollster is loaded and triggered.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">But I get a “Status[403] and reason [Forbidden]” in ceilometer logs while requesting admin/usage.
</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I’m not sure to understand well the auth mechanism. Are we talking about keystone credentials, ec2 credentials, Rados GW user ?...</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">For now, in testing phase, I use “authentication_parameters”, not barbican.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">-JF</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> mardi, 30 août 2022 14:17<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><b><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></b><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<b>outside ELCA companies</b>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:10pt;color:white">Yes, you will need to enable the metric/pollster to be processed. That is done via "polling.yml"
</span><span style="font-size:10pt">file. Also, do not forget that you will need to configure Ceilometer to push this new metric. If you use Gnocchi as the backend, you will need to change/update the gnocchi resource YML file. That file maps resources and
metrics in the Gnocchi backend. The configuration resides in Ceilometer. You can create/define new resource types and map them to specific metrics. It depends on how you structure your solution.<br>
<br>
P.S. You do not need to use "authentication_parameters". You can use the barbican integration to avoid setting your credentials in a file.</span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Tue, Aug 30, 2022 at 9:11 AM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hello,</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I tried to define a Rados GW dynamic pollster and I can see, in Ceilometer logs, that it’s actually loaded. But it looks like it was not triggered, I see no trace
of ceilometer connection in Rados GW logs.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">My definition:</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">- name: "dynamic.radosgw.usage"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">
</span>sample_type: "gauge"<u></u><u></u></p>
<p class="MsoNormal"> unit: "B"<u></u><u></u></p>
<p class="MsoNormal">
<span lang="EN-US">value_attribute: "total.size"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> url_path:
<a href="http://%3cFQDN%3e/object-store/swift/v1/admin/usage" target="_blank">http://<FQDN>/object-store/swift/v1/admin/usage</a></span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> module: "awsauth"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> authentication_object: "S3Auth"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> authentication_parameters: xxxxxxxxxxxxx,yyyyyyyyyyyyy,<FQDN></span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> user_id_attribute: "admin"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> project_id_attribute: "admin"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> resource_id_attribute: "admin"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> response_entries_key: "summary"</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Do I have to set an option in ceilometer.conf, or elsewhere, to get my Rados GW dynamic pollster triggered ?</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">-JF</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Taltavull Jean-François
<br>
<b>Sent:</b> lundi, 29 août 2022 18:41<br>
<b>To:</b> 'Rafael Weingärtner' <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> RE: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Thanks a lot for your quick answer, Rafael !</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">I will explore this approach.</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US">Jean-Francois</span><u></u><u></u></p>
<p class="MsoNormal"><span lang="EN-US"> </span><u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0cm 0cm 0cm 4pt;border-color:currentcolor currentcolor currentcolor blue">
<div>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0cm 0cm;border-color:currentcolor">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Rafael Weingärtner <<a href="mailto:rafaelweingartner@gmail.com" target="_blank">rafaelweingartner@gmail.com</a>>
<br>
<b>Sent:</b> lundi, 29 août 2022 17:54<br>
<b>To:</b> Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>><br>
<b>Cc:</b> openstack-discuss <<a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a>><br>
<b>Subject:</b> Re: [Ceilometer] Pollster cannot get RadosGW metrics when API endpoints are based on URL instead of port number</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<table style="width:100%;border-collapse:collapse" width="100%" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="width:5.25pt;background:red;padding:0.75pt" width="7">
<p class="MsoNormal" style="margin-bottom:7.5pt"><span style="font-size:10pt;color:red"> </span><u></u><u></u></p>
</td>
<td style="background:rgb(255,235,156);padding:3.75pt 11.25pt">
<p class="MsoNormal" style="margin-bottom:7.5pt"><b><i><span style="font-size:10pt;font-family:"Arial",sans-serif;color:red">EXTERNAL MESSAGE
</span></i></b><span style="font-size:10pt;font-family:"Arial",sans-serif;color:black">- This email comes from
<b>outside ELCA companies</b>.</span><u></u><u></u></p>
</td>
</tr>
</tbody>
</table>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt;color:black">You could use a different approach. You can use Dynamic pollster [1], and create your own mechanism to collect data, without needing to
change Ceilometer code. Basically all hard-coded pollsters can be converted to a dynamic pollster that is defined in YML</span><span style="font-size:10pt">.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">[1]
<a href="https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html#the-dynamic-pollsters-system-configuration-for-non-openstack-apis" target="_blank">
https://docs.openstack.org/ceilometer/latest/admin/telemetry-dynamic-pollster.html#the-dynamic-pollsters-system-configuration-for-non-openstack-apis</a></span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
</div>
<p class="MsoNormal"><span style="font-size:10pt"> </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">On Mon, Aug 29, 2022 at 12:51 PM Taltavull Jean-François <<a href="mailto:jean-francois.taltavull@elca.ch" target="_blank">jean-francois.taltavull@elca.ch</a>>
wrote:</span><u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<p class="MsoNormal" style="margin-bottom:12pt"><span style="font-size:10pt">Hi All,<br>
<br>
In our OpenStack deployment, API endpoints are defined by using URLs instead of port numbers and HAProxy forwards requests to the right bakend after having ACLed the URL.<br>
<br>
In the case of our object-store service, based on RadosGW, the internal API endpoint is "<a href="https://%3cFQDN%3e/object-store/swift/v1/AUTH_%3ctenant_id%3e" target="_blank">https://<FQDN>/object-store/swift/v1/AUTH_<tenant_id></a>"<br>
<br>
When Ceilometer RadosGW pollster tries to connect to the RadosGW admin API with the object-store internal endpoint, the URL becomes
<a href="https://%3cFQDN%3e/admin" target="_blank">https://<FQDN>/admin</a>, as shown by HAProxy logs. This URL does not match any API endpoint from HAProxy point of view. The line of code that rewrites the URL is this one:
<a href="https://opendev.org/openstack/ceilometer/src/branch/stable/wallaby/ceilometer/objectstore/rgw.py#L81" target="_blank">
https://opendev.org/openstack/ceilometer/src/branch/stable/wallaby/ceilometer/objectstore/rgw.py#L81</a><br>
<br>
What would you think of adding a mechanism based on new Ceilometer configuration option(s) to control the URL rewriting ?<br>
<br>
Our deployment characteristics:<br>
- OpenStack release: Wallaby<br>
- Ceph and RadosGW version: 15.2.16<br>
- deployment tool: OSA 23.2.1 and ceph-ansible<br>
<br>
<br>
Best regards,<br>
Jean-Francois</span><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- </span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner</span><u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><span style="font-size:10pt"><br clear="all">
<br>
-- <u></u><u></u></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10pt">Rafael Weingärtner<u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</div></blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr">Rafael Weingärtner</div></div>