Open Stack

Hi Pawel,

First question is what version of Octavia are you using? Older
versions required you to set some ACL permissions on the secrets in
Barbican. You can check this by reviewing the load balancing cookbook
for the version of Octavia you are running [1]. There is a drop down
in the upper right corner of the document that allows you to select a
version of the document.

Second question, can you expand on what you mean by "application
credentials"? Is this by using a pre-created token instead of having
the username/password in your environment?

Third, can you check your octavia.conf settings[2]? Check the
following options are either the default (commented out) or set to the
same settings as the default.
[certificates]
cert_manager = barbican_cert_manager
barbican_auth = barbican_acl_auth (Note, this is only valid in the
newer versions of Octavia as noted above)

Fourth (last one), can you provide the associated log output from the
Octavia API process that is handling this request? Debug if you can.

Michael

[1] https://docs.openstack.org/octavia/latest/user/guides/basic-cookbook.html#deploy-a-tls-terminated-https-load-balancer
[2] https://docs.openstack.org/octavia/latest/configuration/configref.html#certificates

On Fri, Jul 19, 2019 at 7:51 AM Pawel Konczalski
<pawel.konczalski at everyware.ch> wrote:
>
> Hi,
>
> i try to create a Octavia HTTPS listener by using application
> credentials but get this error:
>
> Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']
> (HTTP 400) (Request-ID: req-088d6eb0-a285-4089-bc11-ff0c3097123e)
>
>
> # openstack secret list
> +--------------------------------------------------------------------------------------+-------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
> | Secret href | Name  | Created                   | Status | Content
> types                             | Algorithm | Bit length | Secret type
> | Mode | Expiration |
> +--------------------------------------------------------------------------------------+-------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
> |
> https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35
> | cert2 | 2019-07-19T13:42:21+00:00 | ACTIVE | {u'default':
> u'application/octet-stream'} | aes       |        256 | opaque      |
> cbc  | None       |
> |
> https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09
> | cert1 | 2019-07-19T13:42:12+00:00 | ACTIVE | {u'default':
> u'application/octet-stream'} | aes       |        256 | opaque      |
> cbc  | None       |
> +--------------------------------------------------------------------------------------+-------+---------------------------+--------+-------------------------------------------+-----------+------------+-------------+------+------------+
>
>
> # openstack loadbalancer listener create foo-lb1 \
> --name foo-lb1-https-listener \
> --protocol-port 443 \
> --protocol TERMINATED_HTTPS \
> --insert-headers X-Forwarded-For=true,X-Forwarded-Proto=true \
> --default-tls-container=https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09
> \
> --sni-container-refs
> https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09
> https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35
>
>
> --------------------------------
>
> Starting new HTTPS connection (1): octavia.service.dev.example.com:443
> https://octavia.service.dev.example.com:443 "GET
> /v2.0/lbaas/loadbalancers HTTP/1.1" 200 779
> RESP: [200] Connection: keep-alive Content-Length: 779 Content-Type:
> application/json Date: Fri, 19 Jul 2019 13:56:24 GMT Server:
> WSGIServer/0.1 Python/2.7.15rc1 x-openstack-request-id:
> req-50b5a3bb-21ec-4a46-8d5c-61035afd3423
> RESP BODY: {"loadbalancers": [{"provider": "amphora", "description": "",
> "admin_state_up": true, "pools": [{"id":
> "169722d1-0a73-4283-bb42-aee5b662e2e2"}], "created_at":
> "2019-07-19T13:34:52", "provisioning_status": "ACTIVE", "updated_at":
> "2019-07-19T13:39:34", "vip_qos_policy_id": null, "vip_network_id":
> "2064c61c-64a1-466f-983a-af435ae1d51c", "listeners": [{"id":
> "169a91f9-ef5c-4d38-8449-e24b64cf082d"}], "tenant_id":
> "9646533a8d834978a868e81c9b9a39cf", "vip_port_id":
> "dcfc6e44-4092-4f2b-bd50-24e02abb078f", "flavor_id": "", "vip_address":
> "10.0.1.4", "vip_subnet_id": "787035dc-add4-4227-844a-1cf803625abc",
> "project_id": "9646533a8d834978a868e81c9b9a39cf", "id":
> "e2ed48ab-3261-422f-b9b5-a5aa63486ae7", "operating_status": "OFFLINE",
> "name": "foo-lb1"}], "loadbalancers_links": []}
> GET call to
> https://octavia.service.dev.example.com/v2.0/lbaas/loadbalancers used
> request id req-50b5a3bb-21ec-4a46-8d5c-61035afd3423
> REQ: curl -g -i -X POST
> https://octavia.service.dev.example.com/v2.0/lbaas/listeners -H
> "Content-Type: application/json" -H "User-Agent: openstacksdk/0.19.0
> keystoneauth1/3.11.1 python-requests/2.20.1 CPython/2.7.15+" -H
> "X-Auth-Token:
> {SHA256}6414e14f4e78940902b11c89567689e3cc0d3ea62227b87a1e19361685c83584"
> -d '{"listener": {"insert_headers": {"X-Forwarded-For": "true",
> "X-Forwarded-Proto": "true"}, "protocol": "TERMINATED_HTTPS", "name":
> "foo-lb1-https-listener", "default_tls_container_ref":
> "https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09",
> "sni_container_refs":
> ["https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09",
> "https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35"],
> "admin_state_up": true, "protocol_port": 443, "loadbalancer_id":
> "e2ed48ab-3261-422f-b9b5-a5aa63486ae7"}}'
> https://octavia.service.dev.example.com:443 "POST /v2.0/lbaas/listeners
> HTTP/1.1" 400 357
> RESP: [400] Connection: keep-alive Content-Length: 357 Content-Type:
> application/json Date: Fri, 19 Jul 2019 13:56:27 GMT Server:
> WSGIServer/0.1 Python/2.7.15rc1 x-openstack-request-id:
> req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca
> RESP BODY: {"debuginfo": null, "faultcode": "Client", "faultstring":
> "Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']"}
> POST call to
> https://octavia.service.dev.example.com/v2.0/lbaas/listeners used
> request id req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca
> Request returned failure status: 400
> Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']
> (HTTP 400) (Request-ID: req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca)
> Traceback (most recent call last):
>    File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 400, in
> run_subcommand
>      result = cmd.run(parsed_args)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/osc_lib/command/command.py",
> line 41, in run
>      return super(Command, self).run(parsed_args)
>    File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 116,
> in run
>      column_names, data = self.take_action(parsed_args)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/octaviaclient/osc/v2/listener.py",
> line 168, in take_action
>      json=body)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/octaviaclient/api/v2/octavia.py",
> line 38, in wrapper
>      request_id=e.request_id)
> OctaviaClientException: Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']
> (HTTP 400) (Request-ID: req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca)
> clean_up CreateListener: Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']
> (HTTP 400) (Request-ID: req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca)
> Traceback (most recent call last):
>    File "/home/foo/.local/lib/python2.7/site-packages/osc_lib/shell.py",
> line 136, in run
>      ret_val = super(OpenStackShell, self).run(argv)
>    File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 279, in run
>      result = self.run_subcommand(remainder)
>    File "/home/foo/.local/lib/python2.7/site-packages/osc_lib/shell.py",
> line 176, in run_subcommand
>      ret_value = super(OpenStackShell, self).run_subcommand(argv)
>    File "/usr/lib/python2.7/dist-packages/cliff/app.py", line 400, in
> run_subcommand
>      result = cmd.run(parsed_args)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/osc_lib/command/command.py",
> line 41, in run
>      return super(Command, self).run(parsed_args)
>    File "/usr/lib/python2.7/dist-packages/cliff/display.py", line 116,
> in run
>      column_names, data = self.take_action(parsed_args)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/octaviaclient/osc/v2/listener.py",
> line 168, in take_action
>      json=body)
>    File
> "/home/foo/.local/lib/python2.7/site-packages/octaviaclient/api/v2/octavia.py",
> line 38, in wrapper
>      request_id=e.request_id)
> OctaviaClientException: Could not retrieve certificate:
> ['https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09',
> 'https://barbican.service.dev.example.com/v1/secrets/593cc231-92ee-4b0a-8c58-0080052a6b35',
> 'https://barbican.service.dev.example.com/v1/secrets/cb28220c-1339-4fc0-83f7-9cd155e3dc09']
> (HTTP 400) (Request-ID: req-5eef99bf-45c9-43eb-b7c7-2dacaff980ca)
> ------------------------------
>
>
> This issue occurs only when application credentials are used. Creation
> of HTTP listener with applications credentials works fine, also creation
> of HTTPS listener when user are authenticated by user / password.
>
> Does somebody know which additional ACLs / permissions are required to
> fix this?
>
> BR
>
> Pawel