[Kolla] [LetsEncrypt] Internal/External/Backend Certificates
Hi all, another one here that can't seem to get kolla running with letsencrypt certs, just curious as to what settings are required for kolla to pull internal/external/backend certs from letsencrypt? I've looked at the docs and have tried a few settings but keep getting errors, sure i had this working last week before i tore it down ready to deploy for production and now i just cant get it to complete the deployment because of certs, only thing i can think of is i need to run kolla-ansible -i multinode certificates but i thought that was just for self made certs? Also, what are best practices for backend tls certs, self signed or using the internal cert etc?? Is it possible to use letsencrypt at all for backend? Cheers all
So with the settings: kolla_internal_fqdn: "int.cloud.example.com" kolla_external_fqdn: "cloud.example.com" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy-internal.pem" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" openstack_cacert: "/etc/ssl/certs/ca-certificates.crt" It currently pulls only for the external vip but not the internal so clearly i'm missing something, although i can get it to pull in a hacky way by editing letsencrypt-lego-run.sh to pull int.cloud.example.com No matter what i try i can't get any version of this working with anything other than self signed certs: kolla_enable_tls_backend: "yes" kolla_verify_tls_backend: "yes" kolla_tls_backend_cert: "{{ kolla_certificates_dir }}/backend-cert.pem" kolla_tls_backend_key: "{{ kolla_certificates_dir }}/backend-key.pem" kolla_copy_ca_into_containers: "yes" The note at: https://docs.openstack.org/kolla-ansible/2024.1/admin/tls.html states: The back-end TLS cert/key can be the same certificate that is used for the VIP, as long as those certificates are configured to allow requests from both the VIP and internal networks. Yet, i can't figure out how to do that or what combination of keys to use from the vip certs to get that working For now it's running with kolla_enable_tls_backend: no and a manually pulled internal cert but this is not ideal, anyone any pointers?
At this point I've determined it's just not possible, having tried a myriad of settings and asking on here and irc I've found no solution to use an official ca such as LetsEncrypt for backend tls, self signed is possible but causes problems when the vips are https. So as it stands, internal/external vips are tls, backend is not, I'll revisit this when either someone expands on how or the docs are updated to explain how as right now, there is no explainable way to get it working with official certs for all 3.
Am Montag, dem 05.08.2024 um 18:58 +0000 schrieb jjjamesg@proton.me:
At this point I've determined it's just not possible, having tried a myriad of settings and asking on here and irc I've found no solution to use an official ca such as LetsEncrypt for backend tls, self signed is possible but causes problems when the vips are https. So as it stands, internal/external vips are tls, backend is not, I'll revisit this when either someone expands on how or the docs are updated to explain how as right now, there is no explainable way to get it working with official certs for all 3. This seems to be a bug which is currently being fixed, see:
https://bugs.launchpad.net/kolla-ansible/+bug/2076331 and the fix (still being reviewed): https://review.opendev.org/c/openstack/kolla-ansible/+/925971 maybe you can backport it to your branch locally and check if it works for you? it's only a very small change. HTH -- Sven Kieske Senior Cloud Engineer Mail: kieske@osism.tech Web: https://osism.tech OSISM GmbH / Talweg 8 / 75417 Mühlacker / Deutschland Geschäftsführer: Christian Berendt Unternehmenssitz: Mühlacker Amtsgericht: Stuttgart, HRB 756139
I see it's still not been merged after a number of things but i will give it a go. Just to be clear it is typical practice to use self signed for this and proper certs for vips yes? Cheers
Hi, It's obvious that you can't get a certificate for an internal endpoint because the public ACME server can't reach your internal domain right ? So they can't verify you are the owner. If you setup your own ACME server ..it will work ..and you will just change the ACME server from letsencrypt public one to your local one in configuration. For now the acme server is pointed to https://acme-v02.api.letsencrypt.org/directory In CI we are testing internal certificate and it works https://github.com/openstack/kolla-ansible/blob/30331f90f6da377debb92ec4aec4... <<< Check how the entire process work here -> https://letsencrypt.org/docs/challenge-types/ TLDR - Install your ACME server, propagate your trusted CA, change letsencrypt_cert_server to your server and issue your own certificate internally, don't use public service of https://acme-v02.api.letsencrypt.org Thanks Michal Arbet Openstack Engineer Ultimum Technologies a.s. Na Poříčí 1047/26, 11000 Praha 1 Czech Republic +420 604 228 897 michal.arbet@ultimum.io *https://ultimum.io <https://ultimum.io/>* LinkedIn <https://www.linkedin.com/company/ultimum-technologies> | Twitter <https://twitter.com/ultimumtech> | Facebook <https://www.facebook.com/ultimumtechnologies/timeline> út 30. 7. 2024 v 21:59 odesílatel <jjjamesg@proton.me> napsal:
So with the settings:
kolla_internal_fqdn: "int.cloud.example.com" kolla_external_fqdn: "cloud.example.com" kolla_enable_tls_internal: "yes" kolla_enable_tls_external: "yes" kolla_external_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy.pem" kolla_internal_fqdn_cert: "{{ kolla_certificates_dir }}/haproxy-internal.pem" haproxy_backend_cacert: "{{ 'ca-certificates.crt' if kolla_base_distro in ['debian', 'ubuntu'] else 'ca-bundle.trust.crt' }}" haproxy_backend_cacert_dir: "/etc/ssl/certs" openstack_cacert: "/etc/ssl/certs/ca-certificates.crt"
It currently pulls only for the external vip but not the internal so clearly i'm missing something, although i can get it to pull in a hacky way by editing letsencrypt-lego-run.sh to pull int.cloud.example.com
No matter what i try i can't get any version of this working with anything other than self signed certs:
kolla_enable_tls_backend: "yes" kolla_verify_tls_backend: "yes" kolla_tls_backend_cert: "{{ kolla_certificates_dir }}/backend-cert.pem" kolla_tls_backend_key: "{{ kolla_certificates_dir }}/backend-key.pem" kolla_copy_ca_into_containers: "yes"
The note at: https://docs.openstack.org/kolla-ansible/2024.1/admin/tls.html states:
The back-end TLS cert/key can be the same certificate that is used for the VIP, as long as those certificates are configured to allow requests from both the VIP and internal networks.
Yet, i can't figure out how to do that or what combination of keys to use from the vip certs to get that working
For now it's running with kolla_enable_tls_backend: no and a manually pulled internal cert but this is not ideal, anyone any pointers?
participants (3)
-
jjjamesg＠proton.me
-
Michal Arbet
-
Sven Kieske