On 29/07/2025 15:54, Jeremy Stanley wrote:
On 2025-07-29 07:20:39 -0000 (-0000), elnazcloud@gmail.com wrote: [...]
However, our concern is that—even though the secrets are encrypted at deployment time—they still end up as plaintext in the final config files inside the containers (e.g., nova.conf).
i tired to help as best i can inline but im not entirely sure what you gain by doing this. in general if someone can acess the host filesystem and read the nova.conf your security evnolope has already been breached. there is a minimal gain but unless you have already move away form using password/user login ot using revocable rotatable application credentials for service creditals there are probaly lower hanging furit you can adress first in hardening your openstack deployment. the main isue to me with saying "i dont turst storing secrete on the host file systme" is the classing boot straping probelum to regrive data form barbican or vault or any other Castellan backend you will still need a local root of trust, be that a secret sotred in teh host tpm or the client cert used for the remote_file backend. if the attacker can read that then they with a littel more work can just load the data form Castellan if that datasouce is remote not local like an infra barbican then you can audit the access and rotate the key but that has the disadvantage of intoduceing a single point of failure in your deployment. the keystore. so you need to be careful not to use any part of the cloud that uses the keystore to provision the keystore or you can have a chicken and egg situation if you have a pwoer outage. anyway if you figure out a nice way of doing this then im sure others would be interested in hearing about it.
[...]
I think oslo.config's remote_file driver was intended for this: https://docs.openstack.org/oslo.config/latest/configuration/options.html
the remote file diiver is not nessiarly a perfect fit but it an exampel of the plugable interface. you could write a vault or castellan driver for oslo.config but there was one developed in the past https://review.opendev.org/c/openstack/castellan/+/599589 oslo.config provides a stevedore entry point for this interface https://github.com/openstack/oslo.config/blob/68cefad313bd03522e99b3de95f178... the remote file driver is here https://github.com/openstack/oslo.config/blob/68cefad313bd03522e99b3de95f178... as a refence. it does not support http basic auth but i belive if you provide a client key and cert you can do mtls on the remote webserver. the plugin loading s done here https://github.com/openstack/oslo.config/blob/493a08645fe7339b9883f71748f341... going back to your question about castallen specifically you could install castallen in contaienr that dont already have it via a config overried there are exmples of this in the kolla docs https://github.com/openstack/kolla/blob/master/doc/source/admin/template-ove... https://github.com/openstack/kolla/blob/master/doc/source/admin/image-buildi... writing your own oslo.config driver is not common but i blelive at least one operator has written one to use dns svc or txt records to store the service config in dns so if the https://github.com/openstack/castellan/blob/master/castellan/_config_driver.... driver does not work for you should be able to provide your own. you just need to install castellan and add the following to your config overrides [DEFAULT] config_source = castellan_config_group [castellan_config_group] driver = castellan config_file = castellan.conf mapping_file = mapping.conf to provide the mapping.conf and castellan.conf you will likely need ot use https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration... to add the files to the reslevent container that need them and have a separate role to ocpy them.
Of course you still need the backend securely configured with per-service client certs authorized for the specific blobs, and this only works for services relying on oslo.config. I'm not sure if there's a documented walkthrough for this sort of deployment, nor whether Kolla has any integrated support for setting it up.
there is a bit of a chicken and egg situraiton depending on what backend you wanted to use i.e. if you wnated to store thing in barbican you would have to dlepoy it first without useing this then the other sevices. but if you were ot use an external barbican or vault you may be able to make this work today with enough effort. you would have to manulaly overreid the password or other filed you wnat to come form castellan using kolls genering service cofnig overried method. https://docs.openstack.org/kolla-ansible/latest/admin/advanced-configuration... then manually config the secrete in the relevent data store. but to do this properly in a maintainable way you would want a new flag to disbale some or all of the config generation. you may also want to automate the process for manginging the configs in the secret backend but im not enritly sure if you would want ot do that in kolla-ansible. ansibel itself can use a number of inventory datasouces beyond just files so i guess you could perhaps do that security but if your just going to use files on disk to provie the data to kolla it kind of defeats the exercise of doing it this way in the first place.
I've added ops and security-sig to the list of subject tags in case it bubbles this up to the attention of anyone in those circles who knows the answer.