Hi everyone Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet). Now I need to re-install everything. I'm searching the «best» method to do that. My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph. The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading. I'm thinking of kolla, what do you think ? I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ? With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ? What would you recommend ? Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 17:13:54 CET
So I think if you are looking for bare metal deployment specifically, OpenStack-Ansible would be pretty much your only option here. And OSA supports integration with external Ceph clusters (as rest tooling ofc). But both kolla/helm require having docker/k8s. And kolla-ansible is still an independent project. While openstack-helm leveraging kolla images - these are maintained by different groups as of today. Also, I don't really think you will get limited in OpenStack features specifically due to deployment tooling - they all will provide you with roughly the same feature set, just a completely different concepts of how your infrastructure is managed. ср, 5 мар. 2025 г. в 17:19, Albert Shih <Albert.Shih@obspm.fr>:
Hi everyone
Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet).
Now I need to re-install everything. I'm searching the «best» method to do that.
My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph.
The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading.
I'm thinking of kolla, what do you think ?
I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ?
With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ?
What would you recommend ?
Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 17:13:54 CET
Hi, Albert Couple additional clarifications: - Openstack-Helm and Kolla are independent projects with different approaches to deployment/upgrades. Openstack-Helm deploys Openstack on top of K8s (which is a container orchestrator). Kolla also deploys Openstack in containers but it utilizes Ansible for managing containers. Containers are just a convenient way of delivering software when you depend less on the software installed on the host. Due to this, containerized applications are usually considered a bit easier to upgrade. Containers are a thin layer that almost doesn't affect the performance but at the same time allows you to manage resources more carefully. - If you decide to go with K8s and Openstack-Helm then keep in mind that K8s itself requires some resources to operate. And you should have a good understanding of the K8s concepts. Nowadays managing K8s itself isn't difficult at all. Kubeadm is an official tool, but there are other options. - Openstack-Helm is image agnostic and brings all necessary scripts/configuration files with it. So potentially it can be used with Kolla images but we didn't test it with them. Openstack-Helm builds its own images [1]. [1] https://opendev.org/openstack/loci.git On Wed, Mar 5, 2025 at 10:32 AM Dmitriy Rabotyagov <noonedeadpunk@gmail.com> wrote:
So I think if you are looking for bare metal deployment specifically, OpenStack-Ansible would be pretty much your only option here. And OSA supports integration with external Ceph clusters (as rest tooling ofc).
But both kolla/helm require having docker/k8s. And kolla-ansible is still an independent project. While openstack-helm leveraging kolla images - these are maintained by different groups as of today.
Also, I don't really think you will get limited in OpenStack features specifically due to deployment tooling - they all will provide you with roughly the same feature set, just a completely different concepts of how your infrastructure is managed.
ср, 5 мар. 2025 г. в 17:19, Albert Shih <Albert.Shih@obspm.fr>:
Hi everyone
Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet).
Now I need to re-install everything. I'm searching the «best» method to
do
that.
My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph.
The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading.
I'm thinking of kolla, what do you think ?
I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ?
With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ?
What would you recommend ?
Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 17:13:54 CET
-- Best regards, Kozhukalov Vladimir
Le 05/03/2025 à 11:35:25-0600, Vladimir Kozhukalov a écrit Hi,
Couple additional clarifications:
Thanks.
- Openstack-Helm and Kolla are independent projects with different approaches to deployment/upgrades. Openstack-Helm deploys Openstack on top of K8s (which is a container orchestrator). Kolla also deploys Openstack in containers but it utilizes Ansible for managing containers. Containers are just a convenient way of delivering software when you depend less on the software installed on the host. Due to this, containerized applications are usually considered a bit easier to upgrade. Containers are a thin layer that almost doesn't affect the performance but at the same time allows you to manage resources more carefully.
- If you decide to go with K8s and Openstack-Helm then keep in mind that K8s itself requires some resources to operate. And you should have a good understanding of the K8s concepts. Nowadays managing K8s itself isn't difficult at all. Kubeadm is an official tool, but there are other options.
Yes I know (at my expense ;-) ) the k8s is also some nightmare ;-) ;-) But what I not sure I understand is with openstack-helm is «where» the openstack will run ? Is it going to run on worker manage by the k8s ? Because in my k8s (openshift/okd) are running on manually deploy kvm VM, and for our purpose those kvm are pretty «small» (mostly run website). Or the openshift-helm running inside the k8s will deploy the openstack in some bare-metal server ? And those bare-metal server will continue to run if the k8s «disappear» ?
- Openstack-Helm is image agnostic and brings all necessary scripts/ configuration files with it. So potentially it can be used with Kolla images but we didn't test it with them. Openstack-Helm builds its own images [1].
Ok. Thanks. Regards JAS -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 18:50:46 CET
Le 05/03/2025 à 17:31:39+0100, Dmitriy Rabotyagov a écrit Hi,
So I think if you are looking for bare metal deployment specifically, OpenStack-Ansible would be pretty much your only option here. And OSA supports integration with external Ceph clusters (as rest tooling ofc).
Well....I was not very clear about the deployment. I got some bare metal server on what I will run openstack. If the tool can deploy directly on the bare metal that's the cherry on the top. But I can also install the OS manually on the bare metal then deploy the openstack. I don't have 100 servers so manually deployment of the OS is not a issue.
But both kolla/helm require having docker/k8s. And kolla-ansible is
For Helm I can understand, but in which way kolla need a k8s ? I've a OKD (openshift) so I can deploy helm chart.
still an independent project. While openstack-helm leveraging kolla
Ok thanks
images - these are maintained by different groups as of today.
So can I say kolla :--> build docker image for openstack then I can deploy myself with whatever those image to run a openstack, or I can use kolla-ansible to deploy kolla images ?
Also, I don't really think you will get limited in OpenStack features specifically due to deployment tooling - they all will provide you
Ok. Thanks.
with roughly the same feature set, just a completely different concepts of how your infrastructure is managed.
Ok. Thanks Regards. JAS -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mer. 05 mars 2025 18:41:48 CET
I have been using openstack-ansible with some luck and the community has been helpful. For managing the hardware I have been using the-foreman which makes booting and installing the OS on systems much easier. For Kolla there is a book "Mastering OpenStack" that has just been released so it should be a good guide. On 2025-03-05 12:50, Albert Shih wrote:
Le 05/03/2025 à 17:31:39+0100, Dmitriy Rabotyagov a écrit Hi,
So I think if you are looking for bare metal deployment specifically, OpenStack-Ansible would be pretty much your only option here. And OSA supports integration with external Ceph clusters (as rest tooling ofc). Well....I was not very clear about the deployment. I got some bare metal server on what I will run openstack. If the tool can deploy directly on the bare metal that's the cherry on the top. But I can also install the OS manually on the bare metal then deploy the openstack. I don't have 100 servers so manually deployment of the OS is not a issue.
But both kolla/helm require having docker/k8s. And kolla-ansible is For Helm I can understand, but in which way kolla need a k8s ? I've a OKD (openshift) so I can deploy helm chart.
still an independent project. While openstack-helm leveraging kolla Ok thanks
images - these are maintained by different groups as of today. So can I say
kolla :--> build docker image for openstack
then I can deploy myself with whatever those image to run a openstack, or I can use kolla-ansible to deploy kolla images ?
Also, I don't really think you will get limited in OpenStack features specifically due to deployment tooling - they all will provide you Ok. Thanks.
with roughly the same feature set, just a completely different concepts of how your infrastructure is managed. Ok. Thanks
Regards.
JAS
-- Alvin Starr || land: (647)478-6285 Netvel Inc. || home: (905)513-7688 alvin@netvel.net ||
But what I not sure I understand is with openstack-helm is «where» the openstack will run ? Is it going to run on worker manage by the k8s ?
Openstack components (APIs, agents, etc.) like any other workloads that you usually deploy on top of K8s will be running on K8s worker nodes. This assumes VMs also will be running on K8s worker nodes but VMs will NOT be running in containers and will NOT be managed by K8s. Openstack will be fully responsible for managing VMs. You can set labels on k8s nodes and this is how Openstack-Helm defines which Openstack components to run on which nodes. On Wed, Mar 5, 2025 at 12:35 PM Alvin Starr <alvin@netvel.net> wrote:
I have been using openstack-ansible with some luck and the community has been helpful. For managing the hardware I have been using the-foreman which makes booting and installing the OS on systems much easier.
For Kolla there is a book "Mastering OpenStack" that has just been released so it should be a good guide.
Le 05/03/2025 à 17:31:39+0100, Dmitriy Rabotyagov a écrit Hi,
So I think if you are looking for bare metal deployment specifically, OpenStack-Ansible would be pretty much your only option here. And OSA supports integration with external Ceph clusters (as rest tooling ofc). Well....I was not very clear about the deployment. I got some bare metal server on what I will run openstack. If the tool can deploy directly on
On 2025-03-05 12:50, Albert Shih wrote: the
bare metal that's the cherry on the top. But I can also install the OS manually on the bare metal then deploy the openstack. I don't have 100 servers so manually deployment of the OS is not a issue.
But both kolla/helm require having docker/k8s. And kolla-ansible is For Helm I can understand, but in which way kolla need a k8s ? I've a OKD (openshift) so I can deploy helm chart.
still an independent project. While openstack-helm leveraging kolla Ok thanks
images - these are maintained by different groups as of today. So can I say
kolla :--> build docker image for openstack
then I can deploy myself with whatever those image to run a openstack, or I can use kolla-ansible to deploy kolla images ?
Also, I don't really think you will get limited in OpenStack features specifically due to deployment tooling - they all will provide you Ok. Thanks.
with roughly the same feature set, just a completely different concepts of how your infrastructure is managed. Ok. Thanks
Regards.
JAS
-- Alvin Starr || land: (647)478-6285 Netvel Inc. || home: (905)513-7688 alvin@netvel.net ||
-- Best regards, Kozhukalov Vladimir
Le 05/03/2025 à 12:56:36-0600, Vladimir Kozhukalov a écrit Hi,
But what I not sure I understand is with openstack-helm is «where» the openstack will run ? Is it going to run on worker manage by the k8s ?
Openstack components (APIs, agents, etc.) like any other workloads that you usually deploy on top of K8s will be running on K8s worker nodes. This assumes VMs also will be running on K8s worker nodes but VMs will NOT be running in containers and will NOT be managed by K8s. Openstack will be fully responsible for managing VMs.
OK. I only know k8s with OKD/openshift so maybe I'm asking a stupid question, but is that also mean the network traffic will not go through the multiple ha-proxy of OKD ?
You can set labels on k8s nodes and this is how Openstack-Helm defines which Openstack components to run on which nodes.
Ok that's I know ;-) Thanks again. Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: jeu. 06 mars 2025 07:41:05 CET
Hi My two cents - "ease" of upgrade is more related to planning the OpenStack components setup than to the deployment method An alternative for bare metal is use a couple of servers as KVM hypervisors to host management VMs (Keystone, Cinder etc) and install Nova compute on the other ones, automating with Foreman+Puppet -- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples Email: francesco.dinucci@na.infn.it On 05/03/25 17:19, Albert Shih wrote:
Hi everyone
Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet).
Now I need to re-install everything. I'm searching the «best» method to do that.
My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph.
The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading.
I'm thinking of kolla, what do you think ?
I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ?
With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ?
What would you recommend ?
Regards.
is that also mean the network traffic will not go through the multiple ha-proxy of OKD ?
I am not aware of all the details of OKD, but I assume ha-proxy is used as an ingress balancer. So the management traffic (when you access to Openstack API from outside) will be balanced between Openstack API pods. However the VM traffic is managed by Openstack networking components (Neutron, Octavia, ...). On Fri, Mar 7, 2025 at 3:03 AM Francesco Di Nucci < francesco.dinucci@na.infn.it> wrote:
Hi
My two cents - "ease" of upgrade is more related to planning the OpenStack components setup than to the deployment method
An alternative for bare metal is use a couple of servers as KVM hypervisors to host management VMs (Keystone, Cinder etc) and install Nova compute on the other ones, automating with Foreman+Puppet
-- Francesco Di Nucci System Administrator Compute & Networking Service, INFN Naples
Email: francesco.dinucci@na.infn.it
On 05/03/25 17:19, Albert Shih wrote:
Hi everyone
Long time ago I deploy a openstack mostly manually, then I integrated in our deployment tool (puppet).
Now I need to re-install everything. I'm searching the «best» method to do that.
My deployment would be on bare-metal, I don't need/want a deployment of CEPH (I already have my ceph cluster). Just need the cinder to use the ceph.
The main focus is the easy way to upgrade. I don't have a lot of hardware so I can afford to have a «test» cluster for upgrading.
I'm thinking of kolla, what do you think ?
I see they are now openstack-helm project, which If I understand correctly use a k8s to manage the deployment of openstack. Is that correct ?
With those tools is they are any limitation about the result ? Or can I got the most the main features of openstack ?
What would you recommend ?
Regards.
-- Best regards, Kozhukalov Vladimir
Le 07/03/2025 à 10:03:05+0100, Francesco Di Nucci a écrit Hi,
My two cents - "ease" of upgrade is more related to planning the OpenStack components setup than to the deployment method
An alternative for bare metal is use a couple of servers as KVM hypervisors to host management VMs (Keystone, Cinder etc) and install Nova compute on the other ones, automating with Foreman+Puppet
Yeah....we use intensively puppet. But that's not solve the «problem» with openstack. Long time ago (few years) when upgrading openstack they are lot of thing to do (change of config etc.) and the main issue is when you deploy with “apt install” (through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert. For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t. Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mar. 11 mars 2025 10:23:30 CET
through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert.
Well, that's pretty much the reason why in OpenStack-Ansible (and actually Kolla) we don't go this route by default, but installing most of the services as Python packages inside virtual environments. And repositories for supporting tools (like mariadb, rabbitmq, etc) are versioned. So if you have 3 control planes, 1 on Rocky Linux, 1 on Debian and 1 on Ubuntu - all of them will have same versions of mariadb/rabbitmq and OpenStack services, and they can work together. However, your biggest issue with revert is change of database and/or RPC schema, which makes revert way more touch. And I don't think there's anything which can solve the database schema issue during rollback without losing data which was created after upgrade. On Tue, 11 Mar 2025, 10:28 Albert Shih, <Albert.Shih@obspm.fr> wrote:
Le 07/03/2025 à 10:03:05+0100, Francesco Di Nucci a écrit Hi,
My two cents - "ease" of upgrade is more related to planning the
OpenStack
components setup than to the deployment method
An alternative for bare metal is use a couple of servers as KVM hypervisors to host management VMs (Keystone, Cinder etc) and install Nova compute on the other ones, automating with Foreman+Puppet
Yeah....we use intensively puppet.
But that's not solve the «problem» with openstack. Long time ago (few years) when upgrading openstack they are lot of thing to do (change of config etc.) and the main issue is when you deploy with “apt install” (through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert.
For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t.
Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mar. 11 mars 2025 10:23:30 CET
On 11/03/2025 09:35, Dmitriy Rabotyagov wrote:
through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert.
Well, that's pretty much the reason why in OpenStack-Ansible (and actually Kolla) we don't go this route by default, but installing most of the services as Python packages inside virtual environments. And repositories for supporting tools (like mariadb, rabbitmq, etc) are versioned. So if you have 3 control planes, 1 on Rocky Linux, 1 on Debian and 1 on Ubuntu - all of them will have same versions of mariadb/rabbitmq and OpenStack services, and they can work together.
However, your biggest issue with revert is change of database and/or RPC schema, which makes revert way more touch.
And I don't think there's anything which can solve the database schema issue during rollback without losing data which was created after upgrade.
nova and most openstack service don't support downgrade at all so ya unless you have done no resouce creation or moving as part of the upgrade and ahve a db backup, revert is basically not a concept in openstack upgrades. nor is it one that i think we could reasonably support in the future.
On Tue, 11 Mar 2025, 10:28 Albert Shih, <Albert.Shih@obspm.fr> wrote:
Le 07/03/2025 à 10:03:05+0100, Francesco Di Nucci a écrit Hi,
> > My two cents - "ease" of upgrade is more related to planning the OpenStack > components setup than to the deployment method > > An alternative for bare metal is use a couple of servers as KVM hypervisors > to host management VMs (Keystone, Cinder etc) and install Nova compute on > the other ones, automating with Foreman+Puppet
Yeah....we use intensively puppet.
But that's not solve the «problem» with openstack. Long time ago (few years) when upgrading openstack they are lot of thing to do (change of config etc.) and the main issue is when you deploy with “apt install” (through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert.
For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t.
Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: mar. 11 mars 2025 10:23:30 CET
Le 11/03/2025 à 10:35:03+0100, Dmitriy Rabotyagov a écrit
through puppet, manually or ansible or what-ever) it's complicated or even impossible to revert.
Well, that's pretty much the reason why in OpenStack-Ansible (and actually Kolla) we don't go this route by default, but installing most of the services as Python packages inside virtual environments.
Ok. Good to known.
And repositories for supporting tools (like mariadb, rabbitmq, etc) are versioned. So if you have 3 control planes, 1 on Rocky Linux, 1 on Debian and 1 on Ubuntu - all of them will have same versions of mariadb/rabbitmq and OpenStack services, and they can work together.
However, your biggest issue with revert is change of database and/or RPC schema, which makes revert way more touch.
And I don't think there's anything which can solve the database schema issue during rollback without losing data which was created after upgrade.
Yes I know that. But that's is the case for anything relaying on database. But for that It can be done by creating a dump of the database and yes we loose the data create after but it's better that than nothing. Regards. -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: ven. 14 mars 2025 08:54:09 CET
On 3/11/25 10:27, Albert Shih wrote:
For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t.
If that's not random bullshit, could you please be more specific on what Debian broke?!? I've been maintaining Debian packages since the inception of OpenStack in 2011, what you're talking about I never heard of it. Cheers, Thomas Goirand (zigo)
Hi, Proud user of these exact packages here: do not recognise it either? Cheers, Kees On 14-03-2025 15:23, Thomas Goirand wrote:
On 3/11/25 10:27, Albert Shih wrote:
For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t.
If that's not random bullshit, could you please be more specific on what Debian broke?!? I've been maintaining Debian packages since the inception of OpenStack in 2011, what you're talking about I never heard of it.
Cheers,
Thomas Goirand (zigo)
Le 14/03/2025 à 16:35:39+0100, Kees Meijs | Nefos a écrit Hi,
Proud user of these exact packages here: do not recognise it either?
Cheers, Kees
On 14-03-2025 15:23, Thomas Goirand wrote:
On 3/11/25 10:27, Albert Shih wrote:
For example when Debian upgrade xxxx from version a.b.c to a+1.b1.c1 and broke something in openstack we are in deep s*t.
If that's not random bullshit, could you please be more specific on what Debian broke?!? I've been maintaining Debian packages since the inception of OpenStack in 2011, what you're talking about I never heard of it.
You're right. My bad. My sentence was not clear. I didn't mean «the package made by Debian» broke, I mean «the package» himself broke something. I don't remember the exact version but some Ceph version broke some compatibility with (don't remember the version) of openstack. Even if it's quickly fix that's mean you have to revert. You also have some «home made»/«commercial» software who was not very good written and broke after a apt upgrade. Of course it's not Debian to blame but we need to downgrade the packages. Something it's very easy because it's just one or two packages with very few dependencies. But when it's got a lot of dependency....it's quickly become a nightmare. Regards -- Albert SHIH 🦫 🐸 Observatoire de Paris France Heure locale/Local time: dim. 16 mars 2025 09:57:03 CET
participants (8)
-
Albert Shih
-
Alvin Starr
-
Dmitriy Rabotyagov
-
Francesco Di Nucci
-
Kees Meijs | Nefos
-
Sean Mooney
-
Thomas Goirand
-
Vladimir Kozhukalov