[Kolla Ansible] host maintenance
Hi, I wonder if it's supported by Kolla Ansible to deploy a specific host and add it into existing cluster, like replace a control host or compute host? Thanks! Tony
Hi, seems like someone else is trying to migrate an existing setup to kolla 😉 We currently try it step by step. 1. Use kolla images instead of self developed builder. 2. Generate suitable kolla configuration file layout 3. Hopefully kolla-ansible will hand over But we are still in PoC state. Fabian Tony Liu <tonyliu0592@hotmail.com> schrieb am Fr., 21. Aug. 2020, 20:49:
Hi,
I wonder if it's supported by Kolla Ansible to deploy a specific host and add it into existing cluster, like replace a control host or compute host?
Thanks! Tony
Hello, if you are working on a migration to Kolla, there is a nice guide written by StackHPC that provides one example approach for this complicated maneuver: https://www.stackhpc.com/migrating-to-kolla.html Perhaps not relevant to your specific case, but it can offer some guidance! Cheers, /Jason On Aug 21, 2020, at 3:19 PM, Fabian Zimmermann <dev.faz@gmail.com<mailto:dev.faz@gmail.com>> wrote: Hi, seems like someone else is trying to migrate an existing setup to kolla 😉 We currently try it step by step. 1. Use kolla images instead of self developed builder. 2. Generate suitable kolla configuration file layout 3. Hopefully kolla-ansible will hand over But we are still in PoC state. Fabian Tony Liu <tonyliu0592@hotmail.com<mailto:tonyliu0592@hotmail.com>> schrieb am Fr., 21. Aug. 2020, 20:49: Hi, I wonder if it's supported by Kolla Ansible to deploy a specific host and add it into existing cluster, like replace a control host or compute host? Thanks! Tony
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible. Thanks! Tony
-----Original Message----- From: Jason Anderson <jasonanderson@uchicago.edu> Sent: Friday, August 21, 2020 1:45 PM To: Fabian Zimmermann <dev.faz@gmail.com> Cc: Tony Liu <tonyliu0592@hotmail.com>; openstack-discuss <openstack- discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
Hello, if you are working on a migration to Kolla, there is a nice guide written by StackHPC that provides one example approach for this complicated maneuver: https://www.stackhpc.com/migrating-to-kolla.html
Perhaps not relevant to your specific case, but it can offer some guidance!
Cheers, /Jason
On Aug 21, 2020, at 3:19 PM, Fabian Zimmermann <dev.faz@gmail.com <mailto:dev.faz@gmail.com> > wrote:
Hi,
seems like someone else is trying to migrate an existing setup to kolla 😉
We currently try it step by step.
1. Use kolla images instead of self developed builder. 2. Generate suitable kolla configuration file layout 3. Hopefully kolla-ansible will hand over
But we are still in PoC state.
Fabian
Tony Liu <tonyliu0592@hotmail.com <mailto:tonyliu0592@hotmail.com> > schrieb am Fr., 21. Aug. 2020, 20:49:
Hi,
I wonder if it's supported by Kolla Ansible to deploy a specific host and add it into existing cluster, like replace a control host or compute host?
Thanks! Tony
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install". Eric
Thanks Eric! I will run some tests to validate. Tony
-----Original Message----- From: Eric K. Miller <emiller@genesishosting.com> Sent: Friday, August 21, 2020 5:10 PM To: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: RE: [Kolla Ansible] host maintenance
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
Eric
On Sat, 22 Aug 2020 at 01:10, Eric K. Miller <emiller@genesishosting.com> wrote:
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
We need some better docs around this, and I think they will be added soon. Some things to watch out for: * if adding a new controller, ensure that if using --limit, all controllers are included and do not use serial mode * if removing a controller, reconfigure other controllers to update the RabbitMQ & Galera cluster nodes etc.
Eric
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 12:46 AM To: Eric K. Miller <emiller@genesishosting.com> Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Sat, 22 Aug 2020 at 01:10, Eric K. Miller <emiller@genesishosting.com> wrote:
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to
only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
We need some better docs around this, and I think they will be added soon. Some things to watch out for:
* if adding a new controller, ensure that if using --limit, all controllers are included and do not use serial mode
What I tried was to replace a controller, where I don't need to update other controllers, because there is no address update. If there is address update caused by controller change, then all controllers have to be included to get update. What's "serial mode"?
* if removing a controller, reconfigure other controllers to update the RabbitMQ & Galera cluster nodes etc.
In this case, are those services who don't need any updates going to be restarted or untouched? Thanks! Tony
On Mon, 24 Aug 2020 at 17:53, Tony Liu <tonyliu0592@hotmail.com> wrote:
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 12:46 AM To: Eric K. Miller <emiller@genesishosting.com> Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Sat, 22 Aug 2020 at 01:10, Eric K. Miller <emiller@genesishosting.com> wrote:
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to
only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
We need some better docs around this, and I think they will be added soon. Some things to watch out for:
* if adding a new controller, ensure that if using --limit, all controllers are included and do not use serial mode
What I tried was to replace a controller, where I don't need to update other controllers, because there is no address update.
If there is address update caused by controller change, then all controllers have to be included to get update.
While this may work at the moment, we have just merged a change that prevents this. For keystone, we need access to all controllers, to determine whether it is a new cluster or a new node in an existing cluster.
What's "serial mode"?
Ansible has a feature to run plays in batches of some % of the hosts. In Kolla Ansible you can e.g. export ANSIBLE_SERIAL=0.1. It's an advanced use case and needs some care.
* if removing a controller, reconfigure other controllers to update the RabbitMQ & Galera cluster nodes etc.
In this case, are those services who don't need any updates going to be restarted or untouched?
Thanks! Tony
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 11:21 AM To: Tony Liu <tonyliu0592@hotmail.com> Cc: Eric K. Miller <emiller@genesishosting.com>; openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Mon, 24 Aug 2020 at 17:53, Tony Liu <tonyliu0592@hotmail.com> wrote:
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 12:46 AM To: Eric K. Miller <emiller@genesishosting.com> Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Sat, 22 Aug 2020 at 01:10, Eric K. Miller <emiller@genesishosting.com> wrote:
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to
only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
We need some better docs around this, and I think they will be added soon. Some things to watch out for:
* if adding a new controller, ensure that if using --limit, all controllers are included and do not use serial mode
What I tried was to replace a controller, where I don't need to update other controllers, because there is no address update.
If there is address update caused by controller change, then all controllers have to be included to get update.
While this may work at the moment, we have just merged a change that prevents this. For keystone, we need access to all controllers, to determine whether it is a new cluster or a new node in an existing cluster.
What's "serial mode"?
Ansible has a feature to run plays in batches of some % of the hosts. In Kolla Ansible you can e.g. export ANSIBLE_SERIAL=0.1. It's an advanced use case and needs some care.
* if removing a controller, reconfigure other controllers to update the RabbitMQ & Galera cluster nodes etc.
In this case, are those services who don't need any updates going to be restarted or untouched?
Could you comment on this? This is my biggest concern. I'd like to ensure services who don't need update remain untouched. Thanks! Tony
On Mon, 24 Aug 2020 at 19:50, Tony Liu <tonyliu0592@hotmail.com> wrote:
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 11:21 AM To: Tony Liu <tonyliu0592@hotmail.com> Cc: Eric K. Miller <emiller@genesishosting.com>; openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Mon, 24 Aug 2020 at 17:53, Tony Liu <tonyliu0592@hotmail.com> wrote:
-----Original Message----- From: Mark Goddard <mark@stackhpc.com> Sent: Monday, August 24, 2020 12:46 AM To: Eric K. Miller <emiller@genesishosting.com> Cc: openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [Kolla Ansible] host maintenance
On Sat, 22 Aug 2020 at 01:10, Eric K. Miller <emiller@genesishosting.com> wrote:
Actually, in my case, the setup is originally deploy by Kolla Ansible. Other than the initial deployment, I am looking for using Kolla Ansible for maintenance operations. What I am looking for, eg. replace a host, can surely be done by manual steps or customized script. I'd like to know if they are automated by Kolla Ansible.
We do this often by simply using the "limit" flag in Kolla Ansible to
only include the controllers and new compute node (after adding the compute node to the multinode.ini file). Specify "reconfigure" for the action, and not "install".
We need some better docs around this, and I think they will be added soon. Some things to watch out for:
* if adding a new controller, ensure that if using --limit, all controllers are included and do not use serial mode
What I tried was to replace a controller, where I don't need to update other controllers, because there is no address update.
If there is address update caused by controller change, then all controllers have to be included to get update.
While this may work at the moment, we have just merged a change that prevents this. For keystone, we need access to all controllers, to determine whether it is a new cluster or a new node in an existing cluster.
What's "serial mode"?
Ansible has a feature to run plays in batches of some % of the hosts. In Kolla Ansible you can e.g. export ANSIBLE_SERIAL=0.1. It's an advanced use case and needs some care.
* if removing a controller, reconfigure other controllers to update the RabbitMQ & Galera cluster nodes etc.
In this case, are those services who don't need any updates going to be restarted or untouched?
Could you comment on this? This is my biggest concern. I'd like to ensure services who don't need update remain untouched.
In general, Kolla Ansible will only restart containers if the config files or container configuration changes. There is a bug in Ansible which means that this isn't always true, e.g. if nova-api needs to restart, we may also restart nova-conductor on the same host. See https://bugs.launchpad.net/kolla-ansible/+bug/1863510
Thanks! Tony
participants (5)
-
Eric K. Miller
-
Fabian Zimmermann
-
Jason Anderson
-
Mark Goddard
-
Tony Liu