I have a kolla-ansible all-in-one openstack installation working on a Virtual Machine (Ubuntu22.04 on a ESXi 7.0 host) Reference: https://kifarunix.com/create-and-upload-custom-linux-image-into-openstack/?e... All went well and able to get the horizon dashboard and default Cirros image and demo networks up and running. Able to create instances using the default image. To create cloud OS images from the ISO files , I have done additionally the following iinstallation in the same Ubuntu22.04 VM . STEP 1. *$ sudo apt install -y qemu-kvm virt-manager libvirt-daemon-system virtinst libvirt-clients bridge-utils* $ sudo systemctl enable --now libvirtd $ sudo systemctl start libvirtd Everything went well. I am able to get virt-manager up and running and able to create the Cloud Image from an Ubuntu22.04 Server ISO. Then I uploaded this cloud image to the All-IN-ONE kolla-ansible openstack horizon dashboard, the image creation was successful. 2. But when I tried to create an instance by using this image through the horizon dashboard the instance dialog doesn't show any availability zone (nova). I am sure before doing the step 1 I am able to create Linux instances using the default Cirros image multiple times and the instances up and running successfully with the default availability zone "nova" ( I mean kolla-ansible all-in-one openstack installation was working fine before doing the step 1). But now there is no availability zones showing in horizon for creating new instances. Availability Zone There are no Availability Zones. I checked in CLI also $ openstack hypervisor list //it also shows ( down) availability zones also down 3. So I decided to reconfigure the $ kolla-ansible -i all-in-one reconfigure step, but the step fails with the following. RUNNING HANDLER [nova-cell : Restart nova-ssh container] *********************** changed: [localhost] RUNNING HANDLER [nova-cell : Restart nova-libvirt container] ******************* changed: [localhost] RUNNING HANDLER [nova-cell : Checking libvirt container is ready] ************** FAILED - RETRYING: [localhost]: Checking libvirt container is ready (10 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (9 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (8 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (7 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (6 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (5 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (4 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (3 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (1 retries left). fatal: [localhost]: FAILED! => {"attempts": 10, "changed": true, "cmd": "set -o pipefail && docker exec -i nova_libvirt ls /run/libvirtd.pid\n", "delta": "0:00:00.016249", "end": "2024-02-21 16:21:22.750454", "msg": "non-zero return code", "rc": 1, "start": "2024-02-21 16:21:22.734205", "stderr": "Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running", "stderr_lines": ["Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running"], "stdout": "", "stdout_lines": []} Each time I do reconfigure, the reconfiguration fails by reaching the step RUNNING HANDLER [nova-cell : Restart nova-libvirt container] ******************* changed: [localhost] 4. Why it fails, what the root cause of it, I suspect due to step 1 all the errors happened . 4. How can I recover the all-in-one openstack installation back and be able to create instances and images as earlier? Only option is to fresh deploy the kolla-ansible -i all-in-one ? Then all the current VM and images will all be lost ? ( If it in production what the method is to recover ?? OR will loss all data in production by running deployment again ? ) Greetings, Krish
Solved the issue .. Refer to the inline solution written here. On Thu, Feb 22, 2024 at 7:25 AM KK CHN <kkchn.in@gmail.com> wrote:
I have a kolla-ansible all-in-one openstack installation working on a Virtual Machine (Ubuntu22.04 on a ESXi 7.0 host)
Reference: https://kifarunix.com/create-and-upload-custom-linux-image-into-openstack/?e...
All went well and able to get the horizon dashboard and default Cirros image and demo networks up and running. Able to create instances using the default image.
To create cloud OS images from the ISO files , I have done additionally the following iinstallation in the same Ubuntu22.04 VM .
STEP 1.
*$ sudo apt install -y qemu-kvm virt-manager libvirt-daemon-system virtinst libvirt-clients bridge-utils*
$ sudo systemctl enable --now libvirtd
$ sudo systemctl start libvirtd
Everything went well. I am able to get virt-manager up and running and able to create the Cloud Image from an Ubuntu22.04 Server ISO. Then I uploaded this cloud image to the All-IN-ONE kolla-ansible openstack horizon dashboard, the image creation was successful.
2. But when I tried to create an instance by using this image through the horizon dashboard the instance dialog doesn't show any availability zone (nova).
I am sure before doing the step 1 I am able to create Linux instances using the default Cirros image multiple times and the instances up and running successfully with the default availability zone "nova"
( I mean kolla-ansible all-in-one openstack installation was working fine before doing the step 1).
But now there is no availability zones showing in horizon for creating new instances.
Availability Zone There are no Availability Zones.
I checked in CLI also $ openstack hypervisor list //it also shows ( down) availability zones also down
3. So I decided to reconfigure the $ kolla-ansible -i all-in-one reconfigure step, but the step fails with the following.
RUNNING HANDLER [nova-cell : Restart nova-ssh container] *********************** changed: [localhost]
RUNNING HANDLER [nova-cell : Restart nova-libvirt container] ******************* changed: [localhost]
RUNNING HANDLER [nova-cell : Checking libvirt container is ready] ************** FAILED - RETRYING: [localhost]: Checking libvirt container is ready (10 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (9 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (8 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (7 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (6 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (5 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (4 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (3 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left). FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (1 retries left). fatal: [localhost]: FAILED! => {"attempts": 10, "changed": true, "cmd": "set -o pipefail && docker exec -i nova_libvirt ls /run/libvirtd.pid\n", "delta": "0:00:00.016249", "end": "2024-02-21 16:21:22.750454", "msg": "non-zero return code", "rc": 1, "start": "2024-02-21 16:21:22.734205", "stderr": "Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running", "stderr_lines": ["Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running"], "stdout": "", "stdout_lines": []}
Each time I do reconfigure, the reconfiguration fails by reaching the step RUNNING HANDLER [nova-cell : Restart nova-libvirt container] ******************* changed: [localhost]
Remove the libvirt-sock // systemctl disable libvirt or removing the libvirt from your base VM not working .. Don't know why ? but removing libvirt-soc // Does .. *$ sudo rm –rf /run/libvirt/libvirt-soc * // I am not sure why it stuck with libvirt Unfortunately I couldn't test $kolla-ansible -i all-in-one reconfigure // As I have cleaned up all containers, network and images by using the kolla-ansible cleanup tools in the kolla-ansible tool directory. (I know this was not the way to do. I cleaned up the all-in-one installation hurrily .. so no chance to test/reproduce it. The reconfigure step might workd once you remove libvirt-spc // I couldn't test it this time. Any one has to add more inputs please add yours here.
4. Why it fails, what the root cause of it, I suspect due to step 1 all the errors happened .
4. How can I recover the all-in-one openstack installation back and be able to create instances and images as earlier?
Only option is to fresh deploy the kolla-ansible -i all-in-one ? Then all the current VM and images will all be lost ? (
If it in production what the method is to recover ?? OR will loss all data in production by running deployment again ? )
Greetings,
Krish
participants (1)
-
KK CHN