Solved the issue .. Refer to the inline solution written here. 


On Thu, Feb 22, 2024 at 7:25 AM KK CHN <kkchn.in@gmail.com> wrote:
I have a kolla-ansible  all-in-one  openstack installation working on a Virtual Machine (Ubuntu22.04 on a ESXi 7.0  host)

Reference: https://kifarunix.com/create-and-upload-custom-linux-image-into-openstack/?expand_article=1

All went well and able to get the horizon dashboard and default Cirros image and  demo networks up and running. Able to create instances using the default image. 



 To  create cloud OS images from the  ISO files ,   I have done additionally the following iinstallation in the same Ubuntu22.04 VM .  


STEP 1.

$ sudo apt install -y qemu-kvm virt-manager libvirt-daemon-system virtinst libvirt-clients bridge-utils


 $ sudo systemctl enable --now libvirtd

$ sudo systemctl start libvirtd 

  Everything went well. I am able to  get     virt-manager up and running  and able to create the  Cloud Image from an Ubuntu22.04 Server ISO.    Then  I uploaded this cloud image  to  the All-IN-ONE kolla-ansible openstack horizon dashboard, the image creation was successful.


2. But when I tried to create an instance  by using this image  through the horizon dashboard   the instance dialog doesn't show any availability zone (nova).  

I am sure  before doing the step 1  I am able to create Linux instances using the default Cirros image multiple times  and the instances up and running successfully with the default availability zone "nova" 


( I mean kolla-ansible  all-in-one openstack installation was working fine  before doing the step 1).


But now  there is no availability zones showing in horizon for creating new instances. 

There are no Availability Zones.


I checked  in   CLI also  $ openstack hypervisor list   //it also shows (  down)   availability zones also down



3. So I decided to   reconfigure the  $ kolla-ansible -i all-in-one  reconfigure   step,  but the step fails with the following. 



RUNNING HANDLER [nova-cell : Restart nova-ssh container] ***********************
changed: [localhost]

RUNNING HANDLER [nova-cell : Restart nova-libvirt container] *******************
changed: [localhost]

RUNNING HANDLER [nova-cell : Checking libvirt container is ready] **************
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (10 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (9 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (8 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (7 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (6 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (5 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (4 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (3 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left).
FAILED - RETRYING: [localhost]: Checking libvirt container is ready (2 retries left).

FAILED - RETRYING: [localhost]: Checking libvirt container is ready (1 retries left).
fatal: [localhost]: FAILED! => {"attempts": 10, "changed": true, "cmd": "set -o pipefail && docker exec -i nova_libvirt ls /run/libvirtd.pid\n", "delta": "0:00:00.016249", "end": "2024-02-21 16:21:22.750454", "msg": "non-zero return code", "rc": 1, "start": "2024-02-21 16:21:22.734205", "stderr": "Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running", "stderr_lines": ["Error response from daemon: container 46d139a236f57e41cfcf3121b3879ea01eae774d771415c5418b003c839f3257 is not running"], "stdout": "", "stdout_lines": []}


Each time   I do reconfigure,   the  reconfiguration fails by reaching the step  RUNNING HANDLER [nova-cell : Restart nova-libvirt container] *******************

changed: [localhost]

Remove the libvirt-sock   // systemctl disable libvirt or removing the libvirt  from your base VM not working .. Don't know why ?


but removing   libvirt-soc   //  Does ..

$ sudo  rm –rf   /run/libvirt/libvirt-soc     //  I am not sure why it stuck with libvirt  


Unfortunately I couldn't  test 
 $kolla-ansible  -i all-in-one   reconfigure    // As I have cleaned up all containers, network and images  by using the  kolla-ansible cleanup  tools  in the  kolla-ansible tool directory. 
   (I know this was not the  way to do.   I cleaned up the all-in-one installation  hurrily .. so no chance to test/reproduce it.     The reconfigure step   might workd once you remove libvirt-spc   //  I couldn't test it this time. 

Any one has to add more inputs please add yours  here.  


4. Why it fails, what the root cause of it,   I suspect  due to step 1 all the errors happened .


4. How can I recover the   all-in-one openstack installation back  and be able to create instances and images as earlier?


 Only option is to   fresh deploy  the   kolla-ansible -i all-in-one ?   Then all the current VM and images will all be lost ? (

 If it in production what the method is to recover ?? OR will loss all data in production by running deployment again ? ) 


Greetings,

Krish