Hello Sean, I do not know what is "systemd-machined". My compute nodes are ubuntu 20.40, while images are centos source wallaby. Reading the logs is sure that nova compute lost connection with libvirtd, but in a installation where containers are not used (no kolla) this does not happen. What I must check, please ? Is the above bug considered still opened ? Ignazio Il giorno mar 24 ago 2021 alle ore 13:35 Sean Mooney <smooney@redhat.com> ha scritto:
On Tue, 2021-08-24 at 13:00 +0200, Ignazio Cassano wrote:
I tried again. I started with e new installation of wallaby but this time I used centos source instead of ubuntu source. After first deploy I stared a virtual machine. Then I update the images and I ran a new deploy. Instance went down and in the nova compute log I got the following: https://paste.openstack.org/show/808282/
i dont know if its the same issue but you might be hitting this https://bugzilla.redhat.com/show_bug.cgi?id=1963164 tl;dr is systemd-machined previously did not work inside the contaiener so livert fell back to its generic cgroups backend, ignoring machined managing mancines without it but recently systemd-machined started working in the contianer. when you upgrade libvirt switches to its machined backend but it does not have the vms registred in that so it stopps them.
Ignazio
Il giorno mar 24 ago 2021 alle ore 09:02 Ignazio Cassano < ignaziocassano@gmail.com> ha scritto:
Hello, about the above issue, we discussed on yesterday on
openstack-kolla
IRC. The issue is not related to masakary, but it happens when nova-compute is redeployed with the following error in linvirt.log: 021-08-24 06:59:25.012+0000: 3224764: info : libvirt version: 6.0.0, package: 0ubuntu8.12 (Christian Ehrhardt < christian.ehrhardt@canonical.com> Tue, 20 Jul 2021 14:13:56 +0200) 2021-08-24 06:59:25.012+0000: 3224764: info : hostname: tst2-kvm02 2021-08-24 06:59:25.012+0000: 3224764: error : dnsmasqCapsRefreshInternal:714 : Cannot check dnsmasq binary /usr/sbin/dnsmasq: No such file or directory 2021-08-24 06:59:25.857+0000: 3224822: error : qemuMonitorOpenUnix:292 : failed to connect to monitor socket: Connection refused 2021-08-24 06:59:25.876+0000: 3224821: error : qemuMonitorOpenUnix:292 : failed to connect to monitor socket: Connection refused 2021-08-24 06:59:44.670+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-alpha' on this host 2021-08-24 06:59:44.673+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host 2021-08-24 06:59:44.674+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host 2021-08-24 06:59:44.677+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-arm' on this host 2021-08-24 06:59:44.681+0000: 3224749: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-aarch64' on this host 2021-08-24 06:59:44.684+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-cris' on this host 2021-08-24 06:59:44.713+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-lm32' on this host 2021-08-24 06:59:44.716+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-m68k' on this host 2021-08-24 06:59:44.719+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblaze' on this host 2021-08-24 06:59:44.721+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-microblazeel' on this host 2021-08-24 06:59:44.725+0000: 3224749: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips' on this host 2021-08-24 06:59:44.727+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-mipsel' on this host 2021-08-24 06:59:44.731+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64' on this host 2021-08-24 06:59:44.733+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-mips64el' on this host 2021-08-24 06:59:44.736+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc' on this host 2021-08-24 06:59:44.739+0000: 3224749: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host 2021-08-24 06:59:44.741+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host 2021-08-24 06:59:44.744+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64' on this host 2021-08-24 06:59:44.747+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host 2021-08-24 06:59:44.750+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host 2021-08-24 06:59:44.753+0000: 3224749: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-ppc64le' on this host 2021-08-24 06:59:44.755+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv32' on this host 2021-08-24 06:59:44.758+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-riscv64' on this host 2021-08-24 06:59:44.761+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-s390x' on this host 2021-08-24 06:59:44.763+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4' on this host 2021-08-24 06:59:44.766+0000: 3224749: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-sh4eb' on this host 2021-08-24 06:59:44.768+0000: 3224751: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc' on this host 2021-08-24 06:59:44.771+0000: 3224753: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-sparc64' on this host 2021-08-24 06:59:44.773+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-unicore32' on this host 2021-08-24 06:59:44.795+0000: 3224750: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensa' on this host 2021-08-24 06:59:44.798+0000: 3224752: error : virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not supported by '/usr/bin/qemu-system-xtensaeb' on this host
Ignazio
Il giorno lun 23 ago 2021 alle ore 17:51 Ignazio Cassano < ignaziocassano@gmail.com> ha scritto:
Hello All, I am new in kolla so probably my procedure is not correct. Last week I installed wallaby and I deployed on my openstack only two virtual machines. Today I tried to updated images so:
1 pulled new images on my local registry 2 pushed new images on my local registry 3 pushed new images from my local registry to controllers and compute nodes (kolla-ansible pull --limit control,compute 4 run kolla-ansible deploy
All virtual machines went in shutoff state. What is wrong in my procedure ?
Regards Ignazio