[kolla][wallaby] update images stop all vms

Ignazio Cassano ignaziocassano at gmail.com
Tue Aug 24 15:32:35 UTC 2021


Anycase, after an image update instances are down but also restarting them,
the system becomes very slow.
I do not know if because I am not really uploading the openstack images but
only retagging them to simulate an update.
The following are my steps:
- change images tag on local registry
- apply the new tag in globals.yml
- pull the images with new tag on computes and controllers.
- run deploy

Any suggestion, please?
Ignazio


Il Mar 24 Ago 2021, 13:47 Ignazio Cassano <ignaziocassano at gmail.com> ha
scritto:

> Hello Sean,
> I do not know what is "systemd-machined".
> My compute nodes are ubuntu 20.40, while images are centos source wallaby.
> Reading the logs is sure that nova compute lost connection with libvirtd,
> but in a installation where containers are not used (no kolla) this does
> not happen.
> What I must check, please ?
> Is the above bug considered still opened ?
> Ignazio
>
> Il giorno mar 24 ago 2021 alle ore 13:35 Sean Mooney <smooney at redhat.com>
> ha scritto:
>
>> On Tue, 2021-08-24 at 13:00 +0200, Ignazio Cassano wrote:
>> > I tried again. I started with e new installation of wallaby but this
>> time I
>> > used centos source instead of ubuntu source.
>> > After first deploy I stared a virtual machine.
>> > Then I update the images and I ran a new deploy.
>> > Instance went down and in the nova compute log I got the following:
>> > https://paste.openstack.org/show/808282/
>>
>> i dont know if its the same issue but you might be hitting this
>> https://bugzilla.redhat.com/show_bug.cgi?id=1963164
>> tl;dr is systemd-machined previously did not work inside the contaiener
>> so livert fell
>> back to its generic cgroups backend, ignoring machined managing mancines
>> without it but recently systemd-machined started
>> working in the contianer. when you upgrade libvirt switches to its
>> machined backend but it does not have the vms registred in that
>> so it stopps them.
>>
>>
>> >
>> > Ignazio
>> >
>> >
>> > Il giorno mar 24 ago 2021 alle ore 09:02 Ignazio Cassano <
>> > ignaziocassano at gmail.com> ha scritto:
>> >
>> > > Hello, about the above issue, we discussed on yesterday on
>> openstack-kolla
>> > > IRC.
>> > > The issue is not related to masakary, but it happens when
>> nova-compute is
>> > > redeployed with the following error in
>> > > linvirt.log:
>> > > 021-08-24 06:59:25.012+0000: 3224764: info : libvirt version: 6.0.0,
>> > > package: 0ubuntu8.12 (Christian Ehrhardt <
>> christian.ehrhardt at canonical.com>
>> > > Tue, 20 Jul 2021 14:13:56 +0200)
>> > > 2021-08-24 06:59:25.012+0000: 3224764: info : hostname: tst2-kvm02
>> > > 2021-08-24 06:59:25.012+0000: 3224764: error :
>> > > dnsmasqCapsRefreshInternal:714 : Cannot check dnsmasq binary
>> > > /usr/sbin/dnsmasq: No such file or directory
>> > > 2021-08-24 06:59:25.857+0000: 3224822: error :
>> qemuMonitorOpenUnix:292 :
>> > > failed to connect to monitor socket: Connection refused
>> > > 2021-08-24 06:59:25.876+0000: 3224821: error :
>> qemuMonitorOpenUnix:292 :
>> > > failed to connect to monitor socket: Connection refused
>> > > 2021-08-24 06:59:44.670+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-alpha' on this host
>> > > 2021-08-24 06:59:44.673+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-arm' on this host
>> > > 2021-08-24 06:59:44.674+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-arm' on this host
>> > > 2021-08-24 06:59:44.677+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-arm' on this host
>> > > 2021-08-24 06:59:44.681+0000: 3224749: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-aarch64' on this host
>> > > 2021-08-24 06:59:44.684+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-cris' on this host
>> > > 2021-08-24 06:59:44.713+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-lm32' on this host
>> > > 2021-08-24 06:59:44.716+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-m68k' on this host
>> > > 2021-08-24 06:59:44.719+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-microblaze' on this host
>> > > 2021-08-24 06:59:44.721+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-microblazeel' on this host
>> > > 2021-08-24 06:59:44.725+0000: 3224749: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-mips' on this host
>> > > 2021-08-24 06:59:44.727+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-mipsel' on this host
>> > > 2021-08-24 06:59:44.731+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-mips64' on this host
>> > > 2021-08-24 06:59:44.733+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-mips64el' on this host
>> > > 2021-08-24 06:59:44.736+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc' on this host
>> > > 2021-08-24 06:59:44.739+0000: 3224749: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64' on this host
>> > > 2021-08-24 06:59:44.741+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64' on this host
>> > > 2021-08-24 06:59:44.744+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64' on this host
>> > > 2021-08-24 06:59:44.747+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64le' on this host
>> > > 2021-08-24 06:59:44.750+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64le' on this host
>> > > 2021-08-24 06:59:44.753+0000: 3224749: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-ppc64le' on this host
>> > > 2021-08-24 06:59:44.755+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-riscv32' on this host
>> > > 2021-08-24 06:59:44.758+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-riscv64' on this host
>> > > 2021-08-24 06:59:44.761+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-s390x' on this host
>> > > 2021-08-24 06:59:44.763+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-sh4' on this host
>> > > 2021-08-24 06:59:44.766+0000: 3224749: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-sh4eb' on this host
>> > > 2021-08-24 06:59:44.768+0000: 3224751: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-sparc' on this host
>> > > 2021-08-24 06:59:44.771+0000: 3224753: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-sparc64' on this host
>> > > 2021-08-24 06:59:44.773+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-unicore32' on this host
>> > > 2021-08-24 06:59:44.795+0000: 3224750: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-xtensa' on this host
>> > > 2021-08-24 06:59:44.798+0000: 3224752: error :
>> > > virQEMUCapsCacheLookupDefault:5577 : invalid argument: KVM is not
>> supported
>> > > by '/usr/bin/qemu-system-xtensaeb' on this host
>> > >
>> > > Ignazio
>> > >
>> > > Il giorno lun 23 ago 2021 alle ore 17:51 Ignazio Cassano <
>> > > ignaziocassano at gmail.com> ha scritto:
>> > >
>> > > > Hello All,
>> > > > I am new in kolla so probably my procedure is not correct.
>> > > > Last week I installed  wallaby and I deployed on my openstack only
>> two
>> > > > virtual machines.
>> > > > Today I tried to updated images so:
>> > > >
>> > > > 1 pulled new images on my local registry
>> > > > 2 pushed new images on my local registry
>> > > > 3 pushed new images from my local registry to controllers and
>> compute
>> > > > nodes
>> > > > (kolla-ansible pull --limit control,compute
>> > > > 4 run kolla-ansible deploy
>> > > >
>> > > > All virtual machines went in shutoff state.
>> > > > What is wrong in my procedure ?
>> > > >
>> > > > Regards
>> > > > Ignazio
>> > > >
>> > >
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210824/5935ff77/attachment-0001.html>


More information about the openstack-discuss mailing list