Open Stack

I am using kolla-ansible 6.0.0 with openstack_release set to master in
globals.yml in a production environment.

I am trying to fix an oslo_messaging.rpc.client.RemoteError when
undertaking server add volume
(https://bugs.launchpad.net/nova/+bug/1773393) and appear to have
tracked it down to an inconsistency of container versions.

kollo-ansible upgrade runs to completion without error but leaves the
running nova_compute containers (and possibly others) at an earlier
version.

The version running is
CONTAINER ID IMAGE        COMMAND       CREATED     STATUS     PORTS NAMES
944f620a445c 7dc2d4695962 "kolla_start" 4 weeks ago Up 4 hours nova_compute

whereas docker images -a shows
REPOSITORY TAG IMAGE ID CREATED SIZE
kolla/ubuntu-source-nova-compute master f2df8187f14e 15 hours ago 1.29GB
kolla/ubuntu-source-nova-compute <none> 582561ac010f 39 hours ago 1.29GB
kolla/ubuntu-source-nova-compute <none> 7dc2d4695962 3 months ago 1.22GB

This implies f2df8187f14e is the one I should be using.
The image 582561ac010f was after I tried to switch to queens from master
but without success due to a bootstrap_cinder problem:
Error during database migration: 
  "Database schema file with version 122 doesn't exist."
I tried to investigate this but without any obvious resolution.

All compute nodes show the same issue.

As per the notes on
https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html
I have checked and virt_type is set to kvm in nova.conf and so I cannot
see why the upgrade shouldnt have been successful.

How do I get kolla to use the latest version pulled?

Given I have running instances I am a little nervous of doing a deploy or
reconfigure.

Thanks for help.

Dave