Hi all, a couple of weeks ago I updated the containers in my 2023.2 release. kolla-ansible pull kolla-ansible deploy since then the Vexxhost driver in Magnum is no longer working. I have clusters stuck in provisioning and deleting: $ kubectl get cluster -A NAMESPACE NAME CLUSTERCLASS PHASE AGE VERSION magnum-system kube-6r0du magnum-v0.26.0 Provisioning 20d v1.26.7 magnum-system kube-7ljg3 magnum-v0.26.0 Provisioning 18d v1.26.7 magnum-system kube-csf6s magnum-v0.13.4 Deleting 71d v1.26.7 I believe this is due to a versionn mismatch. Many different tools are involved here. All pods in the KIND Management cluster are running: $ kubectl -n kube-system get pods NAME READY STATUS RESTARTS AGE coredns-76f75df574-lfckv 1/1 Running 5 (2d23h ago) 71d coredns-76f75df574-n85fv 1/1 Running 5 (2d23h ago) 71d etcd-magnumcapi-control-plane 1/1 Running 5 (2d23h ago) 71d kindnet-jtdtm 1/1 Running 5 (2d23h ago) 71d kube-apiserver-magnumcapi-control-plane 1/1 Running 5 (2d23h ago) 71d kube-controller-manager-magnumcapi-control-plane 1/1 Running 10 (2d23h ago) 71d kube-proxy-qwjc8 1/1 Running 5 (2d23h ago) 71d kube-scheduler-magnumcapi-control-plane 1/1 Running 9 (2d23h ago) 71d There is nothing obvious in the capi or capo logs. Is there someone that is currently running 2023.2 or newer with the Vexxhost CAPI driver? Would be great if you could share the versions of: Management Cluster $ kind --version kind version 0.22.0 $ export KUBECONFIG=~/config $ kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME magnumcapi-control-plane Ready control-plane 71d v1.29.2 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 5.14.0-503.38.1.el9_5.x86_64 containerd://1.7.13 $ kubectl get provider -A NAMESPACE NAME AGE TYPE PROVIDER VERSION capi-kubeadm-bootstrap-system bootstrap-kubeadm 71d BootstrapProvider kubeadm v1.9.5 capi-kubeadm-control-plane-system control-plane-kubeadm 71d ControlPlaneProvider kubeadm v1.9.5 capi-system cluster-api 71d CoreProvider cluster-api v1.9.5 capo-system infrastructure-openstack 71d InfrastructureProvider openstack v0.10.8 The magnum pods: ~]# podman exec -it magnum_conductor bash ()[ /]$ pip list | grep api magnum-cluster-api 0.26.0 ()[ /]$ helm version version.BuildInfo{Version:"v3.13.2", GitCommit:"2a2fb3b98829f1e0be6fb18af2f6599e0f4e8243", GitTreeState:"clean", GoVersion:"go1.20.10"} ~]# podman exec -it magnum_api bash ()[ /]$ pip list | grep api magnum-cluster-api 0.26.0 ()[ /]$ helm version version.BuildInfo{Version:"v3.13.2", GitCommit:"2a2fb3b98829f1e0be6fb18af2f6599e0f4e8243", GitTreeState:"clean", GoVersion:"go1.20.10"} ()[ /]$ Cheers, Oliver