Folks,
I have made very good progress, now I can see CAPI started building workload cluster but when I noticed its stuck in CREATE_IN_PROGRESS for long time then I hope into workload master node and noticed following error spitting up in logs
Look like some cinder-csi related error (I am using 1.27.x image to build cluster)
38u-xpvv4-fkp9f kubelet[1815]: I1219 21:30:22.631478 1815 scope.go:115] "RemoveContainer" containerID="00c2df6cf4e1e4cac15d0d2698fbe19adedb7c5b49c7102c99e3b273ef57335b"
Dec 19 21:30:22 kube-8f38u-xpvv4-fkp9f kubelet[1815]: E1219 21:30:22.632141 1815 pod_workers.go:1294] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"openstack-cloud-controller-manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=openstack-cloud-controller-manager pod=openstack-cloud-controller-manager-fhnhj_kube-system(7c1162c6-51d0-4207-b462-97e2536d302d)\"" pod="kube-system/openstack-cloud-controller-manager-fhnhj" podUID=7c1162c6-51d0-4207-b462-97e2536d302d
Dec 19 21:30:26 kube-8f38u-xpvv4-fkp9f kubelet[1815]: I1219 21:30:26.631784 1815 scope.go:115] "RemoveContainer" containerID="7f60974870b0eb472549de51777ea6660db2592687eafa05f2dcbd6db25bdfb1"
Dec 19 21:30:26 kube-8f38u-xpvv4-fkp9f kubelet[1815]: E1219 21:30:26.632424 1815 pod_workers.go:1294] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cinder-csi-plugin\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=cinder-csi-plugin pod=csi-cinder-nodeplugin-9fjwp_kube-system(0b8f67de-fdf2-4996-aab4-5998da795434)\"" pod="kube-system/csi-cinder-nodeplugin-9fjwp" podUID=0b8f67de-fdf2-4996-aab4-5998da795434
root@kube-8f38u-xpvv4-fkp9f:/etc/kubernetes# kubectl get deployments -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system calico-kube-controllers 0/1 1 0 23m
kube-system coredns 0/2 2 0 23m
kube-system csi-cinder-controllerplugin 0/1 1 0 23m
root@kube-8f38u-xpvv4-fkp9f:/etc/kubernetes# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kube-8f38u-default-worker-zpg2j-9w5pk Ready <none> 22m v1.27.4
kube-8f38u-default-worker-zpg2j-vp56f Ready <none> 22m v1.27.4
kube-8f38u-xpvv4-fkp9f Ready control-plane 24m v1.27.4
What else should I check or look for?