Hi Namrata,
This is a known issue being tracked under this story: https://storyboard.openstack.org/#!/story/2006846 https://storyboard.openstack.org/#!/story/2006846
As I said before on IRC, the only known fix at the moment is to run Magnum Train with `use_podman=true` label. The solution for atomic is actively being investigated.
Best
Bharat
On 19 Nov 2019, at 16:06, Namrata Sitlani nsitlani03@gmail.com wrote:
Hello Folks,
From Thursday last week (Nov 14), Magnum is unable to spin up working Kubernetes clusters. We run on Rocky Openstack release.
All our Kubernetes pods show CrashLoopBackOff status.
We use the following commands to create the Kubernetes clusters : http://paste.openstack.org/show/786348/ http://paste.openstack.org/show/786348/ .
The deployment fails with the following output : http://paste.openstack.org/show/786287/ http://paste.openstack.org/show/786287/.
We tried deploying Kubernetes v1.13.12, v1.14.8, v1.15.5 and v1.16.2 without success. However, if we use version v1.14.6 we can successfully deploy our clusters. Unfortunately, we cannot use v1.14.6 in production because it is not patched for the CVE-2019-11253 vulnerability .
Since this stopped working for us on Thursday, we think that the image update https://hub.docker.com/r/openstackmagnum/kubernetes-apiserver/tags https://hub.docker.com/r/openstackmagnum/kubernetes-apiserver/tags done 6 days ago is the culprit. (We previously deployed clusters with versions v1.15.5 and v1.14.8 successfully.)
Can you please confirm our findings and help us find a way forward? We can provide more logs if needed.
Thank you very much, Namrata Sitlani