<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Namrata,<div class=""><br class=""></div><div class="">This is a known issue being tracked under this story: <a href="https://storyboard.openstack.org/#!/story/2006846" class="">https://storyboard.openstack.org/#!/story/2006846</a></div><div class=""><br class=""></div><div class="">As I said before on IRC, the only known fix at the moment is to run Magnum Train with `use_podman=true` label. The solution for atomic is actively being investigated.</div><div class=""><br class=""></div><div class="">Best</div><div class=""><br class=""></div><div class="">Bharat<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On 19 Nov 2019, at 16:06, Namrata Sitlani <<a href="mailto:nsitlani03@gmail.com" class="">nsitlani03@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hello Folks,<br class=""><br class="">From Thursday last week (Nov 14), Magnum is unable to spin up working Kubernetes clusters. We run on Rocky Openstack release.<br class=""><br class="">All our Kubernetes pods show CrashLoopBackOff status.<br class=""><br class="">We use the following commands to create the Kubernetes clusters : <a href="http://paste.openstack.org/show/786348/" class="">http://paste.openstack.org/show/786348/</a> .<br class=""><br class="">The deployment fails with the following output : <a href="http://paste.openstack.org/show/786287/" class="">http://paste.openstack.org/show/786287/</a>.<br class=""><br class="">We tried deploying Kubernetes v1.13.12, v1.14.8, v1.15.5 and v1.16.2 without success. However, if we use version v1.14.6 we can successfully deploy our clusters. <br class="">Unfortunately, we cannot use v1.14.6 in production because it is not patched for the CVE-2019-11253 vulnerability .<br class=""><br class="">Since this stopped working for us on Thursday, we think that the image update <a href="https://hub.docker.com/r/openstackmagnum/kubernetes-apiserver/tags" class="">https://hub.docker.com/r/openstackmagnum/kubernetes-apiserver/tags</a> done 6 days ago is the culprit.<br class="">(We previously deployed clusters with versions v1.15.5 and v1.14.8 successfully.)<br class=""><br class="">Can you please confirm our findings and help us find a way forward? We can provide more logs if needed.<br class=""><br class="">Thank you very much,<br class="">Namrata Sitlani</div>
</div></blockquote></div><br class=""></div></body></html>