<div dir="ltr"><div dir="ltr"><div>Overall and besides of what Michal said, yeah, openstack hasn't been designed to be rebootable.</div><div>A looong time ago and before the migration to systemd services there was a script that restarted the screen instances [1]</div><div><br></div><div>That said, not there since 2016 so please restack whenever you reboot. In any case, was there any reason for rebooting?</div><div>I'd check if just restarting service would be enough for you.</div><div><br></div><div>Thanks!<br></div><div><br></div><div>[1] <a href="https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=9ba49cd82631ada84b8458b370ac535985d57725">https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=9ba49cd82631ada84b8458b370ac535985d57725</a><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 15, 2019 at 9:58 AM Michał Dulko <<a href="mailto:mdulko@redhat.com">mdulko@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Mon, 2019-04-15 at 11:44 +0900, Jaewook Oh wrote:<br>
> Hello kuryr-kubernetes team,<br>
> <br>
> currently, I deployed openstack with devstack, and I added "kuryr-kubernetes" to use kubernetes vim.<br>
> At first it worked fine and no error was occurred,<br>
> so I made some c-vnfs with kuryr-kubernetes.<br>
> But after rebooting the host, the systems that worked well before are not working normally at all.<br>
> <br>
> First, I deployed an example one ('redis') with kubectl command, (Not using Tacker, but only using kubectl cli)<br>
> it showed below error,<br>
> <br>
> Events: │obeTime":null,"lastTransitionTime":"2019-04-15T02:28:31Z"}],"hostIP":"192.168.1.20","startTime":"2019-04-15T02:28:31Z"<br>
> Type Reason Age From Message │,"containerStatuses":[{"name":"web-server","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":f<br>
> ---- ------ ---- ---- ------- │alse,"restartCount":0,"image":"kyleoh95/iistrc-k8s-demo2:0.4","imageID":""}],"qosClass":"Guaranteed"}}]}<br>
> Normal Scheduled 2m25s default-scheduler Successfully assigned default/redis-master-6fbbc44567-ms8n│Apr 15 11:36:52 master tacker-server[1881]: from (pid=1881) request /usr/local/lib/python2.7/dist-packages/kubernetes<br>
> k to master │/client/rest.py:219<br>
> Warning MissingClusterDNS 2m25s kubelet, master pod: "redis-master-6fbbc44567-ms8nk_default(8a6f9079-5f27-│Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.633 DEBUG tacker.vnfm.infra_drivers.kubernetes.kuberne<br>
> 11e9-9b42-a4bf01550f1a)". kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" pol│tes_driver [-] status: Pending from (pid=1881) create_wait /opt/stack/tacker/tacker/vnfm/infra_drivers/kubernetes/kube<br>
> icy. Falling back to "Default" policy. │rnetes_driver.py:137<br>
> Warning FailedCreatePodSandBox 19s kubelet, master Failed create pod sandbox: rpc error: code = Unknown desc │Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.634 DEBUG tacker.vnfm.infra_drivers.kubernetes.kuberne<br>
> = failed to set up sandbox container "310e18b859e118ad6abcaa60d29a9e332da516ce57b8cdf47456ea4882ca66f5" network for pod│tes_driver [-] VNF initializing status: ['default', 'svc-vdu1-b9e851'] Pending from (pid=1881) create_wait /opt/stack/<br>
> "redis-master-6fbbc44567-ms8nk": NetworkPlugin cni failed to set up pod "redis-master-6fbbc44567-ms8nk_default" networ│tacker/tacker/vnfm/infra_drivers/kubernetes/kubernetes_driver.py:141<br>
> k: Got invalid status code from CNI daemon.; Traceback (most recent call last): │Apr 15 11:36:52 master tacker-server[1881]: 2019-04-15 11:36:52.634 WARNING tacker.vnfm.infra_drivers.kubernetes.kuber<br>
> File "/opt/stack/kuryr-kubernetes/kuryr_kubernetes/cni/api.py", line 80, in run │netes_driver [-] VNF Creation failed: Resource creation is not completed within 500 seconds as creation of stack defau<br>
> vif = self._add(params)<br>
<br>
"Got invalid status code from CNI daemon" means that you should take a<br>
look at kuryr-cni pods logs.<br>
<br>
> <br>
> Are there any reason for this error? And how can I resolve this problem?<br>
<br>
In general I'm pretty sure DevStack is not designed to be rebootable,<br>
so it's probably some ip route being gone or some service that haven't<br>
restarted. If you really want to debug this instead of firing a new<br>
DevStack, start from checking all "devstack@*" services in systemd.<br>
<br>
> Best Regards,<br>
> Jaewook.<br>
> ================================================<br>
> Jaewook Oh (오재욱)<br>
> IISTRC - Internet Infra System Technology Research Center<br>
> 369 Sangdo-ro, Dongjak-gu,<br>
> 06978, Seoul, Republic of Korea<br>
> Tel : +82-2-820-0841 | Mobile : +82-10-9924-2618<br>
> E-mail : <a href="mailto:jwoh95@dcn.ssu.ac.kr" target="_blank">jwoh95@dcn.ssu.ac.kr</a><br>
> ================================================<br>
<br>
<br>
<br>
</blockquote></div>