<div dir="ltr"><div class="gmail_quote"><br><div dir="ltr"><div class="m_8140226343908425421gmail_signature"><div dir="ltr"><div>I have created question on ask.opestack, but profile is still not approved by moderators. (<a href="https://ask.openstack.org/en/question/114803/network-is-unreachable-in-instance/" target="_blank">https://ask.openstack.org/en/<wbr>question/114803/network-is-<wbr>unreachable-in-instance/</a>) Someone in IRC suggested to send the mail here.</div><div><br></div><div><br></div><div dir="ltr">Problem seems to be similar to, but I am not able to follow the bug to fix the issue - <a href="https://ask.openstack.org/en/question/80462/network-is-unreachable-in-instance/" target="_blank">https://ask.openstack.org/en/<wbr>question/80462/network-is-<wbr>unreachable-in-instance/</a></div><div dir="ltr"><br></div><div dir="ltr">We have 3 controller nodes connected to 12 compute nodes. Eth3 port on all the compute node is down.<br></div><div dir="ltr"><br></div><div dir="ltr">[Thu Jun 14 14:38:57] root@:~# node-ping-check</div><div dir="ltr"><br></div><div dir="ltr"> Ping check of all service-ip, vm-ip, data-ip and floating-ip, ping</div><div dir="ltr"> status result after ip, 1/0 = alive/dead</div><div dir="ltr"><br></div><div dir="ltr"> command start time 2018-06-14 14:39:00</div><div dir="ltr"><br></div><div dir="ltr"> node service-ip vm-ip data-ip floating-ip</div><div dir="ltr">------------------------------<wbr>------------------------------<wbr>---------</div><div dir="ltr"> h002 10.1.12.102 1 10.2.12.102 1 10.3.13.102 1 10.4.13.102 1 (controller 1)</div><div dir="ltr"> h003 10.1.12.103 1 10.2.12.103 1 10.3.13.103 1 10.4.13.103 1 (controller 2)</div><div dir="ltr"> h004 10.1.12.104 1 10.2.12.104 1 10.3.13.104 1 10.4.13.104 1 (controller 3 + Network node)</div><div dir="ltr"> h005 10.1.12.105 1 10.2.12.105 1 10.3.13.105 1 10.4.13.105 0</div><div dir="ltr"> h006 10.1.12.106 1 10.2.12.106 1 10.3.13.106 1 10.4.13.106 0</div><div dir="ltr"> h007 10.1.12.107 1 10.2.12.107 1 10.3.13.107 1 10.4.13.107 0</div><div dir="ltr"> h008 10.1.12.108 1 10.2.12.108 1 10.3.13.108 1 10.4.13.108 0</div><div dir="ltr"> h009 10.1.12.109 1 10.2.12.109 1 10.3.13.109 1 10.4.13.109 0</div><div dir="ltr"> h010 10.1.12.110 1 10.2.12.110 1 10.3.13.110 1 10.4.13.110 0</div><div dir="ltr"> h011 10.1.14.111 1 10.2.14.111 1 10.3.15.111 1 10.4.15.111 0</div><div dir="ltr"> h012 10.1.14.112 1 10.2.14.112 1 10.3.15.112 1 10.4.15.112 0</div><div dir="ltr"> h013 10.1.14.113 1 10.2.14.113 1 10.3.15.113 1 10.4.15.113 0</div><div dir="ltr"> h014 10.1.14.114 1 10.2.14.114 1 10.3.15.114 1 10.4.15.114 0</div><div dir="ltr"> h015 10.1.14.115 1 10.2.14.115 1 10.3.15.115 1 10.4.15.115 0</div><div dir="ltr"> h016 10.1.14.116 0 10.2.14.116 0 10.3.15.116 0 10.4.15.116 0 ( Down because of hardware issue)</div><div dir="ltr"> h017 10.1.14.117 1 10.2.14.117 1 10.3.15.117 1 10.4.15.117 0</div><div dir="ltr"><br></div><div dir="ltr">[Thu Jun 14 14:39:39] root@:~#</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">To bring eth3 on Network node followed following steps:</div><div dir="ltr"><br></div><div dir="ltr">ifconfig eth3 down</div><div dir="ltr">ifconfig eth3 0.0.0.0 up</div><div dir="ltr"><br></div><div dir="ltr">ovs-vsctl del-br br-ex</div><div dir="ltr">ovs-vsctl add-br br-ex</div><div dir="ltr">ovs-vsctl add-port br-ex eth3</div><div dir="ltr">ifconfig br-ex 10.4.13.104 netmask 255.255.255.0</div><div dir="ltr">route add -net 10.4.0.0 netmask 255.255.0.0 gateway 10.4.13.1 dev br-ex</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">On Network node:</div><div dir="ltr"><br></div><div dir="ltr">[2018-06-18 11:38:24] root@h004:~$ ip netns exec qdhcp-eb7573a0-9d91-491c-b150-<wbr>90f0135778ad ping 12.12.0.4</div><div dir="ltr">PING 12.12.0.4 (12.12.0.4) 56(84) bytes of data.</div><div dir="ltr">64 bytes from <a href="http://12.12.0.4" target="_blank">12.12.0.4</a>: icmp_seq=1 ttl=64 time=1.98 ms</div><div dir="ltr">64 bytes from <a href="http://12.12.0.4" target="_blank">12.12.0.4</a>: icmp_seq=2 ttl=64 time=0.529 ms</div><div dir="ltr">^C</div><div dir="ltr">--- 12.12.0.4 ping statistics ---</div><div dir="ltr">2 packets transmitted, 2 received, 0% packet loss, time 1001ms</div><div dir="ltr">rtt min/avg/max/mdev = 0.529/1.254/1.980/0.726 ms</div><div dir="ltr">[2018-06-18 11:38:33] root@h004:~$ ip netns exec qdhcp-eb7573a0-9d91-491c-b150-<wbr>90f0135778ad ssh <a href="mailto:cirros@12.12.0.4" target="_blank">cirros@12.12.0.4</a></div><div dir="ltr">^C</div><div dir="ltr"> [SSH Not working]</div><div dir="ltr"><br></div><div dir="ltr">[2018-06-18 11:45:51] root@h004:~$ ip netns exec qrouter-d770b552-1979-4063-<wbr>ae79-c6ff402c8cbc traceroute 8.8.8.8</div><div dir="ltr">traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets</div><div dir="ltr">connect: Network is unreachable</div><div dir="ltr">[2018-06-18 11:58:01] root@h004:~$ ip netns exec qrouter-d770b552-1979-4063-<wbr>ae79-c6ff402c8cbc traceroute 12.12.0.4</div><div dir="ltr">traceroute to 12.12.0.4 (12.12.0.4), 30 hops max, 60 byte packets</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">On Compute node:</div><div dir="ltr">[2018-06-18 11:56:12] root@h011:~$ ip netns exec qrouter-d770b552-1979-4063-<wbr>ae79-c6ff402c8cbc traceroute 8.8.8.8</div><div dir="ltr">traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets</div><div dir="ltr">connect: Network is unreachable</div><div dir="ltr"><br></div><div dir="ltr">[2018-06-18 11:59:20] root@h011:~$ ip netns exec qrouter-d770b552-1979-4063-<wbr>ae79-c6ff402c8cbc ping 12.12.0.4</div><div dir="ltr">PING 12.12.0.4 (12.12.0.4) 56(84) bytes of data.</div><div dir="ltr">64 bytes from <a href="http://12.12.0.4" target="_blank">12.12.0.4</a>: icmp_seq=1 ttl=64 time=1.61 ms</div><div dir="ltr">64 bytes from <a href="http://12.12.0.4" target="_blank">12.12.0.4</a>: icmp_seq=2 ttl=64 time=0.273 ms</div><div dir="ltr">^C</div><div dir="ltr">--- 12.12.0.4 ping statistics ---</div><div dir="ltr">2 packets transmitted, 2 received, 0% packet loss, time 1001ms</div><div dir="ltr">rtt min/avg/max/mdev = 0.273/0.943/1.614/0.671 ms</div><div dir="ltr">[2018-06-18 12:02:21] root@h011:~$ ip netns exec qrouter-d770b552-1979-4063-<wbr>ae79-c6ff402c8cbc ping x.x.170.61 (floating ip)</div><div dir="ltr">connect: Network is unreachable</div></div></div>
</div>
</div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><span lang="EN-GB" style="font-size:18pt;font-family:Webdings;color:green"><div style="font-family:arial,helvetica,sans-serif;display:inline"></div>P</span><span lang="EN-US" style="font-size:8pt;font-family:Arial,sans-serif;color:green"> </span><b style="font-family:Calibri,sans-serif;font-size:14.44444465637207px"><span lang="EN-US" style="font-size:8pt;color:green">SAVE PAPER – Please do not print this e-mail unless absolutely necessary.</span></b><br></div></div>
</div>