Open Stack

Hi,

> Wiadomość napisana przez Jay See <jayachander.it at gmail.com> w dniu 18.06.2018, o godz. 12:48:
> 
> 
> I have created question on ask.opestack, but profile is still not approved by moderators. (https://ask.openstack.org/en/question/114803/network-is-unreachable-in-instance/) Someone in IRC suggested to send the mail here.
> 
> 
> Problem seems to be similar to, but I am not able to follow the bug to fix the issue - https://ask.openstack.org/en/question/80462/network-is-unreachable-in-instance/
> 
> We have 3 controller nodes connected to 12 compute nodes. Eth3 port on all the compute node is down.
> 
> [Thu Jun 14 14:38:57] root@:~# node-ping-check
> 
>  Ping check of all service-ip, vm-ip, data-ip and floating-ip, ping
>  status result after ip, 1/0 = alive/dead
> 
>  command start time 2018-06-14 14:39:00
> 
>  node    service-ip        vm-ip          data-ip        floating-ip
> ---------------------------------------------------------------------
>  h002   10.1.12.102 1   10.2.12.102 1   10.3.13.102 1   10.4.13.102 1 (controller 1)
>  h003   10.1.12.103 1   10.2.12.103 1   10.3.13.103 1   10.4.13.103 1 (controller 2)
>  h004   10.1.12.104 1   10.2.12.104 1   10.3.13.104 1   10.4.13.104 1 (controller 3 + Network node)
>  h005   10.1.12.105 1   10.2.12.105 1   10.3.13.105 1   10.4.13.105 0
>  h006   10.1.12.106 1   10.2.12.106 1   10.3.13.106 1   10.4.13.106 0
>  h007   10.1.12.107 1   10.2.12.107 1   10.3.13.107 1   10.4.13.107 0
>  h008   10.1.12.108 1   10.2.12.108 1   10.3.13.108 1   10.4.13.108 0
>  h009   10.1.12.109 1   10.2.12.109 1   10.3.13.109 1   10.4.13.109 0
>  h010   10.1.12.110 1   10.2.12.110 1   10.3.13.110 1   10.4.13.110 0
>  h011   10.1.14.111 1   10.2.14.111 1   10.3.15.111 1   10.4.15.111 0
>  h012   10.1.14.112 1   10.2.14.112 1   10.3.15.112 1   10.4.15.112 0
>  h013   10.1.14.113 1   10.2.14.113 1   10.3.15.113 1   10.4.15.113 0
>  h014   10.1.14.114 1   10.2.14.114 1   10.3.15.114 1   10.4.15.114 0
>  h015   10.1.14.115 1   10.2.14.115 1   10.3.15.115 1   10.4.15.115 0
>  h016   10.1.14.116 0   10.2.14.116 0   10.3.15.116 0   10.4.15.116 0  ( Down because of hardware issue)
>  h017   10.1.14.117 1   10.2.14.117 1   10.3.15.117 1   10.4.15.117 0
> 
> [Thu Jun 14 14:39:39] root@:~#
> 
> 
> To bring eth3 on Network node followed following steps:
> 
> ifconfig eth3 down
> ifconfig eth3 0.0.0.0 up
> 
> ovs-vsctl del-br br-ex
> ovs-vsctl add-br br-ex

When You removed and added again br-ex it has no any openflow rules configured. So there will be no traffic passed via this bridge at all please try to restart neutron-openvswitch-agent and check if rules are recreated.
If that will help, You can apply patch https://review.openstack.org/#/c/573696/ and check if that will help :)

> ovs-vsctl add-port br-ex eth3
> ifconfig br-ex 10.4.13.104 netmask 255.255.255.0
> route add -net 10.4.0.0 netmask 255.255.0.0 gateway 10.4.13.1 dev br-ex
> 
> 
> On Network node:
> 
> [2018-06-18 11:38:24] root at h004:~$ ip netns exec qdhcp-eb7573a0-9d91-491c-b150-90f0135778ad ping 12.12.0.4
> PING 12.12.0.4 (12.12.0.4) 56(84) bytes of data.
> 64 bytes from 12.12.0.4: icmp_seq=1 ttl=64 time=1.98 ms
> 64 bytes from 12.12.0.4: icmp_seq=2 ttl=64 time=0.529 ms
> ^C
> --- 12.12.0.4 ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms
> rtt min/avg/max/mdev = 0.529/1.254/1.980/0.726 ms
> [2018-06-18 11:38:33] root at h004:~$ ip netns exec qdhcp-eb7573a0-9d91-491c-b150-90f0135778ad ssh cirros at 12.12.0.4
> ^C
>  [SSH Not working]
> 
> [2018-06-18 11:45:51] root at h004:~$ ip netns exec qrouter-d770b552-1979-4063-ae79-c6ff402c8cbc traceroute 8.8.8.8
> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
> connect: Network is unreachable
> [2018-06-18 11:58:01] root at h004:~$ ip netns exec qrouter-d770b552-1979-4063-ae79-c6ff402c8cbc traceroute 12.12.0.4
> traceroute to 12.12.0.4 (12.12.0.4), 30 hops max, 60 byte packets
> 
> 
> On Compute node:
> [2018-06-18 11:56:12] root at h011:~$ ip netns exec qrouter-d770b552-1979-4063-ae79-c6ff402c8cbc traceroute 8.8.8.8
> traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
> connect: Network is unreachable
> 
> [2018-06-18 11:59:20] root at h011:~$ ip netns exec qrouter-d770b552-1979-4063-ae79-c6ff402c8cbc ping 12.12.0.4
> PING 12.12.0.4 (12.12.0.4) 56(84) bytes of data.
> 64 bytes from 12.12.0.4: icmp_seq=1 ttl=64 time=1.61 ms
> 64 bytes from 12.12.0.4: icmp_seq=2 ttl=64 time=0.273 ms
> ^C
> --- 12.12.0.4 ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms
> rtt min/avg/max/mdev = 0.273/0.943/1.614/0.671 ms
> [2018-06-18 12:02:21] root at h011:~$ ip netns exec qrouter-d770b552-1979-4063-ae79-c6ff402c8cbc ping x.x.170.61  (floating ip)
> connect: Network is unreachable
> 
> 
> 
> -- 
> ​P  SAVE PAPER – Please do not print this e-mail unless absolutely necessary.
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

— 
Slawek Kaplonski
Senior software engineer
Red Hat