<div dir="ltr"><p dir="ltr">The status being DOWN is an issue. That means either the port binding failed because the host_id was not set on the port that matches the hostname where the openvswitch agent is running, or the openvswitch agent failed to identify the port from the container that was plugged in.</p><p>So there are two things. First do a port-show on that port with admin credentials so you can see if binding:host_id is set to the hostname of the node the container is on. </p><p>Second, I assume veth_h0 is the connection to the container? If that's the case, you will need to add the external ID to OVSDB that indicates the Neutron port it corresponds to. Here is an example command of how to add a port in a way that the openvswitch agent can correctly wire it up. Apologies for it being so hacky, it's just a quick script I had to test Neutron agent wiring without using VMs.</p><p>port_body=$(neutron port-show $PORT_UUID)</p><p>port_id=$(echo "$port_body" | grep "| id" | awk '{ print $4 }')</p><p>port_mac=$(echo "$port_body" | grep "| mac_address" | awk '{ print $4 }')</p><p>port_name=${port_id:0:11}</p><p>port_name="tap${port_name}"</p><p>sudo ovs-vsctl --timeout=120 -- --if-exists del-port $port_name -- add-port br-int $port_name -- set Interface $port_name external-ids:iface-id=$port_id external-ids:iface-status=active external-ids:attached-mac=$port_mac type=internal</p><p><br></p><p>The port_name may not need to be changed to match the partial port UUID format, but there used to be old code that depended on that name so it might be safest to first start with making your veth pair name match that format if you can.</p><p>The giveaway is that your veth_h0 port is missing a tag on br-int. If it doesn't have a tag (or if it has a tag of 4095), that means the openvswitch agent hasn't correctly wired it.</p><p><br></p><p><br></p><p><br></p>
<div class="gmail_quote">On Apr 21, 2016 20:57, "Farhad Sunavala" <<a href="mailto:fsbiz@yahoo.com" target="_blank">fsbiz@yahoo.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div style="color:rgb(0,0,0);font-family:HelveticaNeue,'Helvetica Neue',Helvetica,Arial,'Lucida Grande',sans-serif;font-size:16px;background-color:rgb(255,255,255)"><div><span>Thanks Kevin and Kaustubh. Yes, just 1 node (controller, network and compute).</span></div><div><span><br></span></div><div><div style="background-image:initial;background-repeat:initial"><span>SRC C DST<u></u><u></u></span></div></div><div><div style="background-image:initial;background-repeat:initial"><span>| | |<u></u><u></u></span></div></div><div><div style="background-image:initial;background-repeat:initial"><span> OpenvSwitch<u></u><u></u></span></div></div><div><div style="background-image:initial;background-repeat:initial"><u></u> <u></u></div></div><div><div style="background-image:initial;background-repeat:initial"><span>SRC VM = 192.168.0.3<u></u><u></u></span></div></div><div><div style="background-image:initial;background-repeat:initial"><span>DST VM = 192.168.0.4<u></u><u></u></span></div></div><div><div style="background-image:initial;background-repeat:initial">Container C = 192.168.0.5<br></div></div><div><span><br></span></div><div><span><br></span></div><div>root@fs-10-145-105-64:~# neutron port-list</div><div>+--------------------------------------+------+-------------------+-------------------------------------------------------+</div><div>| id | name | mac_address | fixed_ips |</div><div>+--------------------------------------+------+-------------------+-------------------------------------------------------+</div><div>| 371b8923-efb7-499f-b344-a9f38d538a73 | | fa:16:3e:1e:bd:2c | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", |</div><div>| | | | "ip_address": "192.168.0.3"} |</div><div>| 86883e00-9d97-4bed-b05e-c942227fb268 | | fa:16:3e:2b:36:22 | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", |</div><div>| | | | "ip_address": "192.168.0.2"} |</div><div>| a1213dae-ab39-41c0-b2cd-99b20714aa8b | | fa:16:3e:c7:23:be | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", |</div><div>| | | | "ip_address": "192.168.0.5"} |</div><div>| b2339046-da71-43e9-a6f8-cbadf24be654 | | fa:16:3e:06:c5:23 | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", |</div><div>| | | | "ip_address": "192.168.0.1"} |</div><div>| b2b2c53a-c92e-47b6-88bc-db3d3e135b79 | | fa:16:3e:89:0d:fa | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", |</div><div>| | | | "ip_address": "192.168.0.4"} |</div><div></div><div>+--------------------------------------+------+-------------------+-------------------------------------------------------+</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">Just limiting output to SRC (192.168.0.3) and Container C (192.168.0.5)</div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# neutron port-show 371b8923-efb7-499f-b344-a9f38d538a73</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr">| Field | Value |</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr">| admin_state_up | True |</div><div dir="ltr">| allowed_address_pairs | |</div><div dir="ltr">| binding:vnic_type | normal |</div><div dir="ltr">| created_at | 2016-04-21T20:39:37 |</div><div dir="ltr">| description | |</div><div dir="ltr">| device_id | b7995a2e-f5c7-4398-a801-8d71c9c05a54 |</div><div dir="ltr">| device_owner | compute:None |</div><div dir="ltr">| dns_name | |</div><div dir="ltr">| extra_dhcp_opts | |</div><div dir="ltr">| fixed_ips | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", "ip_address": "192.168.0.3"} |</div><div dir="ltr">| id | 371b8923-efb7-499f-b344-a9f38d538a73 |</div><div dir="ltr">| mac_address | fa:16:3e:1e:bd:2c |</div><div dir="ltr">| name | |</div><div dir="ltr">| network_id | b2be2b73-6e17-427e-9ec3-5e6d1a1ef98d |</div><div dir="ltr">| port_security_enabled | False |</div><div dir="ltr">| security_groups | |</div><div dir="ltr">| status | ACTIVE |</div><div dir="ltr">| tenant_id | ba13e4a3e7724db8889852b398988587 |</div><div dir="ltr">| updated_at | 2016-04-22T03:38:48 |</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# neutron port-show a1213dae-ab39-41c0-b2cd-99b20714aa8b</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr">| Field | Value |</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr">| admin_state_up | True |</div><div dir="ltr">| allowed_address_pairs | |</div><div dir="ltr">| binding:vnic_type | normal |</div><div dir="ltr">| created_at | 2016-04-21T20:41:50 |</div><div dir="ltr">| description | |</div><div dir="ltr">| device_id | |</div><div dir="ltr">| device_owner | |</div><div dir="ltr">| dns_name | |</div><div dir="ltr">| extra_dhcp_opts | |</div><div dir="ltr">| fixed_ips | {"subnet_id": "762f6059-a06f-4b3f-a91d-d807bcf9f349", "ip_address": "192.168.0.5"} |</div><div dir="ltr">| id | a1213dae-ab39-41c0-b2cd-99b20714aa8b |</div><div dir="ltr">| mac_address | fa:16:3e:c7:23:be |</div><div dir="ltr">| name | |</div><div dir="ltr">| network_id | b2be2b73-6e17-427e-9ec3-5e6d1a1ef98d |</div><div dir="ltr">| port_security_enabled | False |</div><div dir="ltr">| security_groups | |</div><div dir="ltr">| status | DOWN |</div><div dir="ltr">| tenant_id | ba13e4a3e7724db8889852b398988587 |</div><div dir="ltr">| updated_at | 2016-04-21T20:41:50 |</div><div dir="ltr">+-----------------------+------------------------------------------------------------------------------------+</div><div dir="ltr"><br></div><div dir="ltr">The status=DOWN makes sense since the container is not under nova-control. </div></div><div dir="ltr"><br></div></div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# docker ps</div><div dir="ltr">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</div><div dir="ltr">c8d99952695d ubuntu "/bin/bash" 7 hours ago Up 7 hours backstabbing_spence</div><div dir="ltr">root@fs-10-145-105-64:~# docker inspect -f '{{.State.Pid}}' c8d99952695d</div><div dir="ltr">105529</div><div dir="ltr"><br></div></div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# ip netns</div><div dir="ltr">105529</div><div dir="ltr">qrouter-872d95f1-5ef7-4811-8232-8ced2322863b</div><div dir="ltr">qdhcp-b2be2b73-6e17-427e-9ec3-5e6d1a1ef98d</div><div dir="ltr">83376</div><div dir="ltr">root@fs-10-145-105-64:~# ip netns exec 105529 ifconfig</div><div dir="ltr">eth0 Link encap:Ethernet HWaddr fa:16:3e:c7:23:be</div><div dir="ltr"> inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0</div><div dir="ltr"> inet6 addr: fe80::f816:3eff:fec7:23be/64 Scope:Link</div><div dir="ltr"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</div><div dir="ltr"> RX packets:23908 errors:0 dropped:0 overruns:0 frame:0</div><div dir="ltr"> TX packets:24514 errors:0 dropped:0 overruns:0 carrier:0</div><div dir="ltr"> collisions:0 txqueuelen:1000</div><div dir="ltr"> RX bytes:2303668 (2.3 MB) TX bytes:2329412 (2.3 MB)</div><div dir="ltr"><br></div></div><div dir="ltr"><br></div><div dir="ltr"><span>As you can see the MAC address of container matches what neutron has given along with</span></div><div dir="ltr"><span>the IP address of 192.168.0.5</span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr">root@fs-10-145-105-64:~# brctl show</div><div dir="ltr">bridge name bridge id STP enabled interfaces</div><div dir="ltr"></div><div dir="ltr">root@fs-10-145-1058.-</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">root@fs-10-145-105-64:~# iptables -L</div><div dir="ltr">Chain INPUT (policy ACCEPT)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr">nova-api-INPUT all -- anywhere anywhere</div><div dir="ltr">ACCEPT udp -- anywhere anywhere udp dpt:domain</div><div dir="ltr">ACCEPT tcp -- anywhere anywhere tcp dpt:domain</div><div dir="ltr">ACCEPT udp -- anywhere anywhere udp dpt:bootps</div><div dir="ltr">ACCEPT tcp -- anywhere anywhere tcp dpt:bootps</div><div dir="ltr"><br></div><div dir="ltr">Chain FORWARD (policy ACCEPT)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr">nova-filter-top all -- anywhere anywhere</div><div dir="ltr">nova-api-FORWARD all -- anywhere anywhere</div><div dir="ltr">ACCEPT all -- anywhere <a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> ctstate RELATED,ESTABLISHED</div><div dir="ltr">ACCEPT all -- <a href="http://192.168.122.0/24" target="_blank">192.168.122.0/24</a> anywhere</div><div dir="ltr">ACCEPT all -- anywhere anywhere</div><div dir="ltr">REJECT all -- anywhere anywhere reject-with icmp-port-unreachable</div><div dir="ltr">REJECT all -- anywhere anywhere reject-with icmp-port-unreachable</div><div dir="ltr"><br></div><div dir="ltr">Chain OUTPUT (policy ACCEPT)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr">nova-filter-top all -- anywhere anywhere</div><div dir="ltr">nova-api-OUTPUT all -- anywhere anywhere</div><div dir="ltr">ACCEPT udp -- anywhere anywhere udp dpt:bootpc</div><div dir="ltr"><br></div><div dir="ltr">Chain nova-api-FORWARD (1 references)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr"><br></div><div dir="ltr">Chain nova-api-INPUT (1 references)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr">ACCEPT tcp -- anywhere fs-10.145.105.64 tcp dpt:8775</div><div dir="ltr"><br></div><div dir="ltr">Chain nova-api-OUTPUT (1 references)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr"><br></div><div dir="ltr">Chain nova-api-local (1 references)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr"><br></div><div dir="ltr">Chain nova-filter-top (2 references)</div><div dir="ltr">target prot opt source destination</div><div dir="ltr">nova-api-local all -- anywhere anywhere</div><div dir="ltr"><br></div><div dir="ltr">======================================================================</div><div dir="ltr"><br></div><div dir="ltr">Before sending 3 pings: </div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">root@c8d99952695d:/# ifconfig eth0</div><div dir="ltr">eth0 Link encap:Ethernet HWaddr fa:16:3e:c7:23:be</div><div dir="ltr"> inet addr:192.168.0.5 Bcast:192.168.0.255 Mask:255.255.255.0</div><div dir="ltr"> inet6 addr: fe80::f816:3eff:fec7:23be/64 Scope:Link</div><div dir="ltr"> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1</div><div dir="ltr"> RX packets:23908 errors:0 dropped:0 overruns:0 frame:0</div><div dir="ltr"> TX packets:24514 errors:0 dropped:0 overruns:0 carrier:0</div><div dir="ltr"> collisions:0 txqueuelen:1000</div><div dir="ltr"> RX bytes:2303668 (2.3 MB) TX bytes:2329412 (2.3 MB)</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><span><br></span></div><div dir="ltr">root@fs-10-145-105-64:~# ovs-ofctl dump-flows br-int</div><div dir="ltr">NXST_FLOW reply (xid=0x4):</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26464.927s, table=0, n_packets=0, n_bytes=0, idle_age=26464, priority=20,mpls actions=resubmit(,10)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=25823.697s, table=0, n_packets=0, n_bytes=0, idle_age=25823, priority=10,icmp6,in_port=7,icmp_type=136 actions=resubmit(,24)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=25803.737s, table=0, n_packets=0, n_bytes=0, idle_age=25803, priority=10,icmp6,in_port=8,icmp_type=136 actions=resubmit(,24)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26465.387s, table=0, n_packets=734, n_bytes=39540, idle_age=668, priority=0 actions=NORMAL</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26464.921s, table=10, n_packets=0, n_bytes=0, idle_age=26464, priority=0 actions=drop</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26465.381s, table=23, n_packets=0, n_bytes=0, idle_age=26465, priority=0 actions=drop</div><div dir="ltr"></div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26465.375s, table=24, n_packets=0, n_bytes=0, idle_age=26465, priority=0 actions=drop</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@c8d99952695d:/# ping 192.168.0.3 -c3</div><div dir="ltr">PING 192.168.0.3 (192.168.0.3) 56(84) bytes of data.</div><div dir="ltr">From 192.168.0.5 icmp_seq=1 Destination Host Unreachable</div><div dir="ltr">From 192.168.0.5 icmp_seq=2 Destination Host Unreachable</div><div dir="ltr">From 192.168.0.5 icmp_seq=3 Destination Host Unreachable</div><div dir="ltr"><br></div><div dir="ltr">--- 192.168.0.3 ping statistics ---</div><div dir="ltr">3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2014ms</div><div dir="ltr"><br></div></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">(TCPDUn veth_h0 - container's tap interface on br-int - Note: SRC MAC address matches one given by neutron)</div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# tcpdump -i veth_h0 -nnXXe</div><div dir="ltr">tcpdump: WARNING: veth_h0: no IPv4 address assigned</div><div dir="ltr">tcpdump: verbose output suppressed, use -v or -vv for full protocol decode</div><div dir="ltr">listening on veth_h0, link-type EN10MB (Ethernet), capture size 65535 bytes</div><div dir="ltr">20:51:58.629048 fa:16:3e:c7:23:be > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.0.3 tell 192.168.0.5, length 28</div><div dir="ltr"> 0x0000: ffff ffff ffff fa16 3ec7 23be 0806 0001 ........>.#.....</div><div dir="ltr"> 0x0010: 0800 0604 0001 fa16 3ec7 23be c0a8 0005 ........>.#.....</div><div dir="ltr"> 0x0020: 0000 0000 0000 c0a8 0003 ..........</div><div dir="ltr">20:51:59.625739 fa:16:3e:c7:23:be > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.0.3 tell 192.168.0.5, length 28</div><div dir="ltr"> 0x0000: ffff ffff ffff fa16 3ec7 23be 0806 0001 ........>.#.....</div><div dir="ltr"> 0x0010: 0800 0604 0001 fa16 3ec7 23be c0a8 0005 ........>.#.....</div><div dir="ltr"> 0x0020: 0000 0000 0000 c0a8 0003 ..........</div><div dir="ltr">20:52:00.625741 fa:16:3e:c7:23:be > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.0.3 tell 192.168.0.5, length 28</div><div dir="ltr"> 0x0000: ffff ffff ffff fa16 3ec7 23be 0806 0001 ........>.#.....</div><div dir="ltr"> 0x0010: 0800 0604 0001 fa16 3ec7 23be c0a8 0005 ........>.#.....</div><div dir="ltr"> 0x0020: 0000 0000 0000 c0a8 0003 ..........</div><div dir="ltr"><br></div></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><div dir="ltr">root@fs-10-145-105-64:~# ovs-ofctl dump-flows br-int</div><div dir="ltr">NXST_FLOW reply (xid=0x4):</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26703.277s, table=0, n_packets=0, n_bytes=0, idle_age=26703, priority=20,mpls actions=resubmit(,10)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26062.047s, table=0, n_packets=0, n_bytes=0, idle_age=26062, priority=10,icmp6,in_port=7,icmp_type=136 actions=resubmit(,24)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26042.087s, table=0, n_packets=0, n_bytes=0, idle_age=26042, priority=10,icmp6,in_port=8,icmp_type=136 actions=resubmit(,24)</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26703.737s, table=0, n_packets=737, n_bytes=39666, idle_age=123, priority=0 actions=NORMAL</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26703.271s, table=10, n_packets=0, n_bytes=0, idle_age=26703, priority=0 actions=drop</div><div dir="ltr"> cookie=0xb67c96fbae779e37, duration=26703.731s, table=23, n_packets=0, n_bytes=0, idle_age=26703, priority=0 actions=drop</div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr"><br></div><div dir="ltr">The packet counter has incremented by 3 (737-734) on table=0, actions:NORMAL</div><div dir="ltr"><br></div><div dir="ltr">But, the packets never make it out of the tap interface to SRC (192.168.0.3).</div><div dir="ltr"><br></div><div dir="ltr">Totally stumped. It is reproducible easily.</div><div dir="ltr">Just get the latest devstack from master (or mitaka stable) latest openvswitch (2.5.90) or OVS 2.4.1.</div><div dir="ltr"><br></div><div dir="ltr">Thanks,</div><div dir="ltr">Farhad.</div></div><div dir="ltr"><br></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div dir="ltr"><span><br></span></div><div><span><br></span></div><div><span><br></span></div> <div><br><br></div><div style="display:block"> <div style="font-family:HelveticaNeue,'Helvetica Neue',Helvetica,Arial,'Lucida Grande',sans-serif;font-size:16px"> <div style="font-family:HelveticaNeue,'Helvetica Neue',Helvetica,Arial,'Lucida Grande',sans-serif;font-size:16px"> <div dir="ltr"><font size="2" face="Arial"> On Thursday, April 21, 2016 3:52 PM, Kevin Benton <kevin@benton.pub> wrote:<br></font></div> <br><br> <div><div><div><div dir="ltr">I don't see port details for the container port (<span style="color:rgb(0,0,0);font-family:Helvetica,sans-serif">50686370). That would be the one that needs port_security_enabled=False if your container is not using the MAC address defined in the Neutron Port for that container.</span></div><div><br clear="none"><div>On Thu, Apr 21, 2016 at 3:30 PM, Kaustubh Kelkar <span dir="ltr"><<a rel="nofollow" shape="rect" href="mailto:kaustubh.kelkar@casa-systems.com" target="_blank">kaustubh.kelkar@casa-systems.com</a>></span> wrote:<br clear="none"><blockquote style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div><div lang="EN-US">
<div>
<div><span style="font-size:11pt">Hi,<u></u><u></u></span></div>
<div><span style="font-size:11pt"><u></u> <u></u></span></div>
<div><span style="font-size:11pt">Can you post the output of:<u></u><u></u></span></div>
<div><span style="font-size:11pt">iptables –L<u></u><u></u></span></div>
<div><span style="font-size:11pt">brctl show<u></u><u></u></span></div>
<div><span style="font-size:11pt">ovs-ofctl show br-int<u></u><u></u></span></div>
<div><span style="font-size:11pt"><u></u> <u></u></span></div>
<div><span style="font-size:11pt">I presume all the instances are on the same physical node?<u></u><u></u></span></div>
<div><span style="font-size:11pt"><u></u> <u></u></span></div>
<div><span style="font-size:11pt"><u></u> <u></u></span></div>
<div><span style="font-size:11pt">-Kaustubh<u></u><u></u></span></div>
<div><span style="font-size:11pt"><u></u> <u></u></span></div>
<div>
<div style="border-style:solid none none;border-top-color:rgb(225,225,225);border-top-width:1pt;padding:3pt 0in 0in">
<div><b><span style="font-size:11pt">From:</span></b><span style="font-size:11pt"> Farhad Sunavala [mailto:<a rel="nofollow" shape="rect" href="mailto:fsbiz@yahoo.com" target="_blank">fsbiz@yahoo.com</a>]
<br clear="none">
<b>Sent:</b> Thursday, April 21, 2016 3:15 PM<br clear="none">
<b>To:</b> <a rel="nofollow" shape="rect" href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br clear="none">
<b>Subject:</b> [Openstack] Openstack security - port-security, security-groups with added container<u></u><u></u></span></div>
</div>
</div>
<div><u></u> <u></u></div>
<div>
<div>
<div style="background:white"><span>Hi folks,<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>My setup is as follows.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>SRC C DST<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> OpenvSwitch<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>SRC VM = 192.168.0.3<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>DST VM = 192.168.0.4<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Container C = 192.168.0.5<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Openstack version: latest from master<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>OpenvSwitch: 2.4.1<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Openstack brings up SRC (192.168.0.3) and DST VM (192.168.0.5)<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>I manually bringup container C using docker and connect it to br-int.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>I allocate a neutron port (192.168.0.5) and configure this IP address on the container C.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>SRC,DST VMs work fine.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>Now, my problem is that the container C is not able to pass traffic.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Troubleshooting reveals it is sending packet to br-int, it hits the NORMAL action <u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>flows on br-int but then br-int does not transmit them forward.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>I have disabled security-groups and port-security on all ports.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>It is definitely something to do with security (anti-spoofing) but I am not able to narrow it down.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>Any pointers will be appreciated.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Thanks,<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>Farhad.<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Details of troubleshooting:<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Setup:<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>SRC C DST<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> OpenvSwitch<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>SRC VM = 192.168.0.3<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>DST VM = 192.168.0.4<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>Container C = 192.168.0.5<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>root@fs-10-145-105-64:~# neutron port-list<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+--------------------------------------+------+-------------------+-------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| id | name | mac_address | fixed_ips |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+--------------------------------------+------+-------------------+-------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| 31d7fef4-2689-40d5-886c-b0419295eabb | | fa:16:3e:e7:f8:47 | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | | | "ip_address": "192.168.0.4"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| 3466abf1-aa9a-420a-bb9b-c4ad55561f19 | | fa:16:3e:84:62:af | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | | | "ip_address": "192.168.0.2"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| 50686370-de40-4665-a5a1-39b5b795122b | | fa:16:3e:0d:ae:bc | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | | | "ip_address": "192.168.0.5"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| c6043a1e-ac4d-4138-af14-b91ae8e4d57e | | fa:16:3e:36:95:77 | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | | | "ip_address": "192.168.0.1"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| e8a167b1-8457-4bd7-a18b-82712e0c46dd | | fa:16:3e:85:92:ee | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| | | | "ip_address": "192.168.0.3"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+--------------------------------------+------+-------------------+-------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>root@fs-10-145-105-64:~# neutron port-show e8a167b1-8457-4bd7-a18b-82712e0c46dd<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| Field | Value |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| admin_state_up | True |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| allowed_address_pairs | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:host_id | fs-10-145-105-64 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:profile | {} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": false} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_type | ovs |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vnic_type | normal |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| created_at | 2016-04-21T15:27:51 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| description | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_id | 1a65bcfb-4607-4529-a313-6927d86c9302 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_owner | compute:nova |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| dns_name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| extra_dhcp_opts | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| fixed_ips | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", "ip_address": "192.168.0.3"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| id | e8a167b1-8457-4bd7-a18b-82712e0c46dd |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| mac_address | fa:16:3e:85:92:ee |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| network_id | e4c38c74-c9c2-47a7-802d-446488b82e5f |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| port_security_enabled | True |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| security_groups | fcf62b2a-1dd4-432a-b47b-5d9b79813da7 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| status | ACTIVE |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| tenant_id | 62b9eff93cc74b65a2b52a0d41604e2c |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| updated_at | 2016-04-21T15:27:58 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>root@fs-10-145-105-64:~# neutron port-show 31d7fef4-2689-40d5-886c-b0419295eabb<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| Field | Value |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| admin_state_up | True |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| allowed_address_pairs | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:host_id | fs-10-145-105-64 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:profile | {} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": false} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_type | ovs |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vnic_type | normal |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| created_at | 2016-04-21T15:28:37 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| description | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_id | 9c8a5e3c-0867-4938-b26e-4834a24d0c5d |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_owner | compute:nova |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| dns_name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| extra_dhcp_opts | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| fixed_ips | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", "ip_address": "192.168.0.4"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| id | 31d7fef4-2689-40d5-886c-b0419295eabb |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| mac_address | fa:16:3e:e7:f8:47 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| network_id | e4c38c74-c9c2-47a7-802d-446488b82e5f |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| port_security_enabled | False |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| security_groups | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| status | ACTIVE |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| tenant_id | 62b9eff93cc74b65a2b52a0d41604e2c |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| updated_at | 2016-04-21T18:25:20 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>root@fs-10-145-105-64:~# neutron port-show 31d7fef4-2689-40d5-886c-b0419295eabb<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| Field | Value |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| admin_state_up | True |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| allowed_address_pairs | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:host_id | fs-10-145-105-64 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:profile | {} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_details | {"port_filter": true, "ovs_hybrid_plug": false} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vif_type | ovs |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| binding:vnic_type | normal |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| created_at | 2016-04-21T15:28:37 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| description | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_id | 9c8a5e3c-0867-4938-b26e-4834a24d0c5d |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| device_owner | compute:nova |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| dns_name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| extra_dhcp_opts | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| fixed_ips | {"subnet_id": "19c6dc03-b255-45e9-9bd7-366631b745b6", "ip_address": "192.168.0.4"} |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| id | 31d7fef4-2689-40d5-886c-b0419295eabb |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| mac_address | fa:16:3e:e7:f8:47 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| name | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| network_id | e4c38c74-c9c2-47a7-802d-446488b82e5f |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| port_security_enabled | False |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| security_groups | |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| status | ACTIVE |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| tenant_id | 62b9eff93cc74b65a2b52a0d41604e2c |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>| updated_at | 2016-04-21T18:25:20 |<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>+-----------------------+------------------------------------------------------------------------------------+<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span><u></u> <u></u></span></div>
</div>
<div>
<div style="background:white"><span>root@fs-10-145-105-64:~# ovs-vsctl show<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span>cd040c9d-427b-418b-943c-7ad5522e1d8f<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Bridge br-int<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> fail_mode: secure<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Port "tap3466abf1-aa"<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> tag: 2<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Interface "tap3466abf1-aa"<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> type: internal<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Port "tape8a167b1-84"<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> tag: 2<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Interface "tape8a167b1-84"<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Port br-int<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Interface br-int<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> type: internal<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> Port patch-tun<u></u><u></u></span></div>
</div>
<div>
<div style="background:white"><span> </span></div></div></div></div></div></div></blockquote></div></div></div></div></div></div></div></div></div></div>...</blockquote></div>
</div>