<div dir="ltr">Yeah looking at the output of iptables clearly shows that the packets are falling all the way through iptables and hitting the neutron-openvswi-sg-fallback<div><br></div><div>The relevant sections from iptables-save -c are:</div><div><br></div><div>[23:1932] -A neutron-openvswi-FORWARD -m physdev --physdev-out tapa6ad9dce-b0 --physdev-is-bridged -j neutron-openvswi-sg-chain<br></div><div><div>[23:1932] -A neutron-openvswi-sg-chain -m physdev --physdev-out tapa6ad9dce-b0 --physdev-is-bridged -j neutron-openvswi-ia6ad9dce-b</div><div>[0:0] -A neutron-openvswi-sg-chain -m physdev --physdev-in tapa6ad9dce-b0 --physdev-is-bridged -j neutron-openvswi-oa6ad9dce-b</div></div><div><div>[0:0] -A neutron-openvswi-ia6ad9dce-b -m state --state INVALID -j DROP</div><div>[0:0] -A neutron-openvswi-ia6ad9dce-b -m state --state RELATED,ESTABLISHED -j RETURN</div><div>[0:0] -A neutron-openvswi-ia6ad9dce-b -s <a href="http://162.220.48.123/32">162.220.48.123/32</a> -p udp -m udp --sport 67 --dport 68 -j RETURN</div><div>[0:0] -A neutron-openvswi-ia6ad9dce-b -m set --match-set IPv4c8238a74-1311-4700-9 src -j RETURN</div><div>[23:1932] -A neutron-openvswi-ia6ad9dce-b -j neutron-openvswi-sg-fallback</div></div><div>[23:1932] -A neutron-openvswi-sg-fallback -j DROP<br></div><div><br></div><div>So it looks like the packets are coming in, and being sent to the neutron-openvswi-ia6ad9dce-b table, and then missing all the rules there and being pushed to the neutron-openvswi-sg-fallback table where they are being dropped. This also shows that RELATED and ESTABLISHED packet states are allowed, which also jives with what I was seeing in production.</div><div><br></div><div>So this would seem to indicate a problem with my Security Groups.</div><div><br></div><div>Upon checking the security groups in Nova (nova secgroup-list-rules default) I am seeing the following:</div><div><br></div><div><div>+-------------+-----------+---------+----------+--------------+</div><div>| IP Protocol | From Port | To Port | IP Range | Source Group |</div><div>+-------------+-----------+---------+----------+--------------+</div><div>| | | | | default |</div><div>| | | | | default |</div><div>+-------------+-----------+---------+----------+--------------+</div></div><div><br></div><div>Despite the fact that Horizon still shows the correct routes.</div><div><br></div><div>So this looks to be a settings issue in the Security Group configuration. Sadly I don't have a clue why the information I am seeing in Horizon is out of sync with what I am seeing on the CLI.</div><div><br></div><div>At this point I am glad to know where the problem lies, and how I can go about correcting it so that it isn't a problem going forwards.</div><div><br></div><div>Thanks for assisting in shining a light on what the problem is.</div><div><br></div><div>Warm regards,</div><div><br></div><div>Tom Walsh</div><div>ExpressHosting</div><div><a href="http://expresshosting.net/">http://expresshosting.net/</a></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, May 19, 2015 at 6:18 PM, Brian Haley <span dir="ltr"><<a href="mailto:brian.haley@hp.com" target="_blank">brian.haley@hp.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 5/19/15 8:40 AM, Tom Walsh wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
This is a new relatively new issue that has started occurring on our<br>
OpenStack setup since we upgraded Juno (RDO based) to 2.2-1 (we had been<br>
running Juno 2.0 before without issue). Our setup is:<br>
</blockquote>
<br></span>
Since it started happening on an RDO upgrade my first suggestion is to contact Red Hat.<br>
<br>
I looked at your iptables output and didn't see anything out of the ordinary, although using 'iptables-save -c' might help show which rule is being hit to drop things (assuming it's the DROP rule in the fallback chain).<br>
<br>
Don't know of any upstream changes that would have broken this.<span class="HOEnZb"><font color="#888888"><br>
<br>
-Brian<br>
</font></span></blockquote></div><br></div>