<div dir="ltr">I'm not sure how effective these rules are. I've been able to use OVS+nova-network+libvirt to create nested bridges without issues.</div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, May 2, 2013 at 12:53 AM, Édouard Thuleau <span dir="ltr"><<a href="mailto:thuleau@gmail.com" target="_blank">thuleau@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">And if you use libvirt virt driver, the hypervisor (libvirt+KVM in my case) adds anti MAC and IP spoofing rules on VNICs of VM:<div>
<br></div><div><div>$ virsh nwfilter-list</div><div>UUID Name </div>
<div>----------------------------------------------------------------</div><div>991dbd1a-373b-a005-57b2-5b1f4107f653 allow-arp </div><div>aefdae18-56e8-4e67-c8f1-cc826e2c519c allow-dhcp </div><div>f1e80828-1b4b-dcb3-4136-faaf5beab9e2 allow-dhcp-server </div>
<div>9e497c96-ec4a-4ad8-fcbb-d2917e3af70a allow-incoming-ipv4 </div><div>4d998884-5870-8956-5585-d34f77231e3e allow-ipv4 </div><div>7a4233d3-1fa8-6eb1-f2e8-3e55f773da0d clean-traffic </div><div>fae01ed3-2bd5-e6b4-a63e-daa336655c20 no-arp-ip-spoofing </div>
<div>bf61b4b4-f844-6c36-9bb7-6245b642b0cb no-arp-mac-spoofing </div><div>04cc5d54-08a0-31ca-4b5e-79a2e96d276b no-arp-spoofing </div><div>255d63a3-12a7-32b9-dffb-a2d61c8fcb39 no-ip-multicast </div><div>a0e9b6f3-e099-2b0b-7d4b-69e63587fa39 no-ip-spoofing </div>
<div>83145355-39d1-9dce-4012-3032c110cf82 no-mac-broadcast </div><div>653c47ed-48f0-25ea-bf18-1153a58d3773 no-mac-spoofing </div><div>cc460af0-ee60-7ca8-c09e-a074490711ac no-other-l2-traffic </div><div>5df592f3-dcff-e0f3-73ac-d2eb3baeda11 no-other-rarp-traffic</div>
<div>891e4787-e5c0-d59b-cbd6-41bc3c6b36fc nova-allow-dhcp-server</div><div>418f4ad6-d997-b483-15d9-c7c2c21b4eba nova-base </div><div>fdc1ee23-05a1-0303-6d24-8a300bd57f21 nova-instance-instance-00000004-fa163e3ec9b3</div>
<div>e8cd7fa5-2de9-cfe1-f24f-8a449043c6f3 nova-instance-instance-00000005-fa163ed87bff</div><div>16e11cd9-6e17-3c91-6776-e3bffc70e94b nova-instance-instance-00000006-fa163ecd666a</div><div>c5ba020f-6b6f-d511-8ee1-2e2b49497431 nova-instance-instance-00000007-fa163e1d4e38</div>
<div>2d085283-a4bf-79f8-80f1-20498b8cc475 nova-instance-instance-00000018-fa163ee1842c</div><div>7d4bb9f1-597e-2a36-e340-45ec710b4481 nova-instance-instance-00000088-fa163ef641ad</div><div>79ef4d25-ff42-fd63-f34b-fc1079c391b3 nova-nodhcp </div>
<div>c5ac3035-ac46-3870-ff5c-296b5f4221d3 nova-vpn </div><div>b615cae6-4ca8-882f-42e1-9de541e4844b qemu-announce-self </div><div>b34d17b0-30d6-75c4-19d0-e636d1f99160 qemu-announce-self-rarp</div></div><div>
<br></div><div>The virt driver is control by Nova, so is that Nova should be responsible for network security? Perhaps it could be disabled? But if we disable it, is that Quantum takes good care?</div><span class="HOEnZb"><font color="#888888"><div>
<br>
</div><div>Édouard.</div></font></span><div><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, May 1, 2013 at 7:14 AM, Joe Topjian <span dir="ltr"><<a href="mailto:joe.topjian@cybera.ca" target="_blank">joe.topjian@cybera.ca</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Thank you both for the information. <div><br></div><div>I see that the compute node has some iptables rules for the instance -- one in particular that filters the instance's mac address -- but deleting this rule doesn't resolve the issue. So my guess is that it's the flow table that Salvatore mentioned which is ultimately controlling the filtering. </div>
<div><br></div><div>At the moment, I don't know enough about open vswitch to make custom changes to the flow table. For now, setting the bridge's mac address as the same mac of the virtual interface is a good work around.</div>
<div><br></div><div>Thanks again,</div><div>Joe</div></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 30, 2013 at 5:57 PM, Salvatore Orlando <span dir="ltr"><<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I was not aware that security groups for OVS already enforced anti spoofing rules.<div>That's good to know.</div>
<span><font color="#888888"><div><br></div><div>Salvatore</div></font></span></div><div><div><div class="gmail_extra"><br><br>
<div class="gmail_quote">On 1 May 2013 00:55, Aaron Rosen <span dir="ltr"><<a href="mailto:arosen@nicira.com" target="_blank">arosen@nicira.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Also, the security group stuff locks down the port to be the mac+ip of the quantum port mac+ip. If you create a new bridge and add ethX to it you'll also have to set the mac on your bridge to be the same as ethX (which is the mac that quantum handed out). <span><font color="#888888"><div>
<br>Aaron</div></font></span></div><div><div><div class="gmail_extra"><br><br><div class="gmail_quote">On Tue, Apr 30, 2013 at 4:25 PM, Salvatore Orlando <span dir="ltr"><<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Joe,<div><br></div><div>are you using the OVS plugin with GRE overlays?</div><div>In that case your problem might be the fact that the plugin pushes a OVS flow entry which applies the 'local' vlan tag only to packet directed to the VM's mac [1]</div>
<div><br></div><div>To me, this does not look like a bug; it's probably intended behaviour, as it kind of implements mac spoofing prevention. In the future we might also expect stricter anti-spoof checking; on the other side a change for administratively enabling promiscuos mode might be welcome - this should allow you to do nested OVS.</div>
<div><br></div><div>Salvatore</div><div><br></div><div>[1] <a href="https://github.com/openstack/quantum/blob/master/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py#L448" target="_blank">https://github.com/openstack/quantum/blob/master/quantum/plugins/openvswitch/agent/ovs_quantum_agent.py#L448</a></div>
<div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div>On 30 April 2013 22:08, Joe Topjian <span dir="ltr"><<a href="mailto:joe.topjian@cybera.ca" target="_blank">joe.topjian@cybera.ca</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hello,<div><br></div><div>I have OpenStack (Grizzly) up and running with Quantum. I'm using the Open vSwitch plugin, per-tenant routing, and network namespaces. As far as I'm aware, this is all set up correctly as instances that I create are able to retrieve an IP address via DHCP, reach the metadata server, and reach the outside internet.</div>
<div><br></div><div>The issue that I'm running into is that when I install Open vSwitch on the instance itself, I'm unable to create working bridges. For example:</div><div><br></div><div>ovs-vsctl add-br br-eth0</div>
<div>ovs-vsctl add-port br-eth0 eth0</div><div>(swap IPs from eth0 to br-eth0, kill dhcp, etc etc)</div><div><br></div><div>Traffic isn't flowing properly, though.</div><div><br></div><div>
If I run a continuous ping and run tcpdump on both the instance and the tap interface on the controller, I see arp requests going out of the instance, being received on the tap interface, the tap interface sending a reply, but the reply never reaching the instance.</div>
<div><br></div><div>However, I have found that if I create a bridge with the same MAC as the interface that I'm adding to the bridge, traffic flows correctly:</div><div><br></div><div>ovs-vsctl set bridge br-eth0 other-config:hwaddr=aa:bb:cc:00:11:22</div>
<div><br></div><div>My best guess is that there's something (L2) blocking the flow of traffic, but I'm not exactly sure where to start looking. I think it's safe to assume that Open vSwitch on the OpenStack servers is doing the blocking but I think it's Quantum that's implementing the blocking since if I use Open vSwitch with nova-network, this problem doesn't happen.</div>
<div><br></div><div>Does anyone have any pointers? Or even a fix?<br></div><div><br></div><div>Thanks,</div><div>Joe</div><span><font color="#888888"><div><br></div><div>-- <br>Joe Topjian<div>Systems Administrator</div>
<div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div><div><br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</div></font></span></div>
<br></div></div>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>
<br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div>
<div><br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</div>
</div></div><br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div></div></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br>Joe Topjian<div>Systems Administrator</div><div>Cybera Inc.</div><div><br></div><div><a href="http://www.cybera.ca" target="_blank">www.cybera.ca</a></div><div>
<br></div><div><font color="#666666"><span>Cybera</span><span> is a not-for-profit organization that works to spur and support innovation, for the economic benefit of Alberta, through the use of cyberinfrastructure.</span></font></div>
</div>