<div dir="ltr">Hi,<div><br></div><div>We have the following setup:</div><div><br></div><div> - OpenStack Icehouse (Ubuntu 14.04 LTS)</div><div> - Deployed via puppet-openstack module</div><div> - Neutron (OpenVSwitch version 2.0.2)</div><div> - L3 networking is all handled by physical devices (no Neutron L3 components in use)</div><div> - Neutron VLAN provider networks</div><div> - No tunnelling of any sort</div><div><br></div><div>In order to preserve network ports on a Cisco Nexus 9000 we opted to limit our compute hosts to a single bonded interface (eth0 and eth1, using LACP to form bond0 across two switches) and provide some isolation using VLANs - accepting the fact that it's risky to make use of a single NIC with 2 x 10G ports. That aside, the network configuration of a compute host:</div><div><br></div><div>########################</div><div><div>auto lo</div><div>iface lo inet loopback</div><div><br></div><div>auto eth0</div><div>iface eth0 inet manual</div><div> bond-master bond0</div><div><br></div><div>auto eth1</div><div>iface eth1 inet manual</div><div> bond-master bond0</div><div><br></div><div># Trunk interface to be used for VLAN provider networks and any other local interfaces</div><div>auto bond0</div><div>iface bond0 inet manual</div><div> bond-slaves eth0 eth1</div><div> bond-mode 4</div><div> bond-miimon 80</div><div> bond-downdelay 200</div><div> bond-updelay 200</div><div> bond-lacp-rate 1</div><div> up ifconfig $IFACE 0.0.0.0 up</div><div> up ip link set $IFACE promisc on</div><div> down ip link set $IFACE promisc off</div><div> down ifconfig $IFACE down</div><div><br></div><div># Management IP</div><div>auto bond0.10</div><div>iface bond0.10 inet static</div><div> address 10.0.10.100</div><div> netmask 255.255.255.0</div><div> gateway 10.0.10.1</div><div><br></div><div># Storage network</div><div>auto bond0.20</div><div>iface bond0.20 inet static</div><div> address 10.0.20.100</div><div> netmask 255.255.255.0</div></div><div><br></div><div>########################<br></div><div><br></div><div>Similarly, the ML2 configuration on the controller:</div><div><br></div><div><div>[ml2]</div><div>type_drivers = vlan</div><div>tenant_network_types = vlan</div><div>mechanism_drivers = openvswitch</div><div><br></div><div>[ml2_type_flat]</div><div><br></div><div>[ml2_type_vlan]</div><div>network_vlan_ranges = default:1:1001,default:1006:4094</div><div><br></div><div>[ml2_type_gre]</div><div><br></div><div>[ml2_type_vxlan]</div><div><br></div><div>[securitygroup]</div><div>enable_security_group = True</div><div>firewall_driver=neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver</div><div><br></div><div>[agent]</div><div>l2_population=False</div><div>polling_interval=2</div><div>arp_responder=False</div><div><br></div><div>[ovs]</div><div>enable_tunneling=False</div><div>integration_bridge=br-int</div><div>bridge_mappings=default:br-ex</div></div><div><br></div><div>########################<br></div><div><br></div><div>Finally, the OVS bridges:</div><div><br></div><div><div> Bridge br-int</div><div> fail_mode: secure</div><div> Port br-int</div><div> Interface br-int</div><div> type: internal</div><div> Port int-br-ex</div><div> Interface int-br-ex</div><div> Bridge br-ex</div><div> Port br-ex</div><div> Interface br-ex</div><div> type: internal</div><div> Port phy-br-ex</div><div> Interface phy-br-ex</div><div> Port "bond0"</div><div> Interface "bond0"</div><div> ovs_version: "2.0.2"</div><div><br></div><div>########################<br></div><div><br></div><div>In general, this works absolutely fine for us. Neutron networks are created with relevant segmentation IDs to isolate various ranges/subnets and instances. Security groups are enabled but rules have been added to permit all inbound ICMP, TCP, and UDP traffic. An additional layer of security is provided by host firewalls (iptables) on the VMs.</div><div><br></div><div>The problem we are now facing - for specific reasons, we wish to deploy a virtual network appliance that has an interface on the OpenStack management network, permitting hypervisor hosts to communicate with the said device on the relevant network range (<a href="http://10.0.10.0/24">10.0.10.0/24</a>) without any routing requirements. What we have noted in an attempt to create a Neutron network with segmentation ID 10:</div><div><br></div><div>- All segmentation IDs (VLAN IDs) used by VM instances, other than VLAN 10, continue to function as expected</div><div> - The virtual network appliance interface on VLAN 10 is unreachable by any device on the same range</div><div> - ARP traffic is showing as leaving the compute host, responded to, and noted in tcpdump output on all hosts with a management network interface, but nothing is noted as coming back over the OVS integration bridge</div><div><br></div><div>Doing a bit of OVS doc reading, someone pointed out that 'VLAN splintering' may help. When enabling the feature on bond0 the effect is that traffic starts to register on the virtual network appliance interface but the compute host management interface (bond0.10) goes down.</div><div><br></div><div>Having a trunk interface being used for Neutron VLAN provider networks and an 802.1q (vconfig) interface on the same compute host for management IP purposes appears to be a no-no. The reason for this could be completely obvious to the networking elite out there but we don't completely understand why this is and some clarification would be very much appreciated :)</div><div><br></div><div>A side note - seems that James Denton alluded to an incompatibility to this approach a while back on the OpenStack list (<a href="http://markmail.org/message/fcaklkctwmaeagbw">http://markmail.org/message/fcaklkctwmaeagbw</a>), at least with respect to the Linux bridge ML2 plugin, but no additional details were provided.</div><div><br></div><div>Happy to provide any additional information to get to the bottom of this.</div><div><br></div><div>Regards,</div><div><br></div><div><div class="gmail_signature">--<br>Adnan<br></div></div>
</div></div>