[Openstack-operators] How to deal with MTU issue on routers servicing vxlan tenant networks? (openstack-ansible with LXC containers)

David Young davidy at funkypenguin.co.nz
Wed Dec 6 08:51:12 UTC 2017


Hello,

Thanks for the reply, responses inline below:

> Hello,
>
> I haven't touched this for a while, but could you give us your user_*
> variable overrides?

OK, here we go. Let me know if there’s a preferred way to send large 
data blocks - I considered a gist or a pastebin, but figured that having 
the content archived with the mailing list message would be the best result.

I think the overrides is what you’re asking for? The only MTU-related 
override I have is “container/mtu” for the vxlan network below. I expect 
it doesn’t actually _do/ anything though, because I can’t find the 
string “container_mtu” within any of the related ansible roles (see grep 
for container_mtu vs container_bridge below for illustration). I found 
https://bugs.launchpad.net/openstack-ansible/+bug/1678165 which looked 
related

|root at nbs-dh-09:~# grep container_mtu /etc/ansible/ -ri root at nbs-dh-09:~# 
grep container_bridge /etc/ansible/ -ri 
/etc/ansible/roles/plugins/library/provider_networks:# container_bridge: 
"br-mgmt" /etc/ansible/roles/plugins/library/provider_networks:# 
container_bridge: "br-vxlan" 
/etc/ansible/roles/plugins/library/provider_networks:# container_bridge: 
"br-vlan" /etc/ansible/roles/plugins/library/provider_networks:# 
container_bridge: "br-vlan" 
/etc/ansible/roles/plugins/library/provider_networks:# container_bridge: 
"br-storage" /etc/ansible/roles/plugins/library/provider_networks: 
bind_device = net['network']['container_bridge'] 
/etc/ansible/roles/os_neutron/doc/source/configure-network-services.rst: 
container_bridge: "br-vlan" root at nbs-dh-09:~# |

|global_overrides: internal_lb_vip_address: 10.76.76.11 # # The below 
domain name must resolve to an IP address # in the CIDR specified in 
haproxy_keepalived_external_vip_cidr. # If using different protocols 
(https/http) for the public/internal # endpoints the two addresses must 
be different. # external_lb_vip_address: openstack.dev.safenz.net 
tunnel_bridge: "br-vxlan" management_bridge: "br-mgmt" 
provider_networks: - network: container_bridge: "br-mgmt" 
container_type: "veth" container_interface: "eth1" ip_from_q: 
"container" type: "raw" group_binds: - all_containers - hosts 
is_container_address: true is_ssh_address: true - network: 
container_bridge: "br-vxlan" container_type: "veth" container_interface: 
"eth10" container_mtu: "9000" ip_from_q: "tunnel" type: "vxlan" range: 
"1:1000" net_name: "vxlan" group_binds: - neutron_linuxbridge_agent - 
network: container_bridge: "br-vlan" container_type: "veth" 
container_interface: "eth12" host_bind_override: "eth12" type: "flat" 
net_name: "flat" group_binds: - neutron_linuxbridge_agent - network: 
container_bridge: "br-vlan" container_type: "veth" container_interface: 
"eth11" type: "vlan" range: "1:4094" net_name: "vlan" group_binds: - 
neutron_linuxbridge_agent - network: container_bridge: "br-storage" 
container_type: "veth" container_interface: "eth2" ip_from_q: "storage" 
type: "raw" group_binds: - glance_api - cinder_api - cinder_volume - 
nova_compute - swift_proxy |

> Here are a few things I watch for mtu related discussions:
> 1) ``lxc_net_mtu``: It is used in lxc_hosts to define the lxc bridge.

Aha. I didn’t know about this, it sounds like what I need. I’ll add this 
and report back.

> 2) Your compute nodes and your controller nodes need to have
> consistent mtus on their bridges.

They are both configured for an MTU of 9000, but the controller nodes 
bridges’ drop their MTU to 1500 when the veth interface paired with the 
neutron-agent LXC container is joined to the bridge (bridges downgrade 
their MTU to the MTU of the lowest participating interface)

> 3) Neutron needs a configuration override.

I’ve set this in neutron.conf on all neutron LXC containers, and on the 
compute nodes too:
|global_physnet_mtu = 1550|

And likewise in /etc/neutron/plugins/ml2/ml2_conf.ini:

|# Set a global MTU of 1550 (to allow VXLAN at 1500) path_mtu = 1550 # 
Drop VLAN and FLAT providers back to 1500, to align with outside FWs 
physical_network_mtus = vlan:1500,flat:1500 |

> 4) the lxc containers need to be properly defined: each network should
> have a mtu defined, or alternatively, you can define a default mtu for
> all the networks defined in openstack_user_config with
> ``lxc_container_default_mtu``. (This one is the one that spawns up the
> veth pair to the lxc container)

I didn’t know about this one either, it didn’t exist in any of the 
default ansible-provided sample configs, but now that I’ve grepped in 
the ansible roles for “mtu”, it’s obvious. I’ll try this too.

|root at nbs-dh-09:~# grep -ri lxc_container_default_mtu 
/etc/openstack_deploy/* root at nbs-dh-09:~# grep -ri 
lxc_container_default_mtu /etc/ansible/ 
/etc/ansible/roles/lxc_container_create/defaults/main.yml:lxc_container_default_mtu: 
"1500" 
/etc/ansible/roles/lxc_container_create/templates/container-interface.ini.j2:lxc.network.mtu 
= {{ item.value.mtu|default(lxc_container_default_mtu) }} 
/etc/ansible/roles/lxc_container_create/templates/debian-interface.cfg.j2: 
mtu {{ item.value.mtu|default(lxc_container_default_mtu) }} 
/etc/ansible/roles/lxc_container_create/templates/rhel-interface.j2:MTU={{ 
item.value.mtu|default(lxc_container_default_mtu) }} root at nbs-dh-09:~# |

> 5) The container interfaces need to have this proper mtu. This is
> taking the same configuration as 4) above, so it should work out of
> the box.

Agreed, that seems to be the case currently with 1500, I’d expect it to 
be true with the updated value

> 6) If your instance is reaching its router with no mtu issue, you may
> still have issues for the Northbound trafic. Check how you configured
> this northbound and if the interfaces have proper mtu. If there are
> veth pairs to create pseudo links, check their mtus too.
>
> I think it's a good start for the conversation...

Thank you, this is very helpful. I’ll give it a try and respond.

Re #1 and #4, do I need to destroy / recreate my existing LXC 
containers, or will rerunning the playbooks be enough to update the MTUs?

Many thanks,
David

​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20171206/e2b24f1a/attachment.html>


More information about the OpenStack-operators mailing list