Open Stack

So..

On 07/12/2017 03:12, Jean-Philippe Evrard wrote:

> For the mtu, it would be impactful to do it on a live environment. I
> expect that if you change the container configuration, it would
> restart.

It’s a busy lab environment, but given that it’s fully HA (2 
controllers), I didn’t anticipate a significant problem with changing 
container configuration one-at-a-time.

However, the change has had an unexpected side effect - one of the 
controllers (I haven’t rebooted the other one yet) seems to have lost 
the ability to bring up lxcbr0, and so while it can start all its 
containers, none of them have any management connectivity on eth0, which 
of course breaks all sorts of things.

I.e.

|root at nbs-dh-10:~# systemctl status networking.service ● 
networking.service - Raise network interfaces Loaded: loaded 
(/lib/systemd/system/networking.service; enabled; vendor preset: 
enabled) Drop-In: /run/systemd/generator/networking.service.d 
└─50-insserv.conf-$network.conf Active: failed (Result: exit-code) since 
Thu 2017-12-07 06:37:00 NZDT; 14min ago Docs: man:interfaces(5) Process: 
2717 ExecStart=/sbin/ifup -a --read-environment (code=exited, 
status=1/FAILURE) Process: 2656 ExecStartPre=/bin/sh -c [ 
"$CONFIGURE_INTERFACES" != "no" ] && [ -n "$(ifquery --read-environment 
--list --exclude=lo)" ] && udevadm settle (code=e Main PID: 2717 
(code=exited, status=1/FAILURE) Dec 07 06:36:58 nbs-dh-10 systemd[1]: 
Starting Raise network interfaces... Dec 07 06:36:58 nbs-dh-10 
ifup[2717]: RTNETLINK answers: Invalid argument Dec 07 06:36:58 
nbs-dh-10 ifup[2717]: /sbin/ifup: waiting for lock on 
/run/network/ifstate.enp4s0 Dec 07 06:36:58 nbs-dh-10 ifup[2717]: 
/sbin/ifup: waiting for lock on /run/network/ifstate.br-mgmt Dec 07 
06:37:00 nbs-dh-10 ifup[2717]: /sbin/ifup: waiting for lock on 
/run/network/ifstate.br-vlan Dec 07 06:37:00 nbs-dh-10 ifup[2717]: 
Failed to bring up lxcbr0. Dec 07 06:37:00 nbs-dh-10 systemd[1]: 
networking.service: Main process exited, code=exited, status=1/FAILURE 
Dec 07 06:37:00 nbs-dh-10 systemd[1]: Failed to start Raise network 
interfaces. Dec 07 06:37:00 nbs-dh-10 systemd[1]: networking.service: 
Unit entered failed state. Dec 07 06:37:00 nbs-dh-10 systemd[1]: 
networking.service: Failed with result 'exit-code'. root at nbs-dh-10:~# |

I’ve manually reversed the “lxc.network.mtu = 1550” entry in 
/etc/lxc/lxc-openstack.conf, but this doesn’t seem to have made a 
difference.

What’s also odd is that lxcbr0 appears to be perfectly normal:

|root at nbs-dh-10:~# brctl show lxcbr0 bridge name bridge id STP enabled 
interfaces lxcbr0 8000.fe0a7fa28303 no 04063403_eth0 075266dc_eth0 
160c9b30_eth0 38ac19ae_eth0 4f57300f_eth0 59b2b5a5_eth0 5b7bbeb4_eth0 
64a1fcdd_eth0 6c99f5fe_eth0 6f93ebb2_eth0 70ce61e5_eth0 745ba80d_eth0 
85df2fa5_eth0 99e6adf8_eth0 cbdfa2f3_eth0 e15dc279_eth0 ea67ce7e_eth0 
ed5c7af9_eth0 root at nbs-dh-10:~# |

… But, no matter the value of lxc.network.mtu, it doesn’t change from 
1500 (I suppose this could actually have reduced itself based on the 
lower MTUs of the member interfaces though):

|root at nbs-dh-10:~# ifconfig lxcbr0 lxcbr0 Link encap:Ethernet HWaddr 
fe:0c:5d:1c:36:da inet addr:10.0.3.1 Bcast:10.0.3.255 Mask:255.255.255.0 
inet6 addr: fe80::f4b0:bff:fec3:63b0/64 Scope:Link UP BROADCAST RUNNING 
MULTICAST MTU:1500 Metric:1 RX packets:499 errors:0 dropped:0 overruns:0 
frame:0 TX packets:10 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:1000 RX bytes:128882 (128.8 KB) TX bytes:828 
(828.0 B) root at nbs-dh-10:~# |

Any debugging suggestions?

Thanks,
D

​
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20171207/0d9634e8/attachment.html>