Openstack instances cannot access to Internet [linuxbridge]
Jaime
jsaezdeb at ucm.es
Thu Feb 11 11:45:40 UTC 2021
I am having serious issues in the deployment of the Openstack scenario
related to the Linux Bridge.
This is the scenario:
- Controller machine:
- Management Interface `enp2s0`: 138.100.10.25.
- Compute machine:
- Management Interface `enp2s0`: 138.100.10.26.
- Provider Interface `enp0s20f0u4`: 138.100.10.27.
Openstack Train scenario has been successfully deployed in Centos 8,
choosing networking option 2 (self-service network).
To verify the functionality, an image has been uploaded, created an
Openstack flavor and security group, and launched a couple of cirrOS
instances for connection testing.
We have created a provider network following [this
tutorial](https://docs.openstack.org/newton/install-guide-rdo/launch-instance-networks-provider.html)
and a selfservice network following [this
one](https://docs.openstack.org/newton/install-guide-rdo/launch-instance-networks-selfservice.html).
The network scenario is the next one:
As can be seen in the network topology, an external network
138.100.10.0/21 (provider) and an internal network 192.168.1.1
(selfservice) have been created, connected through a router by the
interfaces 138.100.10.198 and 192.168.1.1, both active.
Our problem is that our Linux bridge is not working as expected: the
Openstack cirrOS instances has no internet access.
This is the controller `ip a` and `brctl show` command output:
```
[upm at modena ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
master brq84f65ccb-c9 state UP group default qlen 1000
link/ether 24:4b:fe:7c:78:b8 brd ff:ff:ff:ff:ff:ff
3: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
state DOWN group default qlen 1000
link/ether 52:54:00:15:5b:02 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
4: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master
virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:15:5b:02 brd ff:ff:ff:ff:ff:ff
17: tapa467f377-b1 at if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue master brq84f65ccb-c9 state UP group default qlen 1000
link/ether d6:90:e8:fe:90:23 brd ff:ff:ff:ff:ff:ff link-netns
qdhcp-84f65ccb-c945-437b-9013-9c71422bb10e
18: brq84f65ccb-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP group default qlen 1000
link/ether 24:4b:fe:7c:78:b8 brd ff:ff:ff:ff:ff:ff
inet 138.100.10.25/21 brd 138.100.15.255 scope global brq84f65ccb-c9
valid_lft forever preferred_lft forever
inet6 fe80::6c31:4cff:fe2d:7820/64 scope link
valid_lft forever preferred_lft forever
19: tap7a390547-5b at if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
noqueue master brqd811bcfa-94 state UP group default qlen 1000
link/ether 26:6d:a4:fc:73:51 brd ff:ff:ff:ff:ff:ff link-netns
qdhcp-d811bcfa-945a-4633-9266-60ccafa28d86
20: vxlan-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
master brqd811bcfa-94 state UNKNOWN group default qlen 1000
link/ether fa:cb:4e:e2:83:46 brd ff:ff:ff:ff:ff:ff
21: brqd811bcfa-94: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
noqueue state UP group default qlen 1000
link/ether 12:a8:05:bf:98:98 brd ff:ff:ff:ff:ff:ff
inet6 fe80::88c2:ecff:fe72:761f/64 scope link
valid_lft forever preferred_lft forever
22: tap3eb4fbcf-41 at if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
noqueue master brqd811bcfa-94 state UP group default qlen 1000
link/ether 12:a8:05:bf:98:98 brd ff:ff:ff:ff:ff:ff link-netns
qrouter-278c944f-7e75-4a93-affe-b9ff93f4c6a5
23: tapfd2dca1f-f7 at if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue master brq84f65ccb-c9 state UP group default qlen 1000
link/ether 86:cc:64:22:4d:db brd ff:ff:ff:ff:ff:ff link-netns
qrouter-278c944f-7e75-4a93-affe-b9ff93f4c6a5
[upm at modena ~]$ brctl show
bridge name bridge id STP enabled interfaces
brq84f65ccb-c9 8000.244bfe7c78b8 no enp2s0
tapa467f377-b1
tapfd2dca1f-f7
brqd811bcfa-94 8000.12a805bf9898 no tap3eb4fbcf-41
tap7a390547-5b
vxlan-1
virbr0 8000.525400155b02 yes virbr0-nic
```
This is the compute `ip a` and `brctl show` command output:
```shell
[upm at testarossa ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel
state UP group default qlen 1000
link/ether 24:4b:fe:7c:79:a0 brd ff:ff:ff:ff:ff:ff
inet 138.100.10.26/21 brd 138.100.15.255 scope global noprefixroute
enp2s0
valid_lft forever preferred_lft forever
inet6 fe80::c461:f832:d690:c0a7/64 scope link noprefixroute
valid_lft forever preferred_lft forever
3: enp0s20f0u4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master brq84f65ccb-c9 state UP group default qlen 1000
link/ether 00:e0:4c:53:44:58 brd ff:ff:ff:ff:ff:ff
inet6 fe80::ecff:62cc:d9e1:2e53/64 scope link noprefixroute
valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
state DOWN group default qlen 1000
link/ether 52:54:00:66:84:5a brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc fq_codel master
virbr0 state DOWN group default qlen 1000
link/ether 52:54:00:66:84:5a brd ff:ff:ff:ff:ff:ff
6: brqd811bcfa-94: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
noqueue state UP group default qlen 1000
link/ether 8e:94:21:5b:dc:f4 brd ff:ff:ff:ff:ff:ff
inet 138.100.10.27/21 brd 138.100.15.255 scope global noprefixroute
brqd811bcfa-94
valid_lft forever preferred_lft forever
7: brq84f65ccb-c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
noqueue state UP group default qlen 1000
link/ether 00:e0:4c:53:44:58 brd ff:ff:ff:ff:ff:ff
inet 138.100.10.27/21 brd 138.100.15.255 scope global brq84f65ccb-c9
valid_lft forever preferred_lft forever
9: tap9d1c0de6-96: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
fq_codel master brqd811bcfa-94 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:7d:14:63 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe7d:1463/64 scope link
valid_lft forever preferred_lft forever
10: tapaf15e2f0-24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master brq84f65ccb-c9 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:91:6d:ef brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe91:6def/64 scope link
valid_lft forever preferred_lft forever
11: vxlan-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue
master brqd811bcfa-94 state UNKNOWN group default qlen 1000
link/ether 8e:94:21:5b:dc:f4 brd ff:ff:ff:ff:ff:ff
13: tapef3e95d6-7c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
fq_codel master brqd811bcfa-94 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:91:d1:1d brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe91:d11d/64 scope link
valid_lft forever preferred_lft forever
14: tap1a5d77c0-d4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master brq84f65ccb-c9 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:4f:da:0f brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe4f:da0f/64 scope link
valid_lft forever preferred_lft forever
15: tap792bf660-2f: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc
fq_codel master brqd811bcfa-94 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:f1:df:5f brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fef1:df5f/64 scope link
valid_lft forever preferred_lft forever
16: tapf42b8b2f-be: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc
fq_codel master brq84f65ccb-c9 state UNKNOWN group default qlen 1000
link/ether fe:16:3e:9e:46:bb brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc16:3eff:fe9e:46bb/64 scope link
valid_lft forever preferred_lft forever
[upm at testarossa ~]$ brctl show
bridge name bridge id STP enabled interfaces
brq84f65ccb-c9 8000.00e04c534458 no enp0s20f0u4
tap1a5d77c0-d4
tapaf15e2f0-24
tapf42b8b2f-be
brqd811bcfa-94 8000.8e94215bdcf4 no tap792bf660-2f
tap9d1c0de6-96
tapef3e95d6-7c
vxlan-1
virbr0 8000.52540066845a yes virbr0-nic
```
(The output of `ovs-vsctl show` command is empty in both machines).
**Are the Linux Bridges correctly created?**
These are the Linux bridge configuration files:
* Controller `/etc/neutron/plugins/ml2/linuxbridge_agent.ini`:
```
[linux_bridge]
physical_interface_mappings = provider:enp2s0 # enp2s0 is the
interface associated to 138.100.10.25
[vxlan]
enable_vxlan = true
local_ip = 138.100.10.25 # controller has only 1 IP
l2_population = true
```
* Compute `/etc/neutron/plugins/ml2/linuxbridge_agent.ini`:
```
[linux_bridge]
physical_interface_mappings = provider:enp0s20f0u4 # interface
associated to 138.100.10.26
[vxlan]
enable_vxlan = true
local_ip = 138.100.10.27
l2_population = true
```
An **observation** to keep in mind is that compute management interface
(`138.100.10.26`) is inaccessible from anywhere, which I think is not
correct since this prevents us, for example, from accessing the instance
console through the URL.
I have made some conection tests and these are the results:
* Cirros_a `ip a` command output:
```
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast
qlen 1000
link/ether fa:16:3e:91:d1:1d brd ff:ff:ff:ff:ff:ff
inet 192.168.1.222/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe91:d11d/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether fa:16:3e:4f:da:0f brd ff:ff:ff:ff:ff:ff
```
* Cirros_b `ip a` command output:
```
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast
qlen 1000
link/ether fa:16:3e:f1:df:5f brd ff:ff:ff:ff:ff:ff
inet 192.168.1.30/24 brd 192.168.1.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fef1:df5f/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop qlen 1000
link/ether fa:16:3e:9e:46:bb brd ff:ff:ff:ff:ff:ff
```
- There is **connection** between Cirros A and Cirros B (in both
directions).
- There is **connection** between Cirros A/B and self-service gateway
(192.168.1.1) (in both directions).
- There is **connection** between Cirros A/B and provider gateway
(138.100.10.198) (in both directions).
- There is **connection** between Cirros A/B and controller management
interface (138.100.10.25) (in both directions).
- There is **no connection** between Cirros A/B and compute management
interface (138.100.10.26). This interface is not accessible.
- There is **connection** between Cirros A/B and compute provider
interface (138.100.10.27) (in both directions).
I do not know if there is a problem on linux bridge configuration files,
or maybe I need another network interface on controller machine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210211/9cdcfdd0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mgohjoeiopfjpcbi.png
Type: image/png
Size: 36370 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210211/9cdcfdd0/attachment-0001.png>
More information about the openstack-discuss
mailing list
