Hypervisors inaccessable with VM on provider network
mfens98 at uvic.ca
Mon Aug 30 16:27:30 UTC 2021
I am running openstack-victoria on centos8 without OVS and I have
followed the installation directions according to the docs with the
option to have self-service networks as our project does not need all
our vm's accessible from the outside. We do still however occasionally
use the provider network for users who need external access to the VM
and in cases where using a floating ip is not sufficient.
We noticed when there is a VM connected to the provider network we lose
access to the hypervisor (cannot ssh or anything, ping still gives a
response though) and the hypervisor itself loses internet access (cannot
ping or curl some test website). We narrowed it down to two issues. One,
we were using firewalld to keep track of our own firewall rules while
openstack uses iptables to enforce its security group rules. These two
did not work well together due to the second issue where openstack
assigns incoming connections to a conntrack zone, it seemed like
firewalld could not handle this correctly and would drop connections
that should have been allowed under its rules.
After transferring our firewall rules to iptables and masking firewalld,
the hypervisor was accessible over ssh but could not access the internet
due to our rule allowing ESTABLISHED and RELATED connections not being
applied properly when a packet was assigned to a conntrack zone.
The work around we found for this was to add a rule where packets
destined for the hypervisor were not assigned to a conntrack zone (add a
rule to the raw iptables table in the PREROUTING chain to just be
accepted if the destination ip address was the address of the
hypervisor). This worked until a new vm was created as openstack
rebuilds the iptables when a change is made and puts it's own rules
above those created by someone other than openstack. To fix this we
changed the code in neutron (iptables_manager.py, the modify_rules
function) to put this rule only above those made by openstack in the raw
iptables table, PREROUTING chain. This fixed our issue, we are now able
to access the hypervisor when VMs are running on a provider network and
VMs and the hypervisor are accessible and able to access the internet.
Security groups are also properly enforced on the VMs as are our
firewall rules assigned elsewhere in iptables.
I'm not sure if this is exactly a bug since in most cases openstack
should be assigning its own firewall rules first in case there are DROP
rules put in by someone else or I could also have missed some
configuration step in which case I would love to know how I can improve.
We thought others may be having a similar issue.
If you'd like more details or have suggestions I'm happy to receive
More information about the openstack-discuss