Hypervisors inaccessable with VM on provider network
Hello, I am running openstack-victoria on centos8 without OVS and I have followed the installation directions according to the docs with the option to have self-service networks as our project does not need all our vm's accessible from the outside. We do still however occasionally use the provider network for users who need external access to the VM and in cases where using a floating ip is not sufficient. We noticed when there is a VM connected to the provider network we lose access to the hypervisor (cannot ssh or anything, ping still gives a response though) and the hypervisor itself loses internet access (cannot ping or curl some test website). We narrowed it down to two issues. One, we were using firewalld to keep track of our own firewall rules while openstack uses iptables to enforce its security group rules. These two did not work well together due to the second issue where openstack assigns incoming connections to a conntrack zone, it seemed like firewalld could not handle this correctly and would drop connections that should have been allowed under its rules. After transferring our firewall rules to iptables and masking firewalld, the hypervisor was accessible over ssh but could not access the internet due to our rule allowing ESTABLISHED and RELATED connections not being applied properly when a packet was assigned to a conntrack zone. The work around we found for this was to add a rule where packets destined for the hypervisor were not assigned to a conntrack zone (add a rule to the raw iptables table in the PREROUTING chain to just be accepted if the destination ip address was the address of the hypervisor). This worked until a new vm was created as openstack rebuilds the iptables when a change is made and puts it's own rules above those created by someone other than openstack. To fix this we changed the code in neutron (iptables_manager.py, the modify_rules function) to put this rule only above those made by openstack in the raw iptables table, PREROUTING chain. This fixed our issue, we are now able to access the hypervisor when VMs are running on a provider network and VMs and the hypervisor are accessible and able to access the internet. Security groups are also properly enforced on the VMs as are our firewall rules assigned elsewhere in iptables. I'm not sure if this is exactly a bug since in most cases openstack should be assigning its own firewall rules first in case there are DROP rules put in by someone else or I could also have missed some configuration step in which case I would love to know how I can improve. We thought others may be having a similar issue. If you'd like more details or have suggestions I'm happy to receive feedback. Cheers, Matt
participants (1)
-
Matthew Ens