[magnum] [neutron] [ovn] No inter-node pod-to-pod communication due to missing ACLs in OVN

Slawek Kaplonski skaplons at redhat.com
Tue Dec 15 15:39:55 UTC 2020


On Tue, Dec 15, 2020 at 04:11:25PM +0100, Krzysztof Klimonda wrote:
> Hi,
> This email is a follow-up to a discussion I've openened on ovs-discuss ML[1] regarding lack of TCP/UDP connectivity between pods deployed on magnum-managed k8s cluster with calico CNI and IPIP tunneling disabled (calico_ipv4pool_ipip label set to a default value of Off).
> As a short introduction, during magnum testing in ussuri deployment with ml2/ovn neutron driver I've noticed lack of communication between pods deployed on different nodes as part of magnum deployment with calico configured to *not* encapsulate traffic in IPIP tunnel, but route it directly between nodes. In theory, magnum configures adds defined pod network to k8s nodes ports' allowed_address_pairs[2] and then security group is created allowing for ICMP and TCP/UDP traffic between ports belonging to that security group[3]. This doesn't work with ml2/ovn as TCP/UDP traffic between IP addresses in pod network is not matching ACLs defined in OVN.
> I can't verify this behaviour under ml2/ovs for the next couple of weeks, as I'm taking them off for holidays, but perhaps someone knows if that specific usecase (security group rules with remote groups used with allowed address pairs) is supposed to be working, or should magnum use pod network cidr to allow traffic between nodes instead.

Security group rules with remote groups should works with allowed address pairs
for ML2/OVS. Because of that we even have note in our docs that You shouldn't
add e.g. as allowed address pair for one port as it would effectively
open all Your traffic to all Your ports which are using the same SG.

But from the other hand, we have known issues with scalability of the security
groups with remote group ids as reference in ML2/OVS.
If You have many ports which are using such group, every time new port is added,
all other ports has to be updated to add new IP address to the ipset (or OF
rule) and that make take long time. So using e.g. CIDRs in SG rules works better
for sure.

> [1] https://mail.openvswitch.org/pipermail/ovs-discuss/2020-December/050836.html
> [2] https://github.com/openstack/magnum/blob/c556b8964fab129f33e766b1c33908b2eb001df4/magnum/drivers/k8s_fedora_coreos_v1/templates/kubeminion.yaml
> [3] https://github.com/openstack/magnum/blob/c556b8964fab129f33e766b1c33908b2eb001df4/magnum/drivers/k8s_fedora_coreos_v1/templates/kubecluster.yaml#L1038
> -- 
> Best Regards,
>   - Chris

Slawek Kaplonski
Principal Software Engineer
Red Hat

More information about the openstack-discuss mailing list