On Tue, 2021-10-26 at 22:27 +0000, Jason Anderson wrote:
Hello all,
I’m interested in letting Neutron provide the network configuration frontend for networks realized on k8s kubelets. I have been reading a lot about kuryr-kubernetes and it looks like it fits the bill, but some of the older architecture diagrams I’ve seen indicate that OVS and the neutron-openvswitch-agent (or similar) must also run on the kubelet node. Is this still accurate? I am hoping to avoid this because my understanding is that running the OVS agent means giving the kubelet node access to RabbitMQ and potentially storing admin keystone creds on the node as well.
Can kuryr-kubernetes work without such an agent co-located?
Hi, So short answer is - yes it can. And the long answer is that there are some requirements for that to work. It's called the nested mode [1] and currently we treat it as the major way to run K8s with kuryr-kubernetes. The assumption is that the Kubernetes nodes run as VMs on OpenStack and Kuryr services will run as Pods on those nodes. Kuryr requires the main ports of the VMs to be Neutron trunk ports and will create the ports for the Pods as subports of these trunk ports. This removes the need for neutron-openvswitch- agent to exist on the K8s node as Kuryr can bind such ports on its own. The requirements are as follows: * K8s nodes run as VMs on OpenStack. * Trunk extension is enabled in Neutron. * VMs have access to OpenStack API endpoints. * You need Octavia to support K8s Services. In terms of admin credentials - those should not be needed in nested mode, just regular tenant credentials should be fine. If your K8s nodes are required to be baremetal, then maybe using OVN as a Neutron backend instead of OVS will solve the RabbitMQ problem? I think you'll still need the ovn-controller to run on the K8s nodes to bind the Neutron ports there. And I think this mode might actually require admin credentials in order to attach ports to nodes. [1] https://docs.openstack.org/kuryr-kubernetes/latest/nested_vlan_mode.html Thanks, Michał
Thanks! Jason Anderson
---
Chameleon DevOps Lead Department of Computer Science, University of Chicago Mathematics and Computer Science, Argonne National Laboratory