Open Stack

On Tue, 2021-10-26 at 22:27 +0000, Jason Anderson wrote:
> Hello all,
> 
> I’m interested in letting Neutron provide the network configuration
> frontend for networks realized on k8s kubelets. I have been reading a
> lot about kuryr-kubernetes and it looks like it fits the bill, but
> some of the older architecture diagrams I’ve seen indicate that OVS
> and the neutron-openvswitch-agent (or similar) must also run on the
> kubelet node. Is this still accurate? I am hoping to avoid this
> because my understanding is that running the OVS agent means giving
> the kubelet node access to RabbitMQ and potentially storing admin
> keystone creds on the node as well.
> 
> Can kuryr-kubernetes work without such an agent co-located?

Hi,

So short answer is - yes it can. And the long answer is that there are
some requirements for that to work.

It's called the nested mode [1] and currently we treat it as the major
way to run K8s with kuryr-kubernetes. The assumption is that the
Kubernetes nodes run as VMs on OpenStack and Kuryr services will run as
Pods on those nodes. Kuryr requires the main ports of the VMs to be
Neutron trunk ports and will create the ports for the Pods as subports
of these trunk ports. This removes the need for neutron-openvswitch-
agent to exist on the K8s node as Kuryr can bind such ports on its own.

The requirements are as follows:
* K8s nodes run as VMs on OpenStack.
* Trunk extension is enabled in Neutron.
* VMs have access to OpenStack API endpoints.
* You need Octavia to support K8s Services.

In terms of admin credentials - those should not be needed in nested
mode, just regular tenant credentials should be fine.

If your K8s nodes are required to be baremetal, then maybe using OVN as
a Neutron backend instead of OVS will solve the RabbitMQ problem? I
think you'll still need the ovn-controller to run on the K8s nodes to
bind the Neutron ports there. And I think this mode might actually
require admin credentials in order to attach ports to nodes.

[1]
https://docs.openstack.org/kuryr-kubernetes/latest/nested_vlan_mode.html

Thanks,
Michał

> Thanks!
> Jason Anderson
> 
> ---
> 
> Chameleon DevOps Lead
> Department of Computer Science, University of Chicago
> Mathematics and Computer Science, Argonne National Laboratory
>