Open Stack

On Tue, Mar 24, 2020 at 7:28 AM <mdulko at redhat.com> wrote:

> On Mon, 2020-03-23 at 12:17 -0400, Hongbin Lu wrote:
> >
> >
> > On Mon, Mar 23, 2020 at 11:48 AM <mdulko at redhat.com> wrote:
> > > On Sun, 2020-03-22 at 13:28 -0400, Hongbin Lu wrote:
> > > > Hi all,
> > > >
> > > > As we are approaching the end of Ussuri cycle, I would like to take a
> > > > chance to introduce a new feature the Zun team implemented in this
> > > > cycle - CRI integration [1].
>
> <snip!>
>
> > > > Under the hook, a capsule is a podsandbox with one or more containers
> > > > in a CRI runtime (i.e. containerd). Compared to Docker, a CRI runtime
> > > > has a better support for the pod concept so we chose it to implement
> > > > capsule. A caveat is that CRI requires a CNI plugin for the
> > > > networking, so we need to implement a CNI plugin for Zun (called zun-
> > > > cni). The role of CNI plugin is similar as kuryr-libnetwork that we
> > > > are using for Docker except it implements a different networking
> > > > model (CNI). I summaries it as below:
> > >
> > > Hi,
> > >
> > > I noticed that Zun's CNI plugin [1] is basically a simplified version
> > > of kuryr-kubernetes code. While it's totally fine you've copied that, I
> > > wonder what modifications had been made to make it suit Zun? Is there a
> > > chance to converge this to make Zun use kuryr-kubernetes directly so
> > > that we won't develop two versions of that code in parallel?
> >
> > Right. I also investigated the possibilities of reusing the kuryr-
> > kubernetes codebase as well. Definitely, some codes are common among
> > two projects. If we can move the common code to a library (i.e.
> > kuryr-lib), Zun should be able to directly consume the code. In
> > particular, I am interesting to directly consume the CNI binding code
> > (kuryr_kubernetes/cni/binding/) and the VIF versioned object
> > (kuryr_kubernetes/objects).
> >
> > Most parts of kuryr-kubernetes code is coupling with the "list-and-
> > watch" logic against k8s API. Zun is not able to reuse those part of
> > code. However, I do advocate to move all the common code to kuryr-lib
> > so Zun can reuse it whenever it is appropriate.
>
> Uhm, moving more code into kuryr.lib is something Kuryr team would like
> to avoid. Our tendency is rather to stop depending from it, as kuryr-
> kubernetes being a CNI plugin is normally consumed as a container image
> and having any dependencies is a burden there.
>

Kuyur-lib is already a dependency for kuryr-kubernetes:
https://github.com/openstack/kuryr-kubernetes/blob/master/requirements.txt .
Do you mean kuryr-kubernetes is going to remove kuryr-lib as a dependency?
And I don't quite get the "container image" justification. Could you
explain more?


>
> That's why I was asking about modifications to kuryr-daemon code that
> Zun required - to see if we can modify kuryr-daemon to be pluggable
> enough to be consumed by Zun directly.
>

In theory, you can refactor the code and make it pluggable. Suppose you are
able to do that, I would still suggest to move the whole framework out as a
library. That is a prerequisite for Zun (or any other projects) to consume
it, right?


>
> > > Thanks,
> > > Michał
> > >
> > > [1] https://github.com/openstack/zun/tree/master/zun/cni
> > >
> > > > +--------------+------------------------+---------------+
> > > > | Concept      | Container              | Capsule (Pod) |
> > > > +--------------+------------------------+---------------+
> > > > | API endpoint | /containers            | /capsules     |
> > > > | Engine       | Docker                 | CRI runtime   |
> > > > | Network      | kuryr-libnetwork (CNM) | zun-cni (CNI) |
> > > > +--------------+------------------------+---------------+
> > > >
> > > > Typically, a CRI runtime works well with Kata Container which
> > > > provides hypervisor-based isolation for neighboring containers in the
> > > > same node. As a result, it is secure to consolidate pods from
> > > > different tenants into a single node which increases the resource
> > > > utilization. For deployment, a typical stack looks like below:
> > > >
> > > > +----------------------------------------------+
> > > > | k8s control plane                            |
> > > > +----------------------------------------------+
> > > > | Virtual Kubelet (OpenStack provider)         |
> > > > +----------------------------------------------+
> > > > | OpenStack control plane (Zun, Neutron, etc.) |
> > > > +----------------------------------------------+
> > > > | OpenStack data plane                         |
> > > > | (Zun compute agent, Neutron OVS agent, etc.) |
> > > > +----------------------------------------------+
> > > > | Containerd (with CRI plugin)                 |
> > > > +----------------------------------------------+
> > > > | Kata Container                               |
> > > > +----------------------------------------------+
> > > >
> > > > In this stack, if a user creates a deployment or pod in k8s, the k8s
> > > > scheduler will schedule the pod to the virtual node registered by
> > > > Virtual Kubelet. Virtual Kubelet will pick up the pod and let the
> > > > configured cloud provider to handle it. The cloud provider invokes
> > > > Zun API to create a capsule. Upon receiving the API request to create
> > > > a capsule, Zun scheduler will schedule the capsule to a compute node.
> > > > The Zun compute agent in that node will provision the capsule using a
> > > > CRI runtime (containerd in this example). The Zun-CRI runtime
> > > > communication is done via a gRPC protocol through a unix socket. The
> > > > CRI runtime will first create the pod in Kata Container (or runc as
> > > > an alternative) that realizes the pod using a lightweight VM.
> > > > Furthermore, the CRI runtime will use a CNI plugin, which is the zun-
> > > > cni binary, to setup the network. The zun-cni binary is a thin
> > > > executable that dispatches the CNI command to a daemon service called
> > > > zun-cni-daemon. The community is via HTTP within localhost. The zun-
> > > > cni-daemon will look up the Neutron port information from DB and
> > > > perform the port binding.
> > > >
> > > > In conclusion, starting from Ussuri, Zun adds support for CRI-
> > > > compatible runtime. Zun uses CRI runtime to realize the concept of
> > > > pod. Using this feature together with Virtual Kubelet and Kata
> > > > Container, we can offer "serverless kubernetes pod" service which is
> > > > comparable with AWS EKS with Fargate.
> > > >
> > > > [1]
> https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime
> > > > [2] https://github.com/virtual-kubelet/virtual-kubelet
> > > > [3] https://github.com/virtual-kubelet/openstack-zun
> > > > [4]
> https://aws.amazon.com/about-aws/whats-new/2019/12/run-serverless-kubernetes-pods-using-amazon-eks-and-aws-fargate/
> > > > [5]
> https://aws.amazon.com/blogs/aws/amazon-eks-on-aws-fargate-now-generally-available/
> > >
> > >
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200324/5777ebe7/attachment-0001.html>