Open Stack

openstack-sig-k8s update

This Friday, February 16 at 00 UTC, we will updating the K8s community on
the k8s-sig-openstack/openstack-sig-k8s efforts. The major highlights of
our work coming into the Kubernetes 1.10 release include:

1. Preparing the external OpenStack provider for migration into OpenStack
   infrastructure.
2. Removal of experimental Keystone authentication code in upstream in
   favor of community maintained interfaces and web hooks.
3. Integration of K8s e2e testing with the OpenStack provider in
   CNCF/CICD
4. Building out third-party integration testing in OpenStack Infra and
   OpenLab.

If you have other items to address in the update, please contribute
them in the planning etherpad[0].

1. External OpenStack Provider
------------------------------

Dims has been maintaining a fork of the upstream K8s OpenStack provider
in his openstack-cloud-controller-manager (OCCM) archive[1]. The major
point for K8s/OpenStack integration are documented in the repository,
with options including[2]:

    * Using the upstream provider (planned deprecation in 1.11 release).
    * Using the external provider with the Cloud Controller Manager.
      Working scenarios include:
        * External LBaaS with Neutron LBaaSv2
        * Internal LBaaS with Neutron LBaaSv2
        * LVM / iSCSI with Cinder
        * Ceph / RBD with Cinder
    * Keystone authentication and authorization with webhooks.
    * Cinder as a standalone volume provisioner.
    * Cinder with a Flex volume driver.
    * Cinder with a CSI driver.

One major work item for the sig-k8s at the PTG will be in moving OCCM
under OpenStack management. The OCCM code includes the full author and
version history from the Kubernetes repository, and all of the code is
Apache licensed. This is a public notice that the code is being migrated
to OpenStack management.

Included in this migration will be the need to establish gate jobs.
Automating single-node Kubernetes on top of DevStack with conformance
testing is a first high-priority testing task.

2. Removal of experimental Keystone Authentication Code
-------------------------------------------------------

Experimental Keystone Authentication code was removed from upstream
K8s[3]. Webhooks in the OCCM, along with an upstream kubectl auth
provider[4] are the suggested methods for performing Keystone
authorization and authentication from Kubernetes. Savero Proto has
documented how to use Keystone authentication in Kubernetes in a
post on the Switch blog[5].

3. K8s on OpenStack End to End Testing in CNCF/CICD
---------------------------------------------------

Work continues on nightly automated end-to-end (e2e) testing of K8s on
OpenStack using the upstream OpenStack provider. Initial provisioning
code was merged in December in the cross-cloud repository[6]. I met with
the CNCF/CICD team on Tuesday to get a status on testing. There are
still some issues with reliably provisioning load-balancers, and more
work needs to be done make the provisioning robust and ready for e2e
tests. This will be another work item leading into and at the Dublin
PTG. We have two tracking issues within the Kubernetes and Cross-Cloud
repositories[7][8]


4. Third-party integration testing in OpenStack Infra and OpenLab
-----------------------------------------------------------------

In addition to the testing goals for the OCCM in infra as part of standard
gate jobs and nightly test jobs with CNCF/CICD, we also have a goal of
building out third-party jobs within the K8s test infrastructure
suitable for gate testing. This is open work for which we are seeking
volunteers to contribute to it.

Before and at the PTG we would like to with with the Zuul team,
OpenStack infra teams, and OpenLab teams to work out options for
hosting, triggering, and reporting of third-party test jobs.

Dublin PTG
----------

openstack-sig-k8s will have a work session at the Dublin PTG, on Tuesday
February 27. We have an etherpad set up for scheduling work items[9].
In the weeks before the PTG, please add agenda suggestions to the
etherpad.

Thanks!

Chris Hoge
OpenStack Foundation

References
----------

[0] https://etherpad.openstack.org/p/k8s-sig-openstack-update
[1] https://github.com/dims/openstack-cloud-controller-manager
[2] https://github.com/dims/openstack-cloud-controller-manager/blob/master/docs/openstack-kubernetes-integration-options.md
[3] https://github.com/kubernetes/kubernetes/pull/59492
[4] https://github.com/kubernetes/kubernetes/commit/a0cebcb559c5c0ab8a2e50b1ee11cc62f9ebb3a8
[5] https://cloudblog.switch.ch/2018/02/02/openstack-keystone-authentication-for-your-kubernetes-cluster/
[6] https://github.com/crosscloudci/cross-cloud
[7] https://github.com/kubernetes/kubernetes/issues/52249
[8] https://github.com/crosscloudci/crosscloudci/issues/9
[9] https://etherpad.openstack.org/p/sig-k8s-2018-dublin-ptg