Open Stack

Hi fellow kuryrs!

This email is gonna be a bit long, so I'll put it in parts.

Kubernetes integration components
==========================

As you know, we are now in the process of upstreaming the Kuryr Kubernetes
PoC that the Kuryr team at Midokura did. This PoC upstreaming effort has
two main parts:

Kuryr watcher: Based on Python3 asyncio, it connects to the ?watch=true
Kubernetes resource endpoints, then passes the seen events to translators
that end up calling Neutron. With the Neutron resource information returned
by the translators, the watching coroutines update the resource that
originated the event.

Kuryr CNI: Py2 and Py3 compatible. It is called by Kubernetes' Kubelet with
the noop container so that the CNI driver does the network plumbing for it.
Basically we use openstack/kuryr binding code to bind Pod veths to Neutron
ports.

Upstream Deployment design
======================

In the Kuryr-Kubernetes integration vision, Kuryr CNI is installed wherever
Kubelet is and the Kuryr watcher (or watchers once we support HA) runs in a
container somewhere that can access the Kubernetes, Neutron and Keystone
APIs (which does not need to be able to access the watcher host on anything
else that established connections). The idea behind allowing it to be in a
non-privileged container somewhere is that in this way you do not need to
make Neutron/Keystone accessible from the Kubernetes worker nodes, just
like for a lot of Nova compute deployments (obviously, depending on you
networking vendor, you have rabbitmq agent access to Neutron).

If one does not need the extra isolation for the Kuryr Watcher, the Watcher
containers could even be started by Kubernetes and the CNI driver would
just let the watcher container in the Host networking instead of on the
Overlay, so Kubernetes would manage the integration deployment.

OpenStack distributions: when the rubber meets the road
==========================================

If the OpenStack distros, however, would prefer not to run Kuryr Watcher
containerized or they want to, as they probably should, build their own
container (rather than the upstream kuryr/kubernetes one in dockerhub that
is based on alpine linux), they would need to have Python3.5 support. I
understand that at the moment from the most popular OpenStack distros, only
one has Python 3.5 supported.

You can imagine where this is heading... These are the options that I can
see:

a) Work with the OpenStack distributions to ensure python3.5 support is
reached soon for Kuryr and its dependencies (some listed below):

   - babel
   - oslo.concurrency
   - oslo.config
   - oslo.log
   - oslo.utils
   - pbr
   - pyroute2
   - python-neutronclient
   - python-keystoneclient

This also implies that distros should adopt a policy about having OpenStack
services running in Python2, some in Python3, as I think the best is to
have each project move at its own speed (within reason).

b) As Ilya Chukhnakov from Mirantis proposed, drop Python3 for now and
reimplement it with python-requests and eventlet. He'll work on a PoC to
see its feasibility and how it compares to the asyncio based one.
Personal position
=============

I see this as a good opportunity for the OpenStack community at wide to
start having Python3-first (and even python3 only) services and allow
OpenStack projects to take full advantage of all the good things Python3
has to offer and move forward with the rest of the Python community.

There's been some efforts in the part in some projects [1][2] but it seems
implementation was deferred indefinitely probably to the same distribution
issue that we face now.

In light of the recent discussions in this mailing list and the decision
taken by the Technical Committee [3] about alternative languages. I think
it would be very good for the community to set an official plan and
incentivize the projects to move to Python3 in future releases
(unfortunately, library projects like clients and oslo will most likely
have to keep python2 for longer, but it is probably for the best).

While such position is not taken, I would like to hear what the rest of the
Kuryr (and the rest of OpenStack) has to say about the matter and we should
at least evaluate the possibility of having to do the option (b) above.

Sorry for the long wall of text, and looking forward to discuss options (a)
and (b) both in these thread and in the next Kuryr weekly meeting this
coming Monday,

Toni

[1]
https://wiki.openstack.org/wiki/Oslo/blueprints/asyncio#What.27s_wrong_with_eventlet.3F
[2] https://blueprints.launchpad.net/oslo.messaging/+spec/asyncio-executor
[3]
http://eavesdrop.openstack.org/meetings/tc/2016/tc.2016-08-02-20.01.log.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160830/ba3d449c/attachment.html>