Open Stack

Hey everybody!

At this past OpenStack Summit the results of the Interop Challenge were
shown on stage. It was pretty awesome - 17 different people from 17
different clouds ran the same workload. And it worked!

However, one of the reasons it worked is because they all used the
Ansible modules we wrote that are based on the shade library that
contains the business logic needed to hide vendor differences in clouds.
That means that there IS a fantastic OpenStack interoperability story -
but only if you program in Python. That's less awesome.

With that in mind - I'm pleased to announce a new project that aims to
address that - oaktree.

oaktree is a gRPC-based API porcelain service for OpenStack that is
based on the shade library and I'd love some help in writing it.

Basing oaktree on shade gets not only the business logic. Shade already
understands a multi-cloud world. And because we use shade in Infra for
nodepool, it already has caching, batching and thundering herd
protection sorted to be able to hand very high loads efficiently. So
while oaktree is new, the primary logic and fundamentals are all shade
and are battle-tested.

The barrier to deployers adding it to their clouds needs to be as low as
humanly possible. So as we work on it, ensuring that we keep it
dead-simple to install, update and operate must be a primary concern.

Where are we and what's next?

oaktree doesn't do a whole lot that's terribly interesting at the
moment. We have all of the development scaffolding and gate jobs set up
and a few functions implemented.

oaktree exists currently as two repos - oaktree and oaktreemodel:

  http://git.openstack.org/cgit/openstack/oaktree
  http://git.openstack.org/cgit/openstack/oaktreemodel

oaktreemodel contains the Protobuf definitions and the build scripts to
produce Python, C++ and Go code from them. The python code is published
to PyPI as a normal pure-python library. The C++ code is published as a
source tarball and the Go code is checked back in to the same repo so
that go works properly.

oaktree depends on the python oaktreemodel library, and also on shade.
It implements the server portion of the gRPC service definition.

Currently, oaktree can list and search for flavors, images and floating
ips. Exciting right? Most of the work to expose the rest of the API that
shade can provide at the moment is going to be fairly straightforward -
although in each case figuring out the best mapping will take some care.

We have a few major things that need some good community design. These
are also listed in a todo.rst file in the oaktree repo which is part of
the docs:

  http://oaktree.readthedocs.io/en/latest/

The auth story. The native/default auth for gRPC is oauth. It has the
ability for pluggable auth, but that would raise the barrier for new
languages. I'd love it if we can come up with a story that involves
making API users in keystone and authorizing them to use oaktree via an
oauth transaction. The keystone auth backends currently are all about
integrating with other auth management systems, which is great for
environments where you have a web browser, but not so much for ones
where you need to put your auth credentials into a file so that your
scripts can work. I'm waving my hands wildly here - because all I really
have are problems to solve and none of the solutions I have are great.

Glance Image Uploads and Swift Object Uploads (and downloads). Having
those two data operations go through an API proxy seems inefficient.
However, having them not in the API seems like a bad user experience.
Perhaps if we take advantage of the gRPC streaming protocol support
doing a direct streaming passthrough actually wouldn't be awful. Or
maybe the better approach would be for the gRPC call to return a URL and
token for a user to POST/PUT to directly. Literally no clue.

In any case - I'd love help from anyone who thinks this sounds like a
good idea. In a perfect world we'll have something ready for 1.0 by Atlanta.

Join us in #openstack-shade if you want to hack.

Thanks!
Monty