Open Stack

On 11/15/2016 07:16 PM, Jay Pipes wrote:
> Awesome start, Monty :) Comments inline.

Yay - thanks Jay!

> On 11/15/2016 09:56 AM, Monty Taylor wrote:
>> Hey everybody!
>>
>> At this past OpenStack Summit the results of the Interop Challenge were
>> shown on stage. It was pretty awesome - 17 different people from 17
>> different clouds ran the same workload. And it worked!
>>
>> However, one of the reasons it worked is because they all used the
>> Ansible modules we wrote that are based on the shade library that
>> contains the business logic needed to hide vendor differences in clouds.
>> That means that there IS a fantastic OpenStack interoperability story -
>> but only if you program in Python. That's less awesome.
>>
>> With that in mind - I'm pleased to announce a new project that aims to
>> address that - oaktree.
>>
>> oaktree is a gRPC-based API porcelain service for OpenStack that is
>> based on the shade library and I'd love some help in writing it.
>>
>> Basing oaktree on shade gets not only the business logic. Shade already
>> understands a multi-cloud world. And because we use shade in Infra for
>> nodepool, it already has caching, batching and thundering herd
>> protection sorted to be able to hand very high loads efficiently. So
>> while oaktree is new, the primary logic and fundamentals are all shade
>> and are battle-tested.
> 
> ++ muy bueno.
> 
>> The barrier to deployers adding it to their clouds needs to be as low as
>> humanly possible. So as we work on it, ensuring that we keep it
>> dead-simple to install, update and operate must be a primary concern.
>>
>> Where are we and what's next?
>>
>> oaktree doesn't do a whole lot that's terribly interesting at the
>> moment. We have all of the development scaffolding and gate jobs set up
>> and a few functions implemented.
>>
>> oaktree exists currently as two repos - oaktree and oaktreemodel:
>>
>>   http://git.openstack.org/cgit/openstack/oaktree
>>   http://git.openstack.org/cgit/openstack/oaktreemodel
>>
>> oaktreemodel contains the Protobuf definitions and the build scripts to
>> produce Python, C++ and Go code from them. The python code is published
>> to PyPI as a normal pure-python library. The C++ code is published as a
>> source tarball and the Go code is checked back in to the same repo so
>> that go works properly.
> 
> Very nice. I recently started playing around with gRPC myself for some
> ideas I had about replacing part of nova-compute with a Golang worker
> service that can tolerate lengthy disconnections with a centralized
> control plane (hello, v[E]CPE!).

Well, I've got the protoc -> golang generation working in the gate, so
one step down.

> It's been (quite) a few years since I last used protobufs (hey, remember
> Drizzle?) but it's been a blast getting back into protobufs development.
> Now that I see you're using a similar approach for oaktree, I'm
> definitely interested in contributing.

Yah - turns out they're pretty awesome. They are less flexible in many
respects to REST - but so far I'm finding the limitations are actually
quite nice.

Also - you'll note that oaktreemodel has inherited code from Drizzle's
build system. :)

>> oaktree depends on the python oaktreemodel library, and also on shade.
>> It implements the server portion of the gRPC service definition.
>>
>> Currently, oaktree can list and search for flavors, images and floating
>> ips. Exciting right? Most of the work to expose the rest of the API that
>> shade can provide at the moment is going to be fairly straightforward -
>> although in each case figuring out the best mapping will take some care.
>>
>> We have a few major things that need some good community design. These
>> are also listed in a todo.rst file in the oaktree repo which is part of
>> the docs:
>>
>>   http://oaktree.readthedocs.io/en/latest/
>>
>> The auth story. The native/default auth for gRPC is oauth. It has the
>> ability for pluggable auth, but that would raise the barrier for new
>> languages. I'd love it if we can come up with a story that involves
>> making API users in keystone and authorizing them to use oaktree via an
>> oauth transaction.
> 
> ++
> 
>> The keystone auth backends currently are all about
>> integrating with other auth management systems, which is great for
>> environments where you have a web browser, but not so much for ones
>> where you need to put your auth credentials into a file so that your
>> scripts can work. I'm waving my hands wildly here - because all I really
>> have are problems to solve and none of the solutions I have are great.
>>
>> Glance Image Uploads and Swift Object Uploads (and downloads). Having
>> those two data operations go through an API proxy seems inefficient.
> 
> Uh, yeah :)
> 
>> However, having them not in the API seems like a bad user experience.
>> Perhaps if we take advantage of the gRPC streaming protocol support
>> doing a direct streaming passthrough actually wouldn't be awful. Or
>> maybe the better approach would be for the gRPC call to return a URL and
>> token for a user to POST/PUT to directly. Literally no clue.
>>
>> In any case - I'd love help from anyone who thinks this sounds like a
>> good idea. In a perfect world we'll have something ready for 1.0 by
>> Atlanta.
> 
> I'll try my best to dig into the code this week/end.

Yay! Thrilled to have you dig in.