Open Stack

Awesome start, Monty :) Comments inline.

On 11/15/2016 09:56 AM, Monty Taylor wrote:
> Hey everybody!
>
> At this past OpenStack Summit the results of the Interop Challenge were
> shown on stage. It was pretty awesome - 17 different people from 17
> different clouds ran the same workload. And it worked!
>
> However, one of the reasons it worked is because they all used the
> Ansible modules we wrote that are based on the shade library that
> contains the business logic needed to hide vendor differences in clouds.
> That means that there IS a fantastic OpenStack interoperability story -
> but only if you program in Python. That's less awesome.
>
> With that in mind - I'm pleased to announce a new project that aims to
> address that - oaktree.
>
> oaktree is a gRPC-based API porcelain service for OpenStack that is
> based on the shade library and I'd love some help in writing it.
>
> Basing oaktree on shade gets not only the business logic. Shade already
> understands a multi-cloud world. And because we use shade in Infra for
> nodepool, it already has caching, batching and thundering herd
> protection sorted to be able to hand very high loads efficiently. So
> while oaktree is new, the primary logic and fundamentals are all shade
> and are battle-tested.

++ muy bueno.

> The barrier to deployers adding it to their clouds needs to be as low as
> humanly possible. So as we work on it, ensuring that we keep it
> dead-simple to install, update and operate must be a primary concern.
>
> Where are we and what's next?
>
> oaktree doesn't do a whole lot that's terribly interesting at the
> moment. We have all of the development scaffolding and gate jobs set up
> and a few functions implemented.
>
> oaktree exists currently as two repos - oaktree and oaktreemodel:
>
>   http://git.openstack.org/cgit/openstack/oaktree
>   http://git.openstack.org/cgit/openstack/oaktreemodel
>
> oaktreemodel contains the Protobuf definitions and the build scripts to
> produce Python, C++ and Go code from them. The python code is published
> to PyPI as a normal pure-python library. The C++ code is published as a
> source tarball and the Go code is checked back in to the same repo so
> that go works properly.

Very nice. I recently started playing around with gRPC myself for some 
ideas I had about replacing part of nova-compute with a Golang worker 
service that can tolerate lengthy disconnections with a centralized 
control plane (hello, v[E]CPE!).

It's been (quite) a few years since I last used protobufs (hey, remember 
Drizzle?) but it's been a blast getting back into protobufs development. 
Now that I see you're using a similar approach for oaktree, I'm 
definitely interested in contributing.

> oaktree depends on the python oaktreemodel library, and also on shade.
> It implements the server portion of the gRPC service definition.
>
> Currently, oaktree can list and search for flavors, images and floating
> ips. Exciting right? Most of the work to expose the rest of the API that
> shade can provide at the moment is going to be fairly straightforward -
> although in each case figuring out the best mapping will take some care.
>
> We have a few major things that need some good community design. These
> are also listed in a todo.rst file in the oaktree repo which is part of
> the docs:
>
>   http://oaktree.readthedocs.io/en/latest/
>
> The auth story. The native/default auth for gRPC is oauth. It has the
> ability for pluggable auth, but that would raise the barrier for new
> languages. I'd love it if we can come up with a story that involves
> making API users in keystone and authorizing them to use oaktree via an
> oauth transaction.

++

 > The keystone auth backends currently are all about
> integrating with other auth management systems, which is great for
> environments where you have a web browser, but not so much for ones
> where you need to put your auth credentials into a file so that your
> scripts can work. I'm waving my hands wildly here - because all I really
> have are problems to solve and none of the solutions I have are great.
>
> Glance Image Uploads and Swift Object Uploads (and downloads). Having
> those two data operations go through an API proxy seems inefficient.

Uh, yeah :)

> However, having them not in the API seems like a bad user experience.
> Perhaps if we take advantage of the gRPC streaming protocol support
> doing a direct streaming passthrough actually wouldn't be awful. Or
> maybe the better approach would be for the gRPC call to return a URL and
> token for a user to POST/PUT to directly. Literally no clue.
>
> In any case - I'd love help from anyone who thinks this sounds like a
> good idea. In a perfect world we'll have something ready for 1.0 by Atlanta.

I'll try my best to dig into the code this week/end.

Best,
-jay

> Join us in #openstack-shade if you want to hack.
>
> Thanks!
> Monty
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>