[openstack-dev] oaktree - a friendly end-user oriented API layer - anybody want to help?
jaypipes at gmail.com
Wed Nov 16 01:16:59 UTC 2016
Awesome start, Monty :) Comments inline.
On 11/15/2016 09:56 AM, Monty Taylor wrote:
> Hey everybody!
> At this past OpenStack Summit the results of the Interop Challenge were
> shown on stage. It was pretty awesome - 17 different people from 17
> different clouds ran the same workload. And it worked!
> However, one of the reasons it worked is because they all used the
> Ansible modules we wrote that are based on the shade library that
> contains the business logic needed to hide vendor differences in clouds.
> That means that there IS a fantastic OpenStack interoperability story -
> but only if you program in Python. That's less awesome.
> With that in mind - I'm pleased to announce a new project that aims to
> address that - oaktree.
> oaktree is a gRPC-based API porcelain service for OpenStack that is
> based on the shade library and I'd love some help in writing it.
> Basing oaktree on shade gets not only the business logic. Shade already
> understands a multi-cloud world. And because we use shade in Infra for
> nodepool, it already has caching, batching and thundering herd
> protection sorted to be able to hand very high loads efficiently. So
> while oaktree is new, the primary logic and fundamentals are all shade
> and are battle-tested.
++ muy bueno.
> The barrier to deployers adding it to their clouds needs to be as low as
> humanly possible. So as we work on it, ensuring that we keep it
> dead-simple to install, update and operate must be a primary concern.
> Where are we and what's next?
> oaktree doesn't do a whole lot that's terribly interesting at the
> moment. We have all of the development scaffolding and gate jobs set up
> and a few functions implemented.
> oaktree exists currently as two repos - oaktree and oaktreemodel:
> oaktreemodel contains the Protobuf definitions and the build scripts to
> produce Python, C++ and Go code from them. The python code is published
> to PyPI as a normal pure-python library. The C++ code is published as a
> source tarball and the Go code is checked back in to the same repo so
> that go works properly.
Very nice. I recently started playing around with gRPC myself for some
ideas I had about replacing part of nova-compute with a Golang worker
service that can tolerate lengthy disconnections with a centralized
control plane (hello, v[E]CPE!).
It's been (quite) a few years since I last used protobufs (hey, remember
Drizzle?) but it's been a blast getting back into protobufs development.
Now that I see you're using a similar approach for oaktree, I'm
definitely interested in contributing.
> oaktree depends on the python oaktreemodel library, and also on shade.
> It implements the server portion of the gRPC service definition.
> Currently, oaktree can list and search for flavors, images and floating
> ips. Exciting right? Most of the work to expose the rest of the API that
> shade can provide at the moment is going to be fairly straightforward -
> although in each case figuring out the best mapping will take some care.
> We have a few major things that need some good community design. These
> are also listed in a todo.rst file in the oaktree repo which is part of
> the docs:
> The auth story. The native/default auth for gRPC is oauth. It has the
> ability for pluggable auth, but that would raise the barrier for new
> languages. I'd love it if we can come up with a story that involves
> making API users in keystone and authorizing them to use oaktree via an
> oauth transaction.
> The keystone auth backends currently are all about
> integrating with other auth management systems, which is great for
> environments where you have a web browser, but not so much for ones
> where you need to put your auth credentials into a file so that your
> scripts can work. I'm waving my hands wildly here - because all I really
> have are problems to solve and none of the solutions I have are great.
> Glance Image Uploads and Swift Object Uploads (and downloads). Having
> those two data operations go through an API proxy seems inefficient.
Uh, yeah :)
> However, having them not in the API seems like a bad user experience.
> Perhaps if we take advantage of the gRPC streaming protocol support
> doing a direct streaming passthrough actually wouldn't be awful. Or
> maybe the better approach would be for the gRPC call to return a URL and
> token for a user to POST/PUT to directly. Literally no clue.
> In any case - I'd love help from anyone who thinks this sounds like a
> good idea. In a perfect world we'll have something ready for 1.0 by Atlanta.
I'll try my best to dig into the code this week/end.
> Join us in #openstack-shade if you want to hack.
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev