Open Stack

On Tue, Nov 15, 2016 at 5:16 PM, Jay Pipes <jaypipes at gmail.com> wrote:

> Awesome start, Monty :) Comments inline.
>
> On 11/15/2016 09:56 AM, Monty Taylor wrote:
>
>> Hey everybody!
>>
>> At this past OpenStack Summit the results of the Interop Challenge were
>> shown on stage. It was pretty awesome - 17 different people from 17
>> different clouds ran the same workload. And it worked!
>>
>> However, one of the reasons it worked is because they all used the
>> Ansible modules we wrote that are based on the shade library that
>> contains the business logic needed to hide vendor differences in clouds.
>> That means that there IS a fantastic OpenStack interoperability story -
>> but only if you program in Python. That's less awesome.
>>
>> With that in mind - I'm pleased to announce a new project that aims to
>> address that - oaktree.
>>
>> oaktree is a gRPC-based API porcelain service for OpenStack that is
>> based on the shade library and I'd love some help in writing it.
>>
>> Basing oaktree on shade gets not only the business logic. Shade already
>> understands a multi-cloud world. And because we use shade in Infra for
>> nodepool, it already has caching, batching and thundering herd
>> protection sorted to be able to hand very high loads efficiently. So
>> while oaktree is new, the primary logic and fundamentals are all shade
>> and are battle-tested.
>>
>
> ++ muy bueno.
>
> The barrier to deployers adding it to their clouds needs to be as low as
>> humanly possible. So as we work on it, ensuring that we keep it
>> dead-simple to install, update and operate must be a primary concern.
>>
>> Where are we and what's next?
>>
>> oaktree doesn't do a whole lot that's terribly interesting at the
>> moment. We have all of the development scaffolding and gate jobs set up
>> and a few functions implemented.
>>
>> oaktree exists currently as two repos - oaktree and oaktreemodel:
>>
>>   http://git.openstack.org/cgit/openstack/oaktree
>>   http://git.openstack.org/cgit/openstack/oaktreemodel
>>
>> oaktreemodel contains the Protobuf definitions and the build scripts to
>> produce Python, C++ and Go code from them. The python code is published
>> to PyPI as a normal pure-python library. The C++ code is published as a
>> source tarball and the Go code is checked back in to the same repo so
>> that go works properly.
>>
>
> Very nice. I recently started playing around with gRPC myself for some
> ideas I had about replacing part of nova-compute with a Golang worker
> service that can tolerate lengthy disconnections with a centralized control
> plane (hello, v[E]CPE!).
>
> It's been (quite) a few years since I last used protobufs (hey, remember
> Drizzle?) but it's been a blast getting back into protobufs development.
> Now that I see you're using a similar approach for oaktree, I'm definitely
> interested in contributing.
>
> oaktree depends on the python oaktreemodel library, and also on shade.
>> It implements the server portion of the gRPC service definition.
>>
>> Currently, oaktree can list and search for flavors, images and floating
>> ips. Exciting right? Most of the work to expose the rest of the API that
>> shade can provide at the moment is going to be fairly straightforward -
>> although in each case figuring out the best mapping will take some care.
>>
>> We have a few major things that need some good community design. These
>> are also listed in a todo.rst file in the oaktree repo which is part of
>> the docs:
>>
>>   http://oaktree.readthedocs.io/en/latest/
>>
>> The auth story. The native/default auth for gRPC is oauth. It has the
>> ability for pluggable auth, but that would raise the barrier for new
>> languages. I'd love it if we can come up with a story that involves
>> making API users in keystone and authorizing them to use oaktree via an
>> oauth transaction.
>>
>
> ++
>
> > The keystone auth backends currently are all about
>
>> integrating with other auth management systems, which is great for
>> environments where you have a web browser, but not so much for ones
>> where you need to put your auth credentials into a file so that your
>> scripts can work. I'm waving my hands wildly here - because all I really
>> have are problems to solve and none of the solutions I have are great.
>>
>> Glance Image Uploads and Swift Object Uploads (and downloads). Having
>> those two data operations go through an API proxy seems inefficient.
>>
>
> Uh, yeah :)
>
> However, having them not in the API seems like a bad user experience.
>> Perhaps if we take advantage of the gRPC streaming protocol support
>> doing a direct streaming passthrough actually wouldn't be awful. Or
>> maybe the better approach would be for the gRPC call to return a URL and
>> token for a user to POST/PUT to directly. Literally no clue.
>>
>> In any case - I'd love help from anyone who thinks this sounds like a
>> good idea. In a perfect world we'll have something ready for 1.0 by
>> Atlanta.
>>
>
> I'll try my best to dig into the code this week/end.
>
> Best,
> -jay
>
>
> Join us in #openstack-shade if you want to hack.
>>
>> Thanks!
>> Monty
>>
>>
>> ____________________________________________________________
>> ______________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


You know I'm going to be diving back into this. There are some challenges
we've already talked about when it comes to Shade->OakTree and
OakTree->OakTree type of communications.

--Morgan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161115/65aff719/attachment.html>