<html><body><p>No Morgan. You were supposed to stay quiet on this so we could spread vial behind the scenes rumors on how Monty is trying to bring back CORBA!!! My apologies to all the young folks not familiar with CORBA...<br><br>On a serious note this work has the potential to be extremely valuable and I am look forward to seeing how it matures. Is there an easy way for busy folks to stay up to date on how this progresses? Ideally the interop challenge work (which is continuing forward) should hopefully be able to take advantage of the innovations that this project will deliver.<br><br><br>Thanks,<br><br>Brad<br><br>Brad Topol, Ph.D.<br>IBM Distinguished Engineer<br>OpenStack<br>(919) 543-0646<br>Internet: btopol@us.ibm.com<br>Assistant: Kendra Witherspoon (919) 254-0680<br><br><img width="16" height="16" src="cid:1__=0ABB0AFEDFF54EAE8f9e8a93df938690918c0AB@" border="0" alt="Inactive hide details for Morgan Fainberg ---11/15/2016 08:42:50 PM---On Tue, Nov 15, 2016 at 5:16 PM, Jay Pipes <jaypipes@gmai"><font color="#424282">Morgan Fainberg ---11/15/2016 08:42:50 PM---On Tue, Nov 15, 2016 at 5:16 PM, Jay Pipes <jaypipes@gmail.com> wrote: > Awesome start, Monty :) Com</font><br><br><font size="2" color="#5F5F5F">From: </font><font size="2">Morgan Fainberg <morgan.fainberg@gmail.com></font><br><font size="2" color="#5F5F5F">To: </font><font size="2">"OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org></font><br><font size="2" color="#5F5F5F">Date: </font><font size="2">11/15/2016 08:42 PM</font><br><font size="2" color="#5F5F5F">Subject: </font><font size="2">Re: [openstack-dev] oaktree - a friendly end-user oriented API layer - anybody want to help?</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><br><br><font size="4">On Tue, Nov 15, 2016 at 5:16 PM, Jay Pipes <</font><a href="mailto:jaypipes@gmail.com" target="_blank"><u><font size="4" color="#0000FF">jaypipes@gmail.com</font></u></a><font size="4">> wrote:</font><ul><font size="4">Awesome start, Monty :) Comments inline.<br><br>On 11/15/2016 09:56 AM, Monty Taylor wrote:</font><br><font size="4">Hey everybody!<br><br>At this past OpenStack Summit the results of the Interop Challenge were<br>shown on stage. It was pretty awesome - 17 different people from 17<br>different clouds ran the same workload. And it worked!<br><br>However, one of the reasons it worked is because they all used the<br>Ansible modules we wrote that are based on the shade library that<br>contains the business logic needed to hide vendor differences in clouds.<br>That means that there IS a fantastic OpenStack interoperability story -<br>but only if you program in Python. That's less awesome.<br><br>With that in mind - I'm pleased to announce a new project that aims to<br>address that - oaktree.<br><br>oaktree is a gRPC-based API porcelain service for OpenStack that is<br>based on the shade library and I'd love some help in writing it.<br><br>Basing oaktree on shade gets not only the business logic. Shade already<br>understands a multi-cloud world. And because we use shade in Infra for<br>nodepool, it already has caching, batching and thundering herd<br>protection sorted to be able to hand very high loads efficiently. So<br>while oaktree is new, the primary logic and fundamentals are all shade<br>and are battle-tested.</font><br><font size="4"><br>++ muy bueno.<br></font><br><font size="4">The barrier to deployers adding it to their clouds needs to be as low as<br>humanly possible. So as we work on it, ensuring that we keep it<br>dead-simple to install, update and operate must be a primary concern.<br><br>Where are we and what's next?<br><br>oaktree doesn't do a whole lot that's terribly interesting at the<br>moment. We have all of the development scaffolding and gate jobs set up<br>and a few functions implemented.<br><br>oaktree exists currently as two repos - oaktree and oaktreemodel:<br><br> </font><a href="http://git.openstack.org/cgit/openstack/oaktree" target="_blank"><u><font size="4" color="#0000FF">http://git.openstack.org/cgit/openstack/oaktree</font></u></a><font size="4"><br> </font><a href="http://git.openstack.org/cgit/openstack/oaktreemodel" target="_blank"><u><font size="4" color="#0000FF">http://git.openstack.org/cgit/openstack/oaktreemodel</font></u></a><font size="4"><br><br>oaktreemodel contains the Protobuf definitions and the build scripts to<br>produce Python, C++ and Go code from them. The python code is published<br>to PyPI as a normal pure-python library. The C++ code is published as a<br>source tarball and the Go code is checked back in to the same repo so<br>that go works properly.</font><br><font size="4"><br>Very nice. I recently started playing around with gRPC myself for some ideas I had about replacing part of nova-compute with a Golang worker service that can tolerate lengthy disconnections with a centralized control plane (hello, v[E]CPE!).<br><br>It's been (quite) a few years since I last used protobufs (hey, remember Drizzle?) but it's been a blast getting back into protobufs development. Now that I see you're using a similar approach for oaktree, I'm definitely interested in contributing.<br></font><br><font size="4">oaktree depends on the python oaktreemodel library, and also on shade.<br>It implements the server portion of the gRPC service definition.<br><br>Currently, oaktree can list and search for flavors, images and floating<br>ips. Exciting right? Most of the work to expose the rest of the API that<br>shade can provide at the moment is going to be fairly straightforward -<br>although in each case figuring out the best mapping will take some care.<br><br>We have a few major things that need some good community design. These<br>are also listed in a todo.rst file in the oaktree repo which is part of<br>the docs:<br><br> </font><a href="http://oaktree.readthedocs.io/en/latest/" target="_blank"><u><font size="4" color="#0000FF">http://oaktree.readthedocs.io/en/latest/</font></u></a><font size="4"><br><br>The auth story. The native/default auth for gRPC is oauth. It has the<br>ability for pluggable auth, but that would raise the barrier for new<br>languages. I'd love it if we can come up with a story that involves<br>making API users in keystone and authorizing them to use oaktree via an<br>oauth transaction.</font><br><font size="4"><br>++<br><br>> The keystone auth backends currently are all about</font><br><font size="4">integrating with other auth management systems, which is great for<br>environments where you have a web browser, but not so much for ones<br>where you need to put your auth credentials into a file so that your<br>scripts can work. I'm waving my hands wildly here - because all I really<br>have are problems to solve and none of the solutions I have are great.<br><br>Glance Image Uploads and Swift Object Uploads (and downloads). Having<br>those two data operations go through an API proxy seems inefficient.</font><br><font size="4"><br>Uh, yeah :)<br></font><br><font size="4">However, having them not in the API seems like a bad user experience.<br>Perhaps if we take advantage of the gRPC streaming protocol support<br>doing a direct streaming passthrough actually wouldn't be awful. Or<br>maybe the better approach would be for the gRPC call to return a URL and<br>token for a user to POST/PUT to directly. Literally no clue.<br><br>In any case - I'd love help from anyone who thinks this sounds like a<br>good idea. In a perfect world we'll have something ready for 1.0 by Atlanta.</font><br><font size="4"><br>I'll try my best to dig into the code this week/end.<br><br>Best,<br>-jay</font><br><font size="4"><br></font><br><font size="4">Join us in #openstack-shade if you want to hack.<br><br>Thanks!<br>Monty<br><br><br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: </font><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"><u><font size="4" color="#0000FF">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</font></u></a><u><font size="4" color="#0000FF"><br></font></u><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><u><font size="4" color="#0000FF">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></u></a><font size="4"><br></font><br><font size="4"><br>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: </font><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank"><u><font size="4" color="#0000FF">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</font></u></a><u><font size="4" color="#0000FF"><br></font></u><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank"><u><font size="4" color="#0000FF">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</font></u></a></ul><br><br><font size="4">You know I'm going to be diving back into this. There are some challenges we've already talked about when it comes to Shade->OakTree and OakTree->OakTree type of communications.</font><br><br><font size="4">--Morgan</font><tt>__________________________________________________________________________<br>OpenStack Development Mailing List (not for usage questions)<br>Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe<br></tt><tt><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></tt><tt><br></tt><br><br><BR>
</body></html>