Open Stack

On Fri, Aug 4, 2017 at 11:52 AM, Monty Taylor <mordred at inaugust.com> wrote:
[...]
> * Both shade and python-openstackclient have investigated using openstacksdk
> as their REST layer but were unable to because it puts some abstractions in
> layers that make it impossible to do some of the advanced things we need.

OSC _does_ use the SDK for all Network API commands.  That was a
mistake in the sense that we did it before SDK 1.0 was released and
still do not have 1.0.

> * openstacksdk has internal implementations of things that exist at
> different points in the stack. We just added full support for version
> service and version discovery to keystoneauth, but openstacksdk has its own
> layer for that so it both can't use the ksa implementation and is not
> compliant with the API-WG consume guidelines.

This was intended to make it free from dependencies, when first
concieved, none of these other libs existed.

I am coming around to believe that we need to slice a couple more
things up so they can be composed as needed rather then bite off the
entire thing in once piece.

[...]

> I'd propose we have the shade team adopt the python-openstacksdk codebase.

++

> This is obviously an aggressive suggestion and essentially represents a
> takeover of a project. We don't have the luxury of humans to work on things
> that we once had, so I think as a community we should be realistic about the
> benefits of consolidation and the downside to continuing to have 2 different
> python SDKs.

++

I thought it would be natural for OSC to adopt the SDK someday if
Brian did not get around to making it official, but the current
circumstances make it clear that we (OSC) do not have the resources to
do this.  This proposal is much better and leads to a natural
coalescence of the parallel goals and projects.

> Doing that implies the following:
>
> * Rework the underlying guts of openstacksdk to make it possible to replace
> shade's REST layer with openstacksdk. openstacksdk still doesn't have a 1.0
> release, so we can break the few things we'll need to break.

Sigh.  OSC has been using the Network components of the SDK for a long
time in spite of SDK not being at 1.0.  In retrospect that was a
mistake on my part but I believed at the time that 1.0 was close and
we had already ignored Network for far too long.  We already have one
compatibility layer in the SDK due to the proxy refactor work that was
supposed to be the last thing before 1.0.

[...]

> * Merge the two repos and retire one of them. Specifics on the mechanics of
> this below, but this will either result in moving the resource and service
> layer in openstacksdk into shade and adding appropriate attributes to the
> shade.OpenStackCloud object, or moving the shade.OpenStackCloud into
> something like openstack.cloud and making a shade backwards-compate shim. I
> lean towards the first, as we've been telling devs "use shade to talk to
> OpenStack" at hackathons and bootcamps and I'd rather avoid the messaging
> shift. However, pointing to an SDK called "The Python OpenStack SDK" and
> telling people to use it certainly has its benefits from a messaging
> perspective.

I don't have a big concern about which repo is maintained.  For OSC I
want what amounts to a low-level REST API, one example of which can be
found in openstackclient.api.*.  Currently Shade is not quite the
right thing to back a CLI but now has the layer I want, and SDK does
not have that layer at all (it was proposed very early on and not
merged).

Is it better to have a single monolithic thing that has three
conceptual layers internally that can be individually consumed or to
have three things that get layered as needed?

> * drop keystoneauth.Session subclass. It's over-riding things at the wrong
> layer. keystoneauth Adapter is the thing it wants to be.

FWIW, OSC has this problem too...

> * suitability for python-openstackclient. Dean and Steve have been laying in
> the groundwork for doing direct-REST in python-openstackclient because
> python-*client are a mess from an end-user perspective and openstacksdk
> isn't suitable. If we can sync on requirements hopefully we can produce
> something that python-openstackclient can honestly use for that layer
> instead of needing local code.

As I mentioned, we already use the Networking portions of SDK, even
without a 1.0, and it has bit us already a couple of times.  It has
long been my plan to convert to using the SDK, but that was when I
believed there would also be a lower-level API exposed that did not
require all of the application-level goodness and abstractions.

I personally feel like splitting out the low-level REST API layers
into a stand-alone piece that shade, SDK and OSC can all benefit from
would be our best course, but then I have been wrong about this
layering thing in the past, so I throw it out there to have something
that can be used to push against to get what everyone else seems to
want.

dt

-- 

Dean Troyer
dtroyer at gmail.com