[openstack-dev] [python-openstacksdk] Status of python-openstacksdk project
mordred at inaugust.com
Fri Aug 4 23:05:13 UTC 2017
On 08/04/2017 04:03 PM, Dean Troyer wrote:
> On Fri, Aug 4, 2017 at 11:52 AM, Monty Taylor <mordred at inaugust.com> wrote:
>> * Both shade and python-openstackclient have investigated using openstacksdk
>> as their REST layer but were unable to because it puts some abstractions in
>> layers that make it impossible to do some of the advanced things we need.
> OSC _does_ use the SDK for all Network API commands. That was a
> mistake in the sense that we did it before SDK 1.0 was released and
> still do not have 1.0.
Yah - although it got you out of the competing cliff issue, so that was
>> * openstacksdk has internal implementations of things that exist at
>> different points in the stack. We just added full support for version
>> service and version discovery to keystoneauth, but openstacksdk has its own
>> layer for that so it both can't use the ksa implementation and is not
>> compliant with the API-WG consume guidelines.
> This was intended to make it free from dependencies, when first
> concieved, none of these other libs existed.
Oh - totally. It made total sense at the time ... it's just the
surrounding context has changed.
> I am coming around to believe that we need to slice a couple more
> things up so they can be composed as needed rather then bite off the
> entire thing in once piece.
>> I'd propose we have the shade team adopt the python-openstacksdk codebase.
>> This is obviously an aggressive suggestion and essentially represents a
>> takeover of a project. We don't have the luxury of humans to work on things
>> that we once had, so I think as a community we should be realistic about the
>> benefits of consolidation and the downside to continuing to have 2 different
>> python SDKs.
> I thought it would be natural for OSC to adopt the SDK someday if
> Brian did not get around to making it official, but the current
> circumstances make it clear that we (OSC) do not have the resources to
> do this. This proposal is much better and leads to a natural
> coalescence of the parallel goals and projects.
>> Doing that implies the following:
>> * Rework the underlying guts of openstacksdk to make it possible to replace
>> shade's REST layer with openstacksdk. openstacksdk still doesn't have a 1.0
>> release, so we can break the few things we'll need to break.
> Sigh. OSC has been using the Network components of the SDK for a long
> time in spite of SDK not being at 1.0. In retrospect that was a
> mistake on my part but I believed at the time that 1.0 was close and
> we had already ignored Network for far too long. We already have one
> compatibility layer in the SDK due to the proxy refactor work that was
> supposed to be the last thing before 1.0.
I don't think we need to break the things you're using, fwiw. In fact,
we can probably take it as a first-order requirement to not break OSC -
unless we find something SUPER intractable - and even then we should
talk about it first.
>> * Merge the two repos and retire one of them. Specifics on the mechanics of
>> this below, but this will either result in moving the resource and service
>> layer in openstacksdk into shade and adding appropriate attributes to the
>> shade.OpenStackCloud object, or moving the shade.OpenStackCloud into
>> something like openstack.cloud and making a shade backwards-compate shim. I
>> lean towards the first, as we've been telling devs "use shade to talk to
>> OpenStack" at hackathons and bootcamps and I'd rather avoid the messaging
>> shift. However, pointing to an SDK called "The Python OpenStack SDK" and
>> telling people to use it certainly has its benefits from a messaging
> I don't have a big concern about which repo is maintained. For OSC I
> want what amounts to a low-level REST API, one example of which can be
> found in openstackclient.api.*. Currently Shade is not quite the
> right thing to back a CLI but now has the layer I want, and SDK does
> not have that layer at all (it was proposed very early on and not
FWIW - now that we landed version discovery and microversion support in
keystoneauth - I'd say keystoneauth Adapter is actually now the
low-level REST API:
client = keystoneauth1.adapter.Adapter(
endpoint_data = client.get_endpoint_data()
"Microversion range is: ",
# want 2.31 of servers
server_response = client.get('/servers', microversion='2.31')
server = server_response.json()['servers']
# want 2.14 of keypairs
keypair_response = client.get('/keypairs', microversion='2.14')
# Don't care on volume attachments - don't specify one.
volume_attachments_response = client.get(
> Is it better to have a single monolithic thing that has three
> conceptual layers internally that can be individually consumed or to
> have three things that get layered as needed?
It's a good question. I'm leaning to single repo for shade and sdk
mainly because since both do direct REST and dont' have other
dependencies, there's not really much difference from a composability
standpoint. If you just want the low-level SDK functions, use them. If
you want the hide-the-difference functions, use them - doesn't gain much
to have them split.
That said - if merging them is too much work, it's not super IMPORTANT
to merge them either.
>> * drop keystoneauth.Session subclass. It's over-riding things at the wrong
>> layer. keystoneauth Adapter is the thing it wants to be.
> FWIW, OSC has this problem too...
Yah - I'm totally sorry I forgot to try to get your timing wrapper in to
ksa Session this pass so that you could drop that wrapper...
>> * suitability for python-openstackclient. Dean and Steve have been laying in
>> the groundwork for doing direct-REST in python-openstackclient because
>> python-*client are a mess from an end-user perspective and openstacksdk
>> isn't suitable. If we can sync on requirements hopefully we can produce
>> something that python-openstackclient can honestly use for that layer
>> instead of needing local code.
> As I mentioned, we already use the Networking portions of SDK, even
> without a 1.0, and it has bit us already a couple of times. It has
> long been my plan to convert to using the SDK, but that was when I
> believed there would also be a lower-level API exposed that did not
> require all of the application-level goodness and abstractions.
> I personally feel like splitting out the low-level REST API layers
> into a stand-alone piece that shade, SDK and OSC can all benefit from
> would be our best course, but then I have been wrong about this
> layering thing in the past, so I throw it out there to have something
> that can be used to push against to get what everyone else seems to
Nod. I think that's ksa. We're doing almost nothing at the REST layer in
shade at this point - we're just using ksa Adapter. We have a wrapper
around it for two reasons:
- TaskManager support for nodepool
- translating ksa REST exceptions to shade exceptions for backwards compat
Then we do data normalization - but that's a 'hide cloud differences'
thing, not a thing that should be at that lower level.
rods actually just finished removing one additional thing we were doing,
which was stripping the top-level key from the results if there was one.
Turns out doing that really screws with pagination. whoops.
In any case, I agree with the need for a low-level REST interface.
Mostly since last we talked about it the missing pieces have hit ksa.
Looking at openstackclient.api.* though - we may be meaning different
things when we say "low level REST interface".
From the shade/sdk perspective, the extra metadata that's tracked for
each Resource object will, I _believe_ make it easier to add basic new
CRUD for each resource type. I could certainly be VERY wrong about that
More information about the OpenStack-dev