Open Stack

Since a bunch of people had been asking at the Summit, and bunch of
them were interested in contributing, it seemed like a good idea to
put together some status on where the OpenStack SDK project is at.

What follows is the status section of a larger blog post on the SDK at
the Summit, available at
http://blog.briancurtin.com/posts/openstack-sdk-post-summit-update.html


Current Project Status
======================

The OpenStack SDK is quickly heading toward being usable for
application developers. Leading up to the OpenStack Summit we had a
reasonably complete Resource layer and had been working on building
out a higher-level interface, as exposed through the Connection class.
As of now, first cuts of a high-level interface have implementations
in Gerrit for most of the official programs, and we're working to
iterate on what we have in there right now before expanding further.
We also had an impromptu design session on Thursday to cover a couple
of things we'll need to work through.

Project Architecture
********************

At the lowest level, the authentication, session, and transport pieces
have been rounded out and we've been building on them for a while now.
These were some of the first building blocks, and having a reasonably
common approach that multiple service libraries could build on is one
of the project goals.

Session objects are constructed atop Authenticators and Transports.
They get tokens from the Authenticator to insert into your headers,
get endpoints to build up complete URLs, and make HTTP requests on the
Transport, which itself is built on top of requests and handles all
things inbound and outbound from the REST APIs.

On top of that lies the Resource layer, a base class implemented in
openstack/resource.py, which aims to be a 1-1 representation of the
requests or responses the REST APIs are dealing with. For example, the
Server class in openstack/compute/v2/server.py inherits from Resource
and maps to the inputs and outputs of the compute service's `/servers`
endpoint. That Server object contains attributes of type
openstack.resource.prop, which is a class that maps
server-communicated values, such as mapping the accessIPv4 response
body value to an attribute called access_ipv4. This serves two
purposes: one is that it's a place we can bring consistency to the
library when it comes to naming, and two is that props have a type
argument that allows for minimal client-side validation on request
values.

Resource objects are slightly raw to work with directly. They require
you to maintain your own session (it's the first argument of Resource
methods), and they typically only support our thin wrappers around
HTTP verbs. Server.create will take your session and then make a POST
request populated with the props you set on your object.

On top of the Resource layer is the Connection class, which forms our
high-level layer. Connection objects, from openstack/connection.py,
tie together our core pieces - authentication and transport within a
session - and expose namespaces that allow you to work with OpenStack
services from one place. This high-level layer is implemented via
Proxy classes inside of each service's versioned namespace, in their
_proxy.py module.

Right now many of these Proxy implementations are up for review in
Gerrit, but openstack.compute.list_flavors is currently available in
master. It builds on the openstack.compute.v2.flavor Resource, simply
calling its list method inside list_flavors and passing on the Session
that compute was initialized with.

What the high-level looks like
******************************

There are a bunch of example scripts in the works in the Gerrit
reviews, but some of what we're working on looks like the following.

Create a container and object in object storage::

   from openstack import connection
   conn = connection.Connection(auth_url="https://myopenstack:5000/v3",
                                user_name="me", password="secret", ...)
   cnt = conn.object_store.create_container("my_container")
   ob = conn.object_store.create_object(container=cnt, name="my_obj",
                                        data="Hello, world!")

Create a server with a keypair::

   from openstack import connection
   conn = connection.Connection(auth_url="https://myopenstack:5000/v3",
                                user_name="me", password="secret", ...)
   args = {
       "name": "my_server",
       "flavorRef": "big",
       "imageRef": "asdf-1234-qwer-5678",
       "key_name": "my_ssh_key",
   }
   server = conn.compute.create_server(**args)
   servers = conn.compute.list_servers()

Where we're going
*****************

General momentum has carried us into this Connection/Proxy layer,
where we have initial revisions of a number of services, and by
default, we'll just keep pushing on this layer. I expect we'll iterate
on how we want this layer to look, hopefully with input from people
outside of the regular contributors. Outside of that, results from
conversations at the Summit will drive a couple of topics.

1. We need to figure out our story when it comes to versioning APIs at
the high level. Resource classes are under versioned namespaces, and
even the Proxy classes that implement the high level are within the
same versioned namespace, but we currently expose high level objects
through the Connection without a version, as seen in the above
examples.

On one hand, it's pretty nice to not have to think about versions for
APIs that only have a v1, but that won't last. Along with that, we're
working in a dynamic language on growing APIs. Not pinning to a
version of the interface is going to result in a world of pain for
users.

2. We need to think about going even higher level than what we have
now. Monty Taylor's shade (https://github.com/emonty/shade) library
came up both at his "User Experience, SDKs" design session, as well as
during the impromptu OpenStack SDK session we had, and once we get
more of the Connection level figured out, we're going to look at how
we can tackle compound operations.

3. Docs, docs, docs. Terry Howe has been putting in a lot of work on
building up documentation, and now that we're moving along more
smoothly up the stack, I think we'll soon hit the point where code
changes will require doc changes.

I'm also working up a "Getting Started" guide for the project, as we
have some people interested in contributing to the project. Thursday's
python-swiftclient session ended in that team being interested in
shifting their efforts to this SDK, so we need to make sure they can
easily get going and help improve the client and tool landscape.

For the time being, doc builds will appear at
http://python-openstacksdk.readthedocs.org/

4. PyPI releases. Terry put together a version of the package that
could reproduce the examples we showed in our talk on Monday,
comprised of master plus a couple of his in-flight reviews for compute
and network and mine for object store. As we progress and want to try
things out, and to enable people to try along with us, we'll probably
keep cutting more releases under 0.1.

      pip install python-openstacksdk

Keep in mind this is absolutely a work-in-progress, and API stability
isn't yet a thing, so check it out and let us know what you think, but
don't build your business on it.

5. Need to get back into some of the administrivia that we've been
avoiding recently in the name of expanding the Resource layer. The
wiki page could use a refresh to reflect where we're at and what's
going on. We need to start using more blueprints and the issue
tracker, especially as more people become interested in joining the
project. We were able to work without most of that when it was just a
couple of us wanting to get this off the ground, but we need to make
better use of the tools around us.

----

Overall, the SDK is coming along nicely. We had some good talks at the
Summit and got a lot of interest from people and projects, so the
coming months should be another good period of growth for us.