Open Stack

> On Jan 16, 2014, at 2:09 AM, "Flavio Percoco" <flavio at redhat.com> wrote:
> 
>> On 15/01/14 21:35 +0000, Jesse Noller wrote:
>> 
>>> On Jan 15, 2014, at 1:37 PM, Doug Hellmann <doug.hellmann at dreamhost.com> wrote:
>>> 
>>> Several people have mentioned to me that they are interested in, or actively working on, code related to a "common" client library -- something meant to be reused directly as a basis for creating a common library for all of the openstack clients to use. There's a blueprint [1] in oslo, and I believe the keystone devs and unified CLI teams are probably interested in ensuring that the resulting API ends up meeting all of our various requirements.
>>> 
>>> If you're interested in this effort, please subscribe to the blueprint and use that to coordinate efforts so we don't produce more than one common library. ;-)
>>> 
>>> Thanks,
>>> Doug
>>> 
>>> 
>>> [1] https://blueprints.launchpad.net/oslo/+spec/common-client-library-2
>> 
>> *raises hand*
>> 
>> Me me!
>> 
>> I’ve been talking to many contributors about the Developer Experience stuff I emailed out prior to the holidays and I was starting blueprint work, but this is a great pointer. I’m going to have to sync up with Alexei.
>> 
>> I think solving this for openstack developers and maintainers as the blueprint says is a big win in terms of code reuse / maintenance and consistent but more so for *end-user developers* consuming openstack clouds.
>> 
>> Some background - there’s some terminology mismatch but the rough idea is the same:
>> 
>> * A centralized “SDK” (Software Development Kit) would be built condensing the common code and logic and operations into a single namespace.
>> 
>> * This SDK would be able to be used by “downstream” CLIs - essentially the CLIs become a specialized front end - and in some cases, only an argparse or cliff front-end to the SDK methods located in the (for example) openstack.client.api.compute
>> 
>> * The SDK would handle Auth, re-auth (expired tokens, etc) for long-lived clients - all of the openstack.client.api.** classes would accept an Auth object to delegate management / mocking of the Auth / service catalog stuff to. This means developers building applications (say for example, horizon) don’t need to worry about token/expired authentication/etc.
>> 
>> * Simplify the dependency graph & code for the existing tools to enable single binary installs (py2exe, py2app, etc) for end users of the command line tools.
>> 
>> Short version: if a developer wants to consume an openstack cloud; the would have a single SDK with minimal dependencies and import from a single namespace. An example application might look like:
>> 
>> from openstack.api import AuthV2
>> from openstack.api import ComputeV2
>> 
>> myauth = AuthV2(…., connect=True)
>> compute = ComputeV2(myauth)
>> 
>> compute.list_flavors()
> 
> I know this is an example but, could we leave the version out of the
> class name? Having something like:
> 
> from openstack.api.v2 import Compute
> 
>       or
> 
> from openstack.compute.v2 import Instance
> 
> (just made that up)
> 
> for marconi we're using the later.

Definitely; it should be based on namespaces. 

> 
>> This greatly improves the developer experience both internal to openstack and externally. Currently OpenStack has 22+ (counting stackforge) potential libraries a developer may need to install to use a full deployment of OpenStack:
>> 
>> * python-keystoneclient (identity)
>> * python-glanceclient (image)
>> * python-novaclient (compute)
>> * python-troveclient (database)
>> * python-neutronclient (network)
>> * python-ironicclient (bare metal)
>> * python-heatclient (orchestration)
>> * python-cinderclient (block storage)
>> * python-ceilometerclient (telemetry, metrics & billing)
>> * python-swiftclient (object storage)
>> * python-savannaclient (big data)
>> * python-openstackclient (meta client package)
>> * python-marconiclient (queueing)
>> * python-tuskarclient (tripleo / management)
>> * python-melangeclient (dead)
>> * python-barbicanclient (secrets)
>> * python-solumclient (ALM)
>> * python-muranoclient (application catalog)
>> * python-manilaclient (shared filesystems)
>> * python-libraclient (load balancers)
>> * python-climateclient (reservations)
>> * python-designateclient (Moniker/DNS)
>> 
>> If you exclude the above and look on PyPI:
>> 
>> On PyPi (client libraries/SDKs only, excluding the above - not maintained by openstack):
>> 
>> * hpcloud-auth-openstack 1.0
>> * python-openstackclient 0.2.2
>> * rackspace-auth-openstack 1.1
>> * posthaste 0.2.2
>> * pyrax 1.6.2
>> * serverherald 0.0.1
>> * warm 0.3.1
>> * vaporize 0.3.2
>> * swiftsc (https://github.com/mkouhei/swiftsc)
>> * bookofnova 0.007
>> * nova-adminclient 0.1.8
>> * python-quantumclient 2.2.4.3
>> * python-stackhelper 0.0.7.1.gcab1eb0
>> * swift-bench 1.0
>> * swiftly 1.12
>> * txAWS 0.2.3
>> * cfupload 0.5.1
>> * python-reddwarfclient 0.1.2
>> * python-automationclient 1.2.1
>> * rackspace-glanceclient 0.9
>> * rackspace-novaclient 1.4
>> 
>> If you ignore PyPI and just want to install the base say - 7 services, each one of the python-** clients has its own dependency tree and may or may not build from one of the others. If a vendor wants to extend any of them, it’s basically a fork instead of a clean plugin system.
>> 
>> On the CLI front - this would *greatly* simplify the work openstackclient needs to do - it would be able to import from the main SDK and simply provide the noun-verb command line and any other end-user sugar it wanted to. Even if each service wanted to keep its own python-X client instead of relying on openstackclient it would be minimal to depend on the core SDK and then plugin/extend to build a specialized CLI for the project - if you really wanted, you could also extend openstackclient directly.
> 
> Thanks for the info. It's a great way to see where we're standing now
> and the relevance of this argument.
> 
>> 
>> Roughly this is the punch list I was looking at:
>> 
>> 1: a blueprint that explains the rationale behind unifying the Client code from the openstack clients; using a single REST interface, common object hierarchy, etc.
>> 2: A path for implementation of the common SDK including operational code
>> 
>> 4: dealing with a single binary cross platform for the CLI that derives from the common SDK (hard requirement)
>> 5: Standardization of names (e.g Compute != Nova, use the real names, not project names)
>> 6: Allow vendors to alias names for services to match their offerings
> 
> +1
> 
> Keeping the library separated from the CLI binary is a most. Marconi
> client, for instance, is just a library and we are planning to use
> openstack's common client for the CLI.
> 
>> 
>> I’ll begin working on the blueprint you pointed to - given this is more akin to a horizon-like UX project than a sub project of Oslo itself; does it really belong there? I do see the work within the individual clients:
>> 
>> https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z
>> 
>> Jamie’s comments in:
>> 
>> https://review.openstack.org/#/c/63164/
>> 
>> Do concern me as I’d like to not do this as a lowest common denominator; in this case the client code for keystoneclient might be in openstack.api.auth but it would be able to be as advanced as it would like from an api standpoint - and whatever subset of functionality could be exposed in higher level abstractions (such as a CLI). Bonus is that horizon could potentially use this work.
> 
> This one last point is very important as well. The levels of
> abstraction of the common SDK and CLI shouldn't prevent services from
> specializing it.
> 
> FF
> 
> -- 
> @flaper87
> Flavio Percoco
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev