Open Stack

On Thu, Jan 16, 2014 at 12:03 AM, Flavio Percoco <flavio at redhat.com> wrote:

> On 15/01/14 21:35 +0000, Jesse Noller wrote:
>
>>
>> On Jan 15, 2014, at 1:37 PM, Doug Hellmann <doug.hellmann at dreamhost.com>
>> wrote:
>>
>>  Several people have mentioned to me that they are interested in, or
>>> actively working on, code related to a "common" client library -- something
>>> meant to be reused directly as a basis for creating a common library for
>>> all of the openstack clients to use. There's a blueprint [1] in oslo, and I
>>> believe the keystone devs and unified CLI teams are probably interested in
>>> ensuring that the resulting API ends up meeting all of our various
>>> requirements.
>>>
>>> If you're interested in this effort, please subscribe to the blueprint
>>> and use that to coordinate efforts so we don't produce more than one common
>>> library. ;-)
>>>
>>> Thanks,
>>> Doug
>>>
>>>
>>> [1] https://blueprints.launchpad.net/oslo/+spec/common-client-library-2
>>>
>>
>> *raises hand*
>>
>> Me me!
>>
>> I’ve been talking to many contributors about the Developer Experience
>> stuff I emailed out prior to the holidays and I was starting blueprint
>> work, but this is a great pointer. I’m going to have to sync up with Alexei.
>>
>> I think solving this for openstack developers and maintainers as the
>> blueprint says is a big win in terms of code reuse / maintenance and
>> consistent but more so for *end-user developers* consuming openstack clouds.
>>
>> Some background - there’s some terminology mismatch but the rough idea is
>> the same:
>>
>> * A centralized “SDK” (Software Development Kit) would be built
>> condensing the common code and logic and operations into a single namespace.
>>
>> * This SDK would be able to be used by “downstream” CLIs - essentially
>> the CLIs become a specialized front end - and in some cases, only an
>> argparse or cliff front-end to the SDK methods located in the (for example)
>> openstack.client.api.compute
>>
>> * The SDK would handle Auth, re-auth (expired tokens, etc) for long-lived
>> clients - all of the openstack.client.api.** classes would accept an Auth
>> object to delegate management / mocking of the Auth / service catalog stuff
>> to. This means developers building applications (say for example, horizon)
>> don’t need to worry about token/expired authentication/etc.
>>
>> * Simplify the dependency graph & code for the existing tools to enable
>> single binary installs (py2exe, py2app, etc) for end users of the command
>> line tools.
>>
>> Short version: if a developer wants to consume an openstack cloud; the
>> would have a single SDK with minimal dependencies and import from a single
>> namespace. An example application might look like:
>>
>> from openstack.api import AuthV2
>> from openstack.api import ComputeV2
>>
>> myauth = AuthV2(…., connect=True)
>> compute = ComputeV2(myauth)
>>
>> compute.list_flavors()
>>
>>
> I know this is an example but, could we leave the version out of the
> class name? Having something like:
>
> from openstack.api.v2 import Compute
>
>        or
>
> from openstack.compute.v2 import Instance
>
> (just made that up)
>
> for marconi we're using the later.


Just throwing this out there because it seems relevant to client design.

As we've been looking at porting clients to using v2 of the Images API, its
seems more and more to me that including the *server* version in the main
import path is a real obstacle.

IMO any future client libs should write library interfaces based on the
peculiarities of user needs, not based on the vagaries of the server
version. So as a user of this library I would do something like:

  1 from openstack.api import images
  2 client = images.make_me_a_client(auth_url, etcetera) # all version
negotiation is happening here
  3 client.list_images()  # works more or less same no matter who I'm
talking to

Now, there would still likely be hidden implementation code that is
different per server version and which is instantiated in line 2 above, and
maybe that's the library path stuff you are talking about.


>
>
>  This greatly improves the developer experience both internal to openstack
>> and externally. Currently OpenStack has 22+ (counting stackforge) potential
>> libraries a developer may need to install to use a full deployment of
>> OpenStack:
>>
>>  * python-keystoneclient (identity)
>>  * python-glanceclient (image)
>>  * python-novaclient (compute)
>>  * python-troveclient (database)
>>  * python-neutronclient (network)
>>  * python-ironicclient (bare metal)
>>  * python-heatclient (orchestration)
>>  * python-cinderclient (block storage)
>>  * python-ceilometerclient (telemetry, metrics & billing)
>>  * python-swiftclient (object storage)
>>  * python-savannaclient (big data)
>>  * python-openstackclient (meta client package)
>>  * python-marconiclient (queueing)
>>  * python-tuskarclient (tripleo / management)
>>  * python-melangeclient (dead)
>>  * python-barbicanclient (secrets)
>>  * python-solumclient (ALM)
>>  * python-muranoclient (application catalog)
>>  * python-manilaclient (shared filesystems)
>>  * python-libraclient (load balancers)
>>  * python-climateclient (reservations)
>>  * python-designateclient (Moniker/DNS)
>>
>> If you exclude the above and look on PyPI:
>>
>> On PyPi (client libraries/SDKs only, excluding the above - not maintained
>> by openstack):
>>
>> * hpcloud-auth-openstack 1.0
>> * python-openstackclient 0.2.2
>> * rackspace-auth-openstack 1.1
>> * posthaste 0.2.2
>> * pyrax 1.6.2
>> * serverherald 0.0.1
>> * warm 0.3.1
>> * vaporize 0.3.2
>> * swiftsc (https://github.com/mkouhei/swiftsc)
>> * bookofnova 0.007
>> * nova-adminclient 0.1.8
>> * python-quantumclient 2.2.4.3
>> * python-stackhelper 0.0.7.1.gcab1eb0
>> * swift-bench 1.0
>> * swiftly 1.12
>> * txAWS 0.2.3
>> * cfupload 0.5.1
>> * python-reddwarfclient 0.1.2
>> * python-automationclient 1.2.1
>> * rackspace-glanceclient 0.9
>> * rackspace-novaclient 1.4
>>
>> If you ignore PyPI and just want to install the base say - 7 services,
>> each one of the python-** clients has its own dependency tree and may or
>> may not build from one of the others. If a vendor wants to extend any of
>> them, it’s basically a fork instead of a clean plugin system.
>>
>> On the CLI front - this would *greatly* simplify the work openstackclient
>> needs to do - it would be able to import from the main SDK and simply
>> provide the noun-verb command line and any other end-user sugar it wanted
>> to. Even if each service wanted to keep its own python-X client instead of
>> relying on openstackclient it would be minimal to depend on the core SDK
>> and then plugin/extend to build a specialized CLI for the project - if you
>> really wanted, you could also extend openstackclient directly.
>>
>
> Thanks for the info. It's a great way to see where we're standing now
> and the relevance of this argument.
>
>
>
>> Roughly this is the punch list I was looking at:
>>
>> 1: a blueprint that explains the rationale behind unifying the Client
>> code from the openstack clients; using a single REST interface, common
>> object hierarchy, etc.
>> 2: A path for implementation of the common SDK including operational code
>>
>> 4: dealing with a single binary cross platform for the CLI that derives
>> from the common SDK (hard requirement)
>> 5: Standardization of names (e.g Compute != Nova, use the real names, not
>> project names)
>> 6: Allow vendors to alias names for services to match their offerings
>>
>
> +1
>
> Keeping the library separated from the CLI binary is a most. Marconi
> client, for instance, is just a library and we are planning to use
> openstack's common client for the CLI.
>
>
>
>> I’ll begin working on the blueprint you pointed to - given this is more
>> akin to a horizon-like UX project than a sub project of Oslo itself; does
>> it really belong there? I do see the work within the individual clients:
>>
>> https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z
>>
>> Jamie’s comments in:
>>
>> https://review.openstack.org/#/c/63164/
>>
>> Do concern me as I’d like to not do this as a lowest common denominator;
>> in this case the client code for keystoneclient might be in
>> openstack.api.auth but it would be able to be as advanced as it would like
>> from an api standpoint - and whatever subset of functionality could be
>> exposed in higher level abstractions (such as a CLI). Bonus is that horizon
>> could potentially use this work.
>>
>
> This one last point is very important as well. The levels of
> abstraction of the common SDK and CLI shouldn't prevent services from
> specializing it.
>
> FF
>
> --
> @flaper87
> Flavio Percoco
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140116/5121b0e9/attachment.html>