<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif;"><div><div>+1 to a stand alone library for this.</div><div><br></div></div><div>>> 1) We would have to maintain rationale versioning and backwards compatibility of this library. If we start library from scratch we'll have to add/change lots of stuff before we'll reach some stability period.</div><div><br></div><div>I don’t think this is a hard problem to solve. I think something like semantic versioning fits the bill, and allows for backwards-incompatible changes. </div><div><br></div><div>>> 2)Another option would be to follow waterfall process and create a solid library interface before including it to all client projects. However such approach this period can take unknown amount of time and can be easily failed during integration stage cause requirements change or some other reason.</div><div><br></div><div>Again, I think semver could address this. Iterate over 0.x.x series, and projects that don’t want to deal with constantly-changing code wouldn’t need to jump on board until a 1.0.0 release.</div><div><br></div><div>-Doug M.</div><div><br></div><span id="OLK_SRC_BODY_SECTION"><div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt"><span style="font-weight:bold">From: </span> Alexei Kornienko <<a href="mailto:alexei.kornienko@gmail.com">alexei.kornienko@gmail.com</a>><br><span style="font-weight:bold">Reply-To: </span> "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br><span style="font-weight:bold">Date: </span> Thursday, January 16, 2014 at 1:03 PM<br><span style="font-weight:bold">To: </span> "<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br><span style="font-weight:bold">Subject: </span> Re: [openstack-dev] a "common" client library<br></div><div><br></div><div><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><div bgcolor="#FFFFFF" text="#000000"><div class="moz-cite-prefix">Hello Joe,<br><br><blockquote type="cite">continuous refactoring and syncing across 22+ repositories sounds like a nightmare, one that I would like to avoid.</blockquote>
You are right this is not easy.<br><br>
However I have several reasons to do that:<br>
The hardest part is to bring basic stuff in sync across all projects (That's what we are doing now). Later we'll work directly with oslo lib and just sync changes from it.<br><br>
We could introduce a standalone library to avoid the need to sync oslo code across all projects but it brings additional problems:<br><br>
1) We would have to maintain rationale versioning and backwards compatibility of this library. If we start library from scratch we'll have to add/change lots of stuff before we'll reach some stability period.<br><br>
2)Another option would be to follow waterfall process and create a solid library interface before including it to all client projects. However such approach this period can take unknown amount of time and can be easily failed during integration stage cause
requirements change or some other reason.<br><br>
Please let me know what you think.<br><br>
Best Regards,<br>
Alexei<br><br>
On 01/16/2014 08:16 PM, Joe Gordon wrote:<br></div><blockquote cite="mid:CAHXdxOf-+kUbb+yG7YLpjSc95_hvs+Kg0dwqxN=9869FpVyjuQ@mail.gmail.com" type="cite"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jan 16, 2014 at 12:07 PM, Alexei Kornienko <span dir="ltr">
<<a moz-do-not-send="true" href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div><div class="h5"><div>On 01/16/2014 06:15 PM, Jesse Noller wrote:<br></div><blockquote type="cite"><br><div><div>On Jan 16, 2014, at 9:54 AM, Alexei Kornienko <<a moz-do-not-send="true" href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>> wrote:</div><br><blockquote type="cite"><div bgcolor="#FFFFFF" text="#000000"><div>On 01/16/2014 05:25 PM, Jesse Noller wrote:<br></div><blockquote type="cite"><br><div><div>On Jan 16, 2014, at 9:07 AM, Joe Gordon <<a moz-do-not-send="true" href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jan 16, 2014 at 9:45 AM, Jesse Noller <span dir="ltr">
<<a moz-do-not-send="true" href="mailto:jesse.noller@rackspace.com" target="_blank">jesse.noller@rackspace.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div style="word-wrap:break-word"><br><div><div><div><div>On Jan 16, 2014, at 5:53 AM, Chmouel Boudjnah <<a moz-do-not-send="true" href="mailto:chmouel@enovance.com" target="_blank">chmouel@enovance.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 16, 2014 at 12:38 PM, Chris Jones <span dir="ltr">
<<a moz-do-not-send="true" href="mailto:cmsj@tenshu.net" target="_blank">cmsj@tenshu.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
Once a common library is in place, is there any intention to (or resistance against) collapsing the clients into a single project or even a single command (a la busybox)?</blockquote></div><br><br></div><div class="gmail_extra">that's what openstackclient is here for <a moz-do-not-send="true" href="https://github.com/openstack/python-openstackclient" target="_blank">
https://github.com/openstack/python-openstackclient</a><br></div></div></blockquote><div><br></div></div></div><div>After speaking with people working on OSC and looking at the code base in depth; I don’t think this addresses what Chris is implying: OSC wraps the individual CLIs built by each project today, instead of the inverse: a common backend that the individual
CLIs can wrap - the latter is an important distinction as currently, building a single binary install of OSC for say, Windows is difficult given the dependency tree incurred by each of the wrapped CLIs, difference in dependencies, structure, etc.</div><div><br></div><div>Also, wrapping a series of inconsistent back end Client classes / functions / methods means that the layer that presents a consistent user interface (OSC) to the user is made more complex juggling names/renames/commands/etc. </div><div><br></div><div>In the inverted case of what we have today (single backend); as a developer of user interfaces (CLIs, Applications, Web apps (horizon)) you would be able to:</div><div><br></div><div><div>from openstack.common.api import Auth</div></div><div>from openstack.common.api import Compute</div><div>from openstack.common.util import cli_tools</div><div><br></div><div>my_cli = cli_tools.build(…)</div><div><br></div><div>def my_command(cli):</div><div><span style="white-space:pre-wrap"></span>compute = Compute(Auth(cli.tentant…, connect=True))</div><div><span style="white-space:pre-wrap"></span>compute.list_flavors()</div><div><br></div><div>This would mean that *even if the individual clients needed or wanted to keep their specific CLIs, they would be able to use a not “least common denominator” back end (each service can have a rich
<a moz-do-not-send="true" href="http://common.api.compute.py/" target="_blank">common.api.compute.py</a> or api.compute/client.py and extend where needed. However tools like horizon / openstackclient can choose not to leverage the “power user/operator/admin”
components and present a simplified user interface.</div><div><br></div><div>I’m working on a wiki page + blueprint to brainstorm how we could accomplish this based off of what work is in flight today (see doug’s linked blueprint) and sussing out a layout / API strawman for discussion. </div><div><br></div><div>Some of the additions that came out of this email threads and others:</div><div><br></div><div>1. Common backend should provide / offer caching utilities </div><div>2. Auth retries need to be handled by the auth object, and each sub-project delegates to the auth object to manage that.</div><div>3. Verified Mocks / Stubs / Fakes must be provided for proper unit testing </div></div></div></blockquote><div><br></div><div>I am happy to see this work being done, there is definitely a lot of work to be done on the clients.</div><div><br></div><div>This blueprint sounds like its still being fleshed out, so I am wondering what the value is of the current patches <a moz-do-not-send="true" href="https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z</a></div><div><br></div><div>Those patches mainly sync cliutils and apiutils from oslo into the assorted clients. But if this blueprint is about the python API and not the CLI (as that would be the openstack-pythonclient), why sync in apiutils?</div><div><br></div><div>Also does this need to go through oslo-incubator or can this start out as a library? Making this a library earlier on will reduce the number of patches needed to get 20+ repositories to use this.</div><div><br></div></div></div></div></blockquote><br></div><div>Alexei and others have at least started the first stage of a rollout - the blueprint(s) needs additional work, planning and discussion, but his work is a good first step (reduce the duplication of code) although I am worried that the libraries and APIs
/ namespaces will need to change if we continue these discussions which potentially means re-doing work.</div><div><br></div><div>If we take a step back, a rollout process might be:</div><div><br></div><div>1: Solidify the libraries / layout / naming conventions (blueprint)</div><div>2: Solidify the APIs exposed to consumers (blueprint)</div><div>3: Pick up on the common-client-library-2 work which is primarily a migration of common code into oslo today, into the structure defined by 1 & 2</div><div><br></div><div>So, I sort of agree: moving / collapsing code now might be premature. I do strongly agree it should stand on its own as a library rather than an oslo incubator however. We should start with a single, clean namespace / library rather than depending on oslo
directly.</div></blockquote>
Knowing usual openstack workflow I'm afraid that #1,#2 with a waterfall approach may take years to be complete.
<br>
And after they'll be approved it will become clear that this architecture is already outdated.<br>
We try to use iterative approach for clients refactoring.<br>
We started our work from removing code duplication because it already gives a direct positive effect on client projects.<br>
If you can show us better way of moving forward please help us by uploading patches on this topic.<br><br>
Talk is cheap. Show me the code. (c) Linus<br><br></div></blockquote><div><br></div><div>I don’t disagree and I’m pretty sure I and others have already thanked you for starting this but have expressed concerns about the API / layout - and many of the patches on that blueprint are still pending review due to those concerns. I’m not suggesting
we need to spend the next six years designing it, I’m arguing for some basic design. From the current blueprint it is unclear where things will live, where it will live and how does it benefit everyone.</div><div><br></div><div>So I’d like to work *with* you to expand the blueprints to address some basic design / namespacing / issues and not jump straight into refactoring 22+ command line tools without a clear idea of the the desired layout and design.</div></div></blockquote></div></div>
The problem with defining design is that I don't know a convenient way of doing it cause UML, etherpad, blueprint's description are not sufficient to do this clearly.<br>
We try to use approach that I call "continuous refactoring" so as long as something can be improved we improve it.<br>
Our basic goal is to move towards SOLID principles and as long every new commit brings us one step closer to them I consider that overall design is improving.<br>
I've already posted here a draft idea of what we would like to see in the end:
<div class="im"><br></div></div></blockquote><div><br></div><div>continuous refactoring and syncing across 22+ repositories sounds like a nightmare, one that I would like to avoid.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="im"><pre><blockquote type="cite"><pre>1) Transport layer would handle all transport related stuff - HTTP, JSON
encoding, auth, caching, etc.
2) Model layer (Resource classes, BaseManager, etc.) will handle data
representation, validation
3) API layer will handle all project specific stuff - url mapping, etc.
(This will be imported to use client in other applications)
4) Cli level will handle all stuff related to cli mapping - argparse,
argcomplete, etc.</pre></blockquote></pre><br><br><br></div><blockquote type="cite"><div><br><blockquote type="cite"><div bgcolor="#FFFFFF" text="#000000"><blockquote type="cite"><div><br></div><div class="im"><div>jesse</div><div><br></div><br><br><fieldset></fieldset> <br><pre>_______________________________________________
OpenStack-dev mailing list
<a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></pre></div></blockquote><br></div><div class="im">_______________________________________________<br>
OpenStack-dev mailing list<br><a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br><a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br></div></blockquote></div><div class="im"><br><br><fieldset></fieldset> <br><pre>_______________________________________________
OpenStack-dev mailing list
<a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></pre></div></blockquote><br></div><br>
_______________________________________________<br>
OpenStack-dev mailing list<br><a moz-do-not-send="true" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br><a moz-do-not-send="true" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br><br></blockquote></div><br></div></div><br><fieldset class="mimeAttachmentHeader"></fieldset> <br><pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></pre></blockquote><br></div></div></span></body></html>