<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Jan 16, 2014 at 2:03 PM, Alexei Kornienko <span dir="ltr"><<a href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>Hello Joe,<div><br>
<br>
<blockquote type="cite">continuous refactoring and syncing across
22+ repositories sounds like a nightmare, one that I would like
to avoid.</blockquote></div>
You are right this is not easy.<br>
<br>
However I have several reasons to do that:<br>
The hardest part is to bring basic stuff in sync across all
projects (That's what we are doing now). Later we'll work directly
with oslo lib and just sync changes from it.<br></div></div></blockquote><div><br></div><div>My concern here is that is just extra churn for minimal value, assuming there are plans to completely overhaul this in the near future.</div>
<div> </div><div>Perhaps this blueprint is really two:</div><div><br></div><div>1) do short term work to reduplicate the common client functionality. This would cover the current patches and involve oslo-incubator, and is limited in scope and shouldn't require to many oslo sync iterations.</div>
<div><br></div><div>2) major overhaul of client libraries so they are all based off a common base library. This would cover namespace changes, and possible a push to move CLI into python-openstackclient</div><div><br></div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div>
<br>
We could introduce a standalone library to avoid the need to sync
oslo code across all projects but it brings additional problems:<br>
<br>
1) We would have to maintain rationale versioning and backwards
compatibility of this library. If we start library from scratch
we'll have to add/change lots of stuff before we'll reach some
stability period.<br>
<br>
2)Another option would be to follow waterfall process and create a
solid library interface before including it to all client
projects. However such approach this period can take unknown
amount of time and can be easily failed during integration stage
cause requirements change or some other reason.<br>
<br>
Please let me know what you think.<br>
<br>
Best Regards,<br>
Alexei<div><div><br>
<br>
On 01/16/2014 08:16 PM, Joe Gordon wrote:<br>
</div></div></div><div><div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu, Jan 16, 2014 at 12:07 PM,
Alexei Kornienko <span dir="ltr"><<a href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<div>
<div>On 01/16/2014 06:15 PM, Jesse Noller wrote:<br>
</div>
<blockquote type="cite"> <br>
<div>
<div>On Jan 16, 2014, at 9:54 AM, Alexei
Kornienko <<a href="mailto:alexei.kornienko@gmail.com" target="_blank">alexei.kornienko@gmail.com</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000">
<div>On 01/16/2014 05:25 PM, Jesse Noller
wrote:<br>
</div>
<blockquote type="cite"> <br>
<div>
<div>On Jan 16, 2014, at 9:07 AM, Joe
Gordon <<a href="mailto:joe.gordon0@gmail.com" target="_blank">joe.gordon0@gmail.com</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Thu,
Jan 16, 2014 at 9:45 AM, Jesse
Noller <span dir="ltr"> <<a href="mailto:jesse.noller@rackspace.com" target="_blank">jesse.noller@rackspace.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div style="word-wrap:break-word"><br>
<div>
<div>
<div>
<div>On Jan 16, 2014,
at 5:53 AM, Chmouel
Boudjnah <<a href="mailto:chmouel@enovance.com" target="_blank">chmouel@enovance.com</a>>
wrote:</div>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Thu, Jan 16,
2014 at 12:38
PM, Chris
Jones <span dir="ltr">
<<a href="mailto:cmsj@tenshu.net" target="_blank">cmsj@tenshu.net</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
Once a common
library is in
place, is
there any
intention to
(or resistance
against)
collapsing the
clients into a
single project
or even a
single command
(a la
busybox)?</blockquote>
</div>
<br>
<br>
</div>
<div class="gmail_extra">that's
what
openstackclient
is here for <a href="https://github.com/openstack/python-openstackclient" target="_blank">
https://github.com/openstack/python-openstackclient</a><br>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
<div>After speaking with
people working on OSC
and looking at the code
base in depth; I don’t
think this addresses
what Chris is implying:
OSC wraps the individual
CLIs built by each
project today, instead
of the inverse: a common
backend that the
individual CLIs can wrap
- the latter is an
important distinction as
currently, building a
single binary install of
OSC for say, Windows is
difficult given the
dependency tree incurred
by each of the wrapped
CLIs, difference in
dependencies, structure,
etc.</div>
<div><br>
</div>
<div>Also, wrapping a
series of inconsistent
back end Client classes
/ functions / methods
means that the layer
that presents a
consistent user
interface (OSC) to the
user is made more
complex juggling
names/renames/commands/etc. </div>
<div><br>
</div>
<div>In the inverted case
of what we have today
(single backend); as a
developer of user
interfaces (CLIs,
Applications, Web apps
(horizon)) you would be
able to:</div>
<div><br>
</div>
<div>
<div>from
openstack.common.api
import Auth</div>
</div>
<div>from
openstack.common.api
import Compute</div>
<div>from
openstack.common.util
import cli_tools</div>
<div><br>
</div>
<div>my_cli =
cli_tools.build(…)</div>
<div><br>
</div>
<div>def my_command(cli):</div>
<div><span style="white-space:pre-wrap"></span>compute
=
Compute(Auth(cli.tentant…,
connect=True))</div>
<div><span style="white-space:pre-wrap"></span>compute.list_flavors()</div>
<div><br>
</div>
<div>This would mean that
*even if the individual
clients needed or wanted
to keep their specific
CLIs, they would be able
to use a not “least
common denominator” back
end (each service can
have a rich <a href="http://common.api.compute.py/" target="_blank">common.api.compute.py</a>
or api.compute/client.py
and extend where needed.
However tools like
horizon /
openstackclient can
choose not to leverage
the “power
user/operator/admin”
components and present a
simplified user
interface.</div>
<div><br>
</div>
<div>I’m working on a wiki
page + blueprint to
brainstorm how we could
accomplish this based
off of what work is in
flight today (see doug’s
linked blueprint) and
sussing out a layout /
API strawman for
discussion. </div>
<div><br>
</div>
<div>Some of the additions
that came out of this
email threads and
others:</div>
<div><br>
</div>
<div>1. Common backend
should provide / offer
caching utilities </div>
<div>2. Auth retries need
to be handled by the
auth object, and each
sub-project delegates to
the auth object to
manage that.</div>
<div>3. Verified Mocks /
Stubs / Fakes must be
provided for proper unit
testing </div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>I am happy to see this work
being done, there is
definitely a lot of work to be
done on the clients.</div>
<div><br>
</div>
<div>This blueprint sounds like
its still being fleshed out,
so I am wondering what the
value is of the current
patches <a href="https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z" target="_blank">https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z</a></div>
<div><br>
</div>
<div>Those patches mainly sync
cliutils and apiutils from
oslo into the assorted
clients. But if this blueprint
is about the python API and
not the CLI (as that would be
the openstack-pythonclient),
why sync in apiutils?</div>
<div><br>
</div>
<div>Also does this need to go
through oslo-incubator or can
this start out as a library?
Making this a library earlier
on will reduce the number of
patches needed to get 20+
repositories to use this.</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</div>
<div>Alexei and others have at least
started the first stage of a rollout -
the blueprint(s) needs additional work,
planning and discussion, but his work is
a good first step (reduce the
duplication of code) although I am
worried that the libraries and APIs /
namespaces will need to change if we
continue these discussions which
potentially means re-doing work.</div>
<div><br>
</div>
<div>If we take a step back, a rollout
process might be:</div>
<div><br>
</div>
<div>1: Solidify the libraries / layout /
naming conventions (blueprint)</div>
<div>2: Solidify the APIs exposed to
consumers (blueprint)</div>
<div>3: Pick up on the
common-client-library-2 work which is
primarily a migration of common code
into oslo today, into the structure
defined by 1 & 2</div>
<div><br>
</div>
<div>So, I sort of agree: moving /
collapsing code now might be premature.
I do strongly agree it should stand on
its own as a library rather than an oslo
incubator however. We should start with
a single, clean namespace / library
rather than depending on oslo directly.</div>
</blockquote>
Knowing usual openstack workflow I'm afraid
that #1,#2 with a waterfall approach may
take years to be complete. <br>
And after they'll be approved it will become
clear that this architecture is already
outdated.<br>
We try to use iterative approach for clients
refactoring.<br>
We started our work from removing code
duplication because it already gives a
direct positive effect on client projects.<br>
If you can show us better way of moving
forward please help us by uploading patches
on this topic.<br>
<br>
Talk is cheap. Show me the code. (c) Linus<br>
<br>
</div>
</blockquote>
<div><br>
</div>
<div>I don’t disagree and I’m pretty sure I and
others have already thanked you for starting
this but have expressed concerns about the API
/ layout - and many of the patches on that
blueprint are still pending review due to
those concerns. I’m not suggesting we need to
spend the next six years designing it, I’m
arguing for some basic design. From the
current blueprint it is unclear where things
will live, where it will live and how does it
benefit everyone.</div>
<div><br>
</div>
<div>So I’d like to work *with* you to expand
the blueprints to address some basic design /
namespacing / issues and not jump straight
into refactoring 22+ command line tools
without a clear idea of the the desired layout
and design.</div>
</div>
</blockquote>
</div>
</div>
The problem with defining design is that I don't know a
convenient way of doing it cause UML, etherpad,
blueprint's description are not sufficient to do this
clearly.<br>
We try to use approach that I call "continuous
refactoring" so as long as something can be improved we
improve it.<br>
Our basic goal is to move towards SOLID principles and
as long every new commit brings us one step closer to
them I consider that overall design is improving.<br>
I've already posted here a draft idea of what we would
like to see in the end:
<div><br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>continuous refactoring and syncing across 22+
repositories sounds like a nightmare, one that I would
like to avoid.</div>
<div><br>
</div>
<div>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<div>
<pre><blockquote type="cite"><pre>1) Transport layer would handle all transport related stuff - HTTP, JSON
encoding, auth, caching, etc.
2) Model layer (Resource classes, BaseManager, etc.) will handle data
representation, validation
3) API layer will handle all project specific stuff - url mapping, etc.
(This will be imported to use client in other applications)
4) Cli level will handle all stuff related to cli mapping - argparse,
argcomplete, etc.</pre></blockquote>
</pre>
<br>
<br>
<br>
</div>
<blockquote type="cite">
<div> <br>
<blockquote type="cite">
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<div><br>
</div>
<div>
<div>jesse</div>
<div><br>
</div>
<br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</div>
</blockquote>
<br>
</div>
<div>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div>
</blockquote>
</div>
<div> <br>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</div>
</blockquote>
<br>
</div>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
OpenStack-dev mailing list
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>