Open Stack

On Thu, Jan 16, 2014 at 2:03 PM, Alexei Kornienko <
alexei.kornienko at gmail.com> wrote:

>  Hello Joe,
>
>
> continuous refactoring and syncing across 22+ repositories sounds like a
> nightmare, one that I would like to avoid.
>
> You are right this is not easy.
>
> However I have several reasons to do that:
> The hardest part is to bring basic stuff in sync across all projects
> (That's what we are doing now). Later we'll work directly with oslo lib and
> just sync changes from it.
>

My concern here is that is just extra churn for minimal value, assuming
there are plans to completely overhaul this in the near future.

Perhaps this blueprint is really two:

1) do short term work to reduplicate the common client functionality. This
would cover the current patches and involve oslo-incubator, and is limited
in scope and shouldn't require to many oslo sync iterations.

2) major overhaul of client libraries so they are all based off a common
base library. This would cover namespace changes, and possible a push to
move CLI into python-openstackclient


> We could introduce a standalone library to avoid the need to sync oslo
> code across all projects but it brings additional problems:
>
> 1) We would have to maintain rationale versioning and backwards
> compatibility of this library. If we start library from scratch we'll have
> to add/change lots of stuff before we'll reach some stability period.
>
> 2)Another option would be to follow waterfall process and create a solid
> library interface before including it to all client projects. However such
> approach this period can take unknown amount of time and can be easily
> failed during integration stage cause requirements change or some other
> reason.
>
> Please let me know what you think.
>
> Best Regards,
> Alexei
>
>
> On 01/16/2014 08:16 PM, Joe Gordon wrote:
>
>
>
>
> On Thu, Jan 16, 2014 at 12:07 PM, Alexei Kornienko <
> alexei.kornienko at gmail.com> wrote:
>
>>   On 01/16/2014 06:15 PM, Jesse Noller wrote:
>>
>>
>>  On Jan 16, 2014, at 9:54 AM, Alexei Kornienko <
>> alexei.kornienko at gmail.com> wrote:
>>
>>  On 01/16/2014 05:25 PM, Jesse Noller wrote:
>>
>>
>>  On Jan 16, 2014, at 9:07 AM, Joe Gordon <joe.gordon0 at gmail.com> wrote:
>>
>>
>>
>>
>> On Thu, Jan 16, 2014 at 9:45 AM, Jesse Noller <jesse.noller at rackspace.com
>> > wrote:
>>
>>>
>>>   On Jan 16, 2014, at 5:53 AM, Chmouel Boudjnah <chmouel at enovance.com>
>>> wrote:
>>>
>>>
>>> On Thu, Jan 16, 2014 at 12:38 PM, Chris Jones <cmsj at tenshu.net> wrote:
>>>
>>>> Once a common library is in place, is there any intention to (or
>>>> resistance against) collapsing the clients into a single project or even a
>>>> single command (a la busybox)?
>>>
>>>
>>>
>>>  that's what openstackclient is here for
>>> https://github.com/openstack/python-openstackclient
>>>
>>>
>>>   After speaking with people working on OSC and looking at the code
>>> base in depth; I don’t think this addresses what Chris is implying: OSC
>>> wraps the individual CLIs built by each project today, instead of the
>>> inverse: a common backend that the individual CLIs can wrap - the latter is
>>> an important distinction as currently, building a single binary install of
>>> OSC for say, Windows is difficult given the dependency tree incurred by
>>> each of the wrapped CLIs, difference in dependencies, structure, etc.
>>>
>>>  Also, wrapping a series of inconsistent back end Client classes /
>>> functions / methods means that the layer that presents a consistent user
>>> interface (OSC) to the user is made more complex juggling
>>> names/renames/commands/etc.
>>>
>>>  In the inverted case of what we have today (single backend); as a
>>> developer of user interfaces (CLIs, Applications, Web apps (horizon)) you
>>> would be able to:
>>>
>>>  from openstack.common.api import Auth
>>>  from openstack.common.api import Compute
>>> from openstack.common.util import cli_tools
>>>
>>>  my_cli = cli_tools.build(…)
>>>
>>>  def my_command(cli):
>>> compute = Compute(Auth(cli.tentant…, connect=True))
>>> compute.list_flavors()
>>>
>>>  This would mean that *even if the individual clients needed or wanted
>>> to keep their specific CLIs, they would be able to use a not “least common
>>> denominator” back end (each service can have a rich
>>> common.api.compute.py or api.compute/client.py and extend where needed.
>>> However tools like horizon / openstackclient can choose not to leverage the
>>> “power user/operator/admin” components and present a simplified user
>>> interface.
>>>
>>>  I’m working on a wiki page + blueprint to brainstorm how we could
>>> accomplish this based off of what work is in flight today (see doug’s
>>> linked blueprint) and sussing out a layout / API strawman for discussion.
>>>
>>>  Some of the additions that came out of this email threads and others:
>>>
>>>  1. Common backend should provide / offer caching utilities
>>> 2. Auth retries need to be handled by the auth object, and each
>>> sub-project delegates to the auth object to manage that.
>>> 3. Verified Mocks / Stubs / Fakes must be provided for proper unit
>>> testing
>>>
>>
>>  I am happy to see this work being done, there is definitely a lot of
>> work to be done on the clients.
>>
>>  This blueprint sounds like its still being fleshed out, so I am
>> wondering what the value is of the current patches
>> https://review.openstack.org/#/q/topic:bp/common-client-library-2,n,z
>>
>>  Those patches mainly sync cliutils and apiutils from oslo into the
>> assorted clients. But if this blueprint is about the python API and not the
>> CLI (as that would be the openstack-pythonclient), why sync in apiutils?
>>
>>  Also does this need to go through oslo-incubator or can this start out
>> as a library? Making this a library earlier on will reduce the number of
>> patches needed to get 20+ repositories to use this.
>>
>>
>>  Alexei and others have at least started the first stage of a rollout -
>> the blueprint(s) needs additional work, planning and discussion, but his
>> work is a good first step (reduce the duplication of code) although I am
>> worried that the libraries and APIs / namespaces will need to change if we
>> continue these discussions which potentially means re-doing work.
>>
>>  If we take a step back, a rollout process might be:
>>
>>  1: Solidify the libraries / layout / naming conventions (blueprint)
>> 2: Solidify the APIs exposed to consumers (blueprint)
>> 3: Pick up on the common-client-library-2 work which is primarily a
>> migration of common code into oslo today, into the structure defined by 1 &
>> 2
>>
>>  So, I sort of agree: moving / collapsing code now might be premature. I
>> do strongly agree it should stand on its own as a library rather than an
>> oslo incubator however. We should start with a single, clean namespace /
>> library rather than depending on oslo directly.
>>
>> Knowing usual openstack workflow I'm afraid that #1,#2 with a waterfall
>> approach may take years to be complete.
>> And after they'll be approved it will become clear that this architecture
>> is already outdated.
>> We try to use iterative approach for clients refactoring.
>> We started our work from removing code duplication because it already
>> gives a direct positive effect on client projects.
>> If you can show us better way of moving forward please help us by
>> uploading patches on this topic.
>>
>> Talk is cheap. Show me the code. (c) Linus
>>
>>
>>  I don’t disagree and I’m pretty sure I and others have already thanked
>> you for starting this but have expressed concerns about the API / layout -
>> and many of the patches on that blueprint are still pending review due to
>> those concerns. I’m not suggesting we need to spend the next six years
>> designing it, I’m arguing for some basic design. From the current blueprint
>> it is unclear where things will live, where it will live and how does it
>> benefit everyone.
>>
>>  So I’d like to work *with* you to expand the blueprints to address some
>> basic design / namespacing / issues and not jump straight into refactoring
>> 22+ command line tools without a clear idea of the the desired layout and
>> design.
>>
>>  The problem with defining design is that I don't know a convenient way
>> of doing it cause UML, etherpad, blueprint's description are not sufficient
>> to do this clearly.
>> We try to use approach that I call "continuous refactoring" so as long as
>> something can be improved we improve it.
>> Our basic goal is to move towards SOLID principles and as long every new
>> commit brings us one step closer to them I consider that overall design is
>> improving.
>> I've already posted here a draft idea of what we would like to see in the
>> end:
>>
>>
>  continuous refactoring and syncing across 22+ repositories sounds like a
> nightmare, one that I would like to avoid.
>
>
>
>>  1) Transport layer would handle all transport related stuff - HTTP, JSON
>> encoding, auth, caching, etc.
>> 2) Model layer (Resource classes, BaseManager, etc.) will handle data
>> representation, validation
>> 3) API layer will handle all project specific stuff - url mapping, etc.
>> (This will be imported to use client in other applications)
>> 4) Cli level will handle all stuff related to cli mapping - argparse,
>> argcomplete, etc.
>>
>>
>>
>>
>>
>>
>>  jesse
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>  _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> _______________________________________________
> OpenStack-dev mailing listOpenStack-dev at lists.openstack.orghttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140116/45006bfd/attachment.html>