Open Stack

Hi Nikhil,

On Fri, Oct 18, 2013 at 7:55 AM, Nikhil Komawar
<nikhil.komawar at rackspace.com> wrote:
> Hi all,
>
> There seem to be varying ideas about the worker interface design for our
> blueprint async-glance-workers. Fei and Zhi already have posted detailed
> comments on https://review.openstack.org/#/c/46117/21 (PS 21). Venkatesh,
> Alex and I have worked on it at varying periods of times and brainstormed
> different ideas as well. I know Zhi is also, very interested in the design
> and making it as configurable as possible.
>
>
>
> Recently, a patch has been uploaded to
> https://review.openstack.org/#/c/46117 giving a direction to the interface
> we'r hoping to achieve. The following are some of the high level pros and
> cons to this approach which we would like to discuss in the upcoming sync
> up(s) on #openstack-glance :-
>
>
>
> Pros:
>
> Establishes clear distinction between an executor and a script (or script
> package)
>
> eg. executor can be of type eventlet or rpc and script  (or script package)
> can be of type import/export/clone
>
> Establishes a good understanding of the contract between Glance and script
> packages
> Gives the deployer the ability to keep their script packages as simple as
> possible or even add complexity to them if needed without marrying such
> script packages with Glance code.
> A standard way of invoking asynchronous tasks via Glance.
>
> Cons:
>
> Not all script packages would be supported by Glance.
> There would be no modules which include classes like TaskImportScript,
> TaskExportScript within Glance domain logic- if they are needed for some
> sort of inheritance. However, the script packages themselves can have them
> and be as extensible as it gets.
>
> (Trying to use the word "script packages" as I'm trying to help understand
> that they would even be configurable using stevedore and be something
> similar to the example package given here. If using stevedore, they would
> have their own setup file and namespace defined.)
>
>
>
> All the script_packages would be having a standard module say named as
> "main" - which Glance task executor calls and initiates the execution.
> ("main" can be analogous to the python script given in the above example or
> can even be a simple script which runs the task)  Glance provides access to
> it's code on contractual basis - viz. the params that would be passed in are
> defined at the executor level. For example - in case of import we seem to
> need db_api, store_api, notifier_api, task_id, req.context. These can be
> used to invoke calls from within Glance modules like sqlalchemy/api or
> notifier/api etc. From this point on, it's responsibility of the script to
> achieve the result it needs. This design seem to fit in the requirements
> which many people have posted in the review comments without adding too many
> configuration options within glance; keeping it maintainable as well.
>
>
>
> Another idea about the interface design which Zhi layed-out is linked below.
> I have browsed through it however, am not completely familiar with the
> design's end goal. Would like to do a differential analysis sometime soon.
>
> http://paste.openstack.org/show/48644/
>
>
>
> Your opinions are very welcome!
>
>
>
> If you prefer, please try to sync up with us on the #openstack-glance
> channel while we try to fit the design as per the use cases.
>
> Thanks,
>
> -Nikhil
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Thanks for your efforts to this useful part! I know your(s) idea is
worked out by varying periods of times and brainstormed.

My idea can cover follow three key parts at least, those are not
exists currently, and it not only making task executor as configurable
as possible for end user, but also effect internal implementation.

1. As a plugin structure, the task executor should can tell glance
what task type(s) it can support. currently Glance only allow end user
request a task which type must in a fixed built-in list, it's an
unacceptable limitation to end user and task executor plugin developer
IMO. Actually this limitation cause this plugin structure can not be
real plugin-able. (refer:
https://wiki.openstack.org/wiki/Glance-tasks-api#List_Tasks "type"
field). In my draft idea (http://paste.openstack.org/show/48644/),

2. The validation on the input data for task execution should be
handled by particular task executor but not by a single, common (as
common s possible) place, since 1st, it has limited for particular
executor plugin implementation. we don't know all executor's
validation logic in advance, so this will not work for a real plugin
structure (because an executor plugin can be developed after our
Glance release by a vender/ISV, the release only has built-in
validation logic). 2nd, that built-in validation implement will be
ugly, it will using a lot of non-OOP style like if-elseif-else
checking because it need do different validation for different
built-in task type (current we have 3: 'import', 'export, 'clone', but
how about in future?).

3. The key part it that, as I mentioned in my comments at
https://review.openstack.org/#/c/46117/22/glance/domain/__init__.py ,
I consider organize executor plugin by function-level angle is better
then this execution-approach angle, that like following [example]. the
reason to me is that under this structure (PS22 current
implementation) we can not implement the executor in future like using
taskflow (we know it can support more worth function like resuming
task flow when service daemon restart), since the script can not tell
task executor more information to help it construct taskflow-task
objects and make a pattern-flow, the executor can only call
script.execute() directly and don't know any other details, so it's
really not enough. For my opinion, 1st, it can maximize responsibility
separation: it retain base class which organized by execute-approach,
like TaskEventletExecutor, the other hand it allow particular executor
sub-class like XXXImportXXXExecutor implement own execution logic. pls
refer my #48644 paste if you like. 2nd, it can reduce the complexity,
it only have one layer, but yours need two (executor and script/script
package).

[example]
XXXImportXXXExecutor, XXXExportXXXExecutor, XXXIndexingXXXExecutor
but
EventletTaskExecutor, RPCTaskExecutor,

In addition, I also consider Task object should be create by
TaskExecutor object but a fixed, common single implementation. if you
like pls refer my comments at Line #287 in
https://review.openstack.org/#/c/46117/11/glance/domain/__init__.py .

btw, the code in http://paste.openstack.org/show/48644/ is just a very
draft snippet to help show my idea, so it can't work and has some
little _wrong_ to my idea presentation, so if you interested in it, we
can talk it in irc for details, welcome. @nikhil

thanks,
zhiyan