Open Stack

Hi all,
 
There seem to be varying ideas about the worker interface design for our blueprint [https://blueprints.launchpad.net/openstack/?searchtext=async-glance-workers] async-glance-workers. Fei and Zhi already have posted detailed comments on [https://review.openstack.org/#/c/46117/21] https://review.openstack.org/#/c/46117/21 (PS 21). Venkatesh, Alex and I have worked on it at varying periods of times and brainstormed different ideas as well. I know Zhi is also, very interested in the design and making it as configurable as possible.
 
Recently, a patch has been uploaded to [https://review.openstack.org/#/c/46117] https://review.openstack.org/#/c/46117 giving a direction to the interface we'r hoping to achieve. The following are some of the high level pros and cons to this approach which we would like to discuss in the upcoming sync up(s) on #openstack-glance :-
 
Pros:
Establishes clear distinction between an executor and a script (or script package)
eg. executor can be of type eventlet or rpc and script  (or script package) can be of type import/export/clone
Establishes a good understanding of the contract between Glance and script packages
Gives the deployer the ability to keep their script packages as simple as possible or even add complexity to them if needed without marrying such script packages with Glance code.
A standard way of invoking asynchronous tasks via Glance.
Cons:
Not all script packages would be supported by Glance.
There would be no modules which include classes like TaskImportScript, TaskExportScript within Glance domain logic- if they are needed for some sort of inheritance. However, the script packages themselves can have them and be as extensible as it gets.
(Trying to use the word "script packages" as I'm trying to help understand that they would even be configurable using stevedore and be something similar to the example package given [https://github.com/dreamhost/stevedore/tree/master/stevedore/example] here. If using stevedore, they would have their own setup file and namespace defined.) 
 
All the script_packages would be having a standard module say named as "main" - which Glance task executor calls and initiates the execution. ("main" can be analogous to the python [https://github.com/dreamhost/stevedore/blob/master/stevedore/example/load_as_extension.py] script given in the above example or can even be a simple script which runs the task)  Glance provides access to it's code on contractual basis - viz. the params that would be passed in are defined at the executor level. For example - in case of import we seem to need db_api, store_api, notifier_api, task_id, req.context. These can be used to invoke calls from within Glance modules like sqlalchemy/api or notifier/api etc. From this point on, it's responsibility of the script to achieve the result it needs. This design seem to fit in the requirements which many people have posted in the review comments without adding too many configuration options within glance; keeping it maintainable as well.
 
Another idea about the interface design which Zhi layed-out is linked below. I have browsed through it however, am not completely familiar with the design's end goal. Would like to do a differential analysis sometime soon.
[http://paste.openstack.org/show/48644/] http://paste.openstack.org/show/48644/
 
Your opinions are very welcome!
 
If you prefer, please try to sync up with us on the #openstack-glance channel while we try to fit the design as per the use cases.
Thanks,
-Nikhil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131017/d9b5f032/attachment.html>