Open Stack

Winson, 

While you're looking into this and working on the design, may be also think through other executor/engine communications.

We talked about executor communicating to engine over 3 channels (DB, REST, RabbitMQ) which I wasn't happy about ;) and put it off for some time. May be it can be rationalized as part of your design. 

DZ. 

On Feb 24, 2014, at 11:21 AM, W Chan <m4d.coder at gmail.com> wrote:

> Renat,
> 
> Regarding your comments on change https://review.openstack.org/#/c/75609/, I don't think the port to oslo.messaging is just a swap from pika to oslo.messaging.  OpenStack services as I understand is usually implemented as an RPC client/server over a messaging transport.  Sync vs async calls are done via the RPC client call and cast respectively.  The messaging transport is abstracted and concrete implementation is done via drivers/plugins.  So the architecture of the executor if ported to oslo.messaging needs to include a client, a server, and a transport.  The consumer (in this case the mistral engine) instantiates an instance of the client for the executor, makes the method call to handle task, the client then sends the request over the transport to the server.  The server picks up the request from the exchange and processes the request.  If cast (async), the client side returns immediately.  If call (sync), the client side waits for a response from the server over a reply_q (a unique queue for the session in the transport).  Also, oslo.messaging allows versioning in the message. Major version change indicates API contract changes.  Minor version indicates backend changes but with API compatibility.  
> 
> So, where I'm headed with this change...  I'm implementing the basic structure/scaffolding for the new executor service using oslo.messaging (default transport with rabbit).  Since the whole change will take a few rounds, I don't want to disrupt any changes that the team is making at the moment and so I'm building the structure separately.  I'm also adding versioning (v1) in the module structure to anticipate any versioning changes in the future.   I expect the change request will lead to some discussion as we are doing here.  I will migrate the core operations of the executor (handle_task, handle_task_error, do_task_action) to the server component when we agree on the architecture and switch the consumer (engine) to use the new RPC client for the executor instead of sending the message to the queue over pika.  Also, the launcher for ./mistral/cmd/task_executor.py will change as well in subsequent round.  An example launcher is here https://github.com/uhobawuhot/interceptor/blob/master/bin/interceptor-engine.  The interceptor project here is what I use to research how oslo.messaging works.  I hope this is clear. The blueprint only changes how the request and response are being transported.  It shouldn't change how the executor currently works.
> 
> Finally, can you clarify the difference between local vs scalable engine?  I personally do not prefer to explicitly name the engine scalable because this requirement should be in the engine by default and we do not need to explicitly state/separate that.  But if this is a roadblock for the change, I can put the scalable structure back in the change to move this forward.
> 
> Thanks.
> Winson
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140224/ce1507ff/attachment.html>