Open Stack

I am in the full agreement with the proposed approach (risked to copy below, let's see if email handles your diagrams): 

* Single Engine handles multiple executions asynchronously, in non-blocking way.
* Persistence access only from API and/or Engine, Engine writes, API reads. 
* Action runes don't talk to DB - it's very simple protocol and queue is perfectly fine. 
* This is scalable and as fault tolerant as underlying DB and Queue. Engine restart is no loss of info with right persistense; ActionRunner restart  is a lost of Action, which can fail for 100 other reasons thus expected, and with the right retry policy potentially recoverable. 

I'll inline minor points in the etherpad. 



             API     Engine    Queue    Database        Workers
              |       |   |     | |         |            |   |
    -------> +-+      |   |     | |         |            |   |
             |S|      |   |     | |         |            |   |
             |t| --> +-+  |     | |         |            |   |
             |a|     | | < - - - - - - - -> |            |   |
             |r|     | | - - - -> t - - - - - - - - - > +-+  |
             |t| <-- +-+  |     | |         |           | |  |
    <------- +-+      |   |     | |         |           | |  |
              |       |  +-+ <- r <- - - - - - - - - -  +-+  |
    -------> +-+      |  | | <- - - - - - > R            |   |
             |I|      |  | | - -> t - - - - - - - - - > +-+  |
             |n|      |  | | - -> t - - - - - - - - - - - > +-+
             |f|      |  +-+    | |         |           | | | |
             |o| <- - - - - - - - - - - - > |           | | | |
    <------- +-+      |  +-+ <- r <- - - - - - - - - - -+-+ | |
              |       |  | | <- - - - - - > R            |  | |
    -------> +-+      |  +-+    | |         |            |  | |
             |S| --> +-+  |     | |         |            |  | |
             |t|     | | <- - - - - - - - > |            |  | |
             |o| <-- +-+  |     | |         |            |  | |
             |p|      |  +-+ <- r < - - - - - - - - - - - - +-+
    <------- +-+      |  | | <- - - - - - > R            |   |
              |       |  |%|    | |         |            |   |
              |       |  +-+    | |         |            |   |
              |       |   |     | |         |            |   |


On Mar 31, 2014, at 4:09 AM, Kirill Izotov <enykeev at stackstorm.com> wrote:

> I have an idea regarding engine design i want to share with you. But first, it seems like we need a small overview of the current implementations.
> 
> I'm not sure how ML will react on a bunch of ASCII graphs, so here is an etherpad: https://etherpad.openstack.org/p/mistral-engine-overview-and-proposal
> 
> What do you think, guys, is this the way to go?
> 
> -- 
> Kirill Izotov
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140331/dfc0347e/attachment.html>