[openstack-dev] [Mistral] How Mistral handling long running delegate tasks

Renat Akhmerov rakhmerov at mirantis.com
Thu Apr 3 10:14:13 UTC 2014


On 03 Apr 2014, at 12:49, Kirill Izotov <enykeev at stackstorm.com> wrote:

> Actually, the idea to make the concept more expandable is exactly my point =) Mistral's Workbook is roughly the same as TaskFlow Flow and have nothing to do with TaskFlow LogBook. The only major difference in case of Workbook is that we have to store it after it was created using API or DSL. I very much like the idea to inherit form basic Flow and add serialization and additional metadata on top of it, but the concept of retrying within a Workbook\Flow is not exactly what we have in mind. Instead of repeating the Flow, we are repeating the Task (which has the same differences between Mistral and TaskFlow as Workbook has) and to use the trick you have proposed we would have to translate the Task into a Flow. How much sense does it make in respect to expandability and generality?

Just one clarification. Workbook is a wider abstraction. Along with workflow itself it includes action definitions, triggers (even though it’s still controversial even within the team itself). Potentially it may include something else. So having a workbook in TaskFlow with strictly defined behaviour is not a good decision.

>> Ya, I think u are hitting this issue, if a worker acks a message saying its 'working on it' then the worker dies, this is where a message queue ack model won't really help in solving, the engine will believe that the worker is working on it (since hey the worker acked it) but the engine will really have no idea that the worker is actually dead. Of course timeouts can be used to get around this but imho that’s really un-elegant when something like zookeeper (or similar) can be used to be notified of this worker death immediately; which means no timeouts are needed at all, and this separate process or engine can be notified of this death (and resolve it immediately). I don't believe this kind of 'liveness' connection is possible with rabbit (we also must think of other things besides rabbit, like qpid and such) but I might be wrong, I thought once a worker acks a message then whoever receives that ack will believe the work has started and will be finished someday in the future (aka, no connection that the work is actually in progress). Anyways, we can discuss this more since I think its a common confusion point :-)
> As far as i understand, what Renat is proposing here is not to acknowledge the message until it was successfully executed. I'm not sure how exactly RabbitMQ will react in that case, though you are right in the idea that we must investigate the other solutions more carefully.

Joshua, I think your understanding of how it works is not exactly correct (or my understanding actually :)). Let’s talk in IRC, we’ll explain it.

I’d suggest we talk in IRC on Thursday at 8.30 pm (Friday 10.30 in our timezone).

Renat Akhmerov
@ Mirantis Inc.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140403/6ecd6cfa/attachment.html>


More information about the OpenStack-dev mailing list