Open Stack

Thanks, that helps explain those.

Let's see where the conditional evaluation logic goes. Likely it won't be python conditions directly.

I think though we should be able to work with other condition logic. I started https://review.openstack.org/#/c/87417/ today, hopefully can flush it out more in the days to come.

The differences I can see so far are around how a taskflow engine activates this conditional (and what results when the switch chooses a path). In taskflow the whole workflow is analyzed before execution (and translated into a directed graph) as to what is provided and what is required by each task in the workflow. Conditionals change this since its not known ahead of time which path will be selected. For now (in the above review) I am making it so that each path that could be switched to will have to have the same requirements and the same outputs (so that the analysis logic still works correctly). I'm thinking that a switch 'task' will return which choice it made, then this will affect the further path that will be followed (basically all other path choices will be 'abandoned'). This fits pretty well I think into how typically this is done in dataflow-like way and won't affect the state-transitions or ability to resume and such (btw found some neat papers at [1],[2] that show some past history that I didn't know about).

Of course I'm trying to make the above not be its own micro-language as much as possible (a switch object starts to act like one, sadly).

Comments welcome. Code welcome even more :-P

[1] http://www.cs.cmu.edu/~aldrich/papers/onward2009-concurrency.pdf
[2] http://www.cs.ucf.edu/~dcm/Teaching/COT4810-Spring2011/Literature/DataFlowProgrammingLanguages.pdf

From: Kirill Izotov <enykeev at stackstorm.com<mailto:enykeev at stackstorm.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, April 14, 2014 at 8:31 PM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Mistral][TaskFlow] Mistral-TaskFlow Summary

They are all parts of conditional transitions: every task should have a number of possible transitions; each transition consist of a reference to the task we want to transit to and the condition that should evaluate to true for transition to start.

At that point, I'd say that it perfectly fine for TaskFlow to evaluate python conditions rather than implementing YAQL, though there should be a place for us to pass the condition evaluation logic we are using.

--
Kirill Izotov


вторник, 15 апреля 2014 г. в 8:02, Joshua Harlow написал:

Can we describe exactly what references, direct transition, expression evaluation are doing in #2.

Expression evaluation especially seems to be an odd one, what's wrong with pythons expression evaluation? I can't quite see why that would/should exist in taskflow.

I can see it being implemented in mistral, where mistral converts whatever DSL it wants into taskflow primitives and then taskflow runs the code; this decoupling ensures that taskflow does not force a DSL on people that want to use taskflow as a python library (this kind of restriction imho isn't acceptable for a library to do,  and limits taskflows own usage and integration).

Thanks,

Josh

From: Dmitri Zimine <dz at stackstorm.com<mailto:dz at stackstorm.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, April 11, 2014 at 9:55 AM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: [openstack-dev] [Mistral][TaskFlow] Mistral-TaskFlow Summary

We prototyped Mistral / TaskFlow integration and have a follow-up discussions.

SUMMARY: Mistral (Workflow Service) can embed TaskFlow as a workflow library, with some required modifications to function resliently as a service, and for smooth integration. However, the TaskFlow flow controls are insufficient for Mistral use cases.

Details discussed on other thirds.
The prototype scope - [0<https://etherpad.openstack.org/p/mistral-taskflow-prototype>]; code and discussion - [1<https://github.com/enykeev/mistral/pull/1>] and techical highlights - [2<http://lists.openstack.org/pipermail/openstack-dev/2014-April/032461.html>].

DETAILS:

1) Embedding TaskFlow inside Mistral:
* Required: make the engine "lazy" [3<http://lists.openstack.org/pipermail/openstack-dev/2014-March/031134.html>], [4<http://paste.openstack.org/show/75389/>].This is required to support long-running delegates and not loose tasks when engine manager process restarts.

* Persistence: need clarity how to replace or mix-in TaskFlow persistence with Mistral persistence. Renat is taking a look.

* Declaring Flows in YAML DSL: done for simplest flow. Need to prototype for data flow. Rich flow controls are missing in TaskFlow for a representative prototype.

* ActionRunners vs Taskflow Workers - not prototyped. Not a risk: both Mistral and TaskFlow implementations work. But we shall resolve the overlap.

* Ignored for now - unlikely any risks: Keystone integration, Mistral event scheduler, Mistral declarative services and action definition.

2) TaskFlow library features
* Must: flow control - conditional transitions, references, expression evaluation, to express real-life workflows [5<https://github.com/dzimine/mistral-workflows/tree/add-usecases>]. The required flow control primitives are 1) repeater 2) flow in flow 3) direct transition 4) conditional transition 5) multiple data. TaskFlow has 1) and 2), need to add 3/4/5.

* Other details and smaller requests are in the discussion [1<https://github.com/enykeev/mistral/pull/1>]

3) Next Steps proposed:
* Mistal team: summarize the requirements discussed and agreed on [2<http://lists.openstack.org/pipermail/openstack-dev/2014-April/032461.html>] and [3<http://lists.openstack.org/pipermail/openstack-dev/2014-March/031134.html>]
* Mistral team: code sample (tests?) on how Mistral would like to consume TaskFlow lazy engine
* Taskflow team: Provide a design for alternative TaskExecutor approach (prototypes, boxes, arrows, crayons :))
* Decide on lazy engine
* Move the discussion on other elements on integration.

References:
[0] The scope of the prototype: https://etherpad.openstack.org/p/mistral-taskflow-prototype
[1] Prototype code and discussion https://github.com/enykeev/mistral/pull/1
[2] Techical summary http://lists.openstack.org/pipermail/openstack-dev/2014-April/032461.html
[3] Email discussion on TaskFlow lazy eninge http://lists.openstack.org/pipermail/openstack-dev/2014-March/031134.html
[4] IRC discussion Mistral/Taskflow http://paste.openstack.org/show/75389/
[5] Use cases https://github.com/dzimine/mistral-workflows/tree/add-usecases
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140415/67b5bc27/attachment.html>