Open Stack

So this is an interesting question.

Let me explain why I think exposing is_sync is actually relatively dangerous and breaks the task oriented model (at least taskflows view of it).

First let me explain a little bit of what happens in taskflow to create and execute things.

1. User X creates objects derived from task<https://github.com/openstack/taskflow/blob/master/taskflow/task.py#L31> base class that act as the primitives that execute and revert (nothing here specifies that they are synchronous or not, they just do some piece of work and return some results and have the ability to undo those pieces of work).
2. User X combines links tasks (and also patterns themselves) together using taskflows patterns<https://github.com/openstack/taskflow/tree/master/taskflow/patterns> (currently a small set, but could get bigger as needed) to form a larger set of combined work (A -> B -> C for example), these patterns support nesting so [A] in the example can itself expand into a something like (E,F,G tasks) and so on. Lets call the result of this linking the Y object. NOTE: At this point there is still not is_sync or is_async, since here we are just defining ordering constraints that should be enforced at runtime.
3. User gives Y object to engine<https://github.com/openstack/taskflow/tree/master/taskflow/engines> in taskflow, providing the engine persistence<https://github.com/openstack/taskflow/tree/master/taskflow/persistence> model/backend it wants to use (for storing intermediate results, for saving state changes) and tells the engine to execute(). At this point the engine will locate all tasks that have no dependencies on other tasks and start running them asynchronously (how this works depends on the engine type selected), at this point the tasks that can execute begin executing (and they have the potential to signal to others there current progress via the update_progress<https://github.com/openstack/taskflow/blob/master/taskflow/task.py#L78> method (engines support a concept of listeners<https://github.com/openstack/taskflow/tree/master/taskflow/listeners> that can be attached to engines to allow external entities to be informed of progress updates, state changes…). This process repeats until the workflow has completed or it fails (in which case revert() methods start to be called and each task is given a chance to undo whatever it has created, in the near future there will be aways to alter how this reversion happens with a concept of retry_controllers<https://review.openstack.org/#/c/71621/>).

TLDR: So in general u could say that all tasks are unaware that they are running async/sync and in the above model it is up to the engine type to determine how things are ran (since the engine is the controller that actually runs all tasks, making sure the the ordering constraints established in step #2 are retained).

To me this kind of disconnection (not allowing a task to specify it's async/sync) is useful and helps retain sanity. In a way it seems against the task model to have tasks provide this kind of information (leaky abstraction…) and at least in the taskflow view makes tasks have to much control over how they are executed (why should a task care?). It also completely alters the state diagram<https://wiki.openstack.org/wiki/File:Tf_task_state_diagram.png> that is persisted and used for resumption when this information is allowed to be specified by a task. What does it mean for a task to be RUNNING but have the task continues to run asynchronously? In a way isn't this already what the engine (at least in taskflow) is doing internally anyway (running everything asynchronously)? At least in taskflow all task are already running with executors<https://github.com/openstack/taskflow/blob/master/taskflow/engines/action_engine/executor.py> that return future/s that are waited upon (dependent tasks can not run until the future of a predecessor task has completed).

-Josh

From: Renat Akhmerov <rakhmerov at mirantis.com<mailto:rakhmerov at mirantis.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, March 14, 2014 at 5:09 AM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Mistral] Actions design BP

- is_sync() - consider using an attribute instead -  @mistral.async

Well, I had an idea that it may depend on how a particular action instance is parameterized (in other words, a dynamic thing rather than static property). It would just give more flexibility

- can we think of a way to unify sync and async actions from engine's standpoint? So that we don't special-case it in the engine?

To be precise, engine has no knowledge about it. Only executor does and it has to but the difference is pretty small. In case if action is sync it should just call the API method I mentioned above to deliver action result. When we finish our BPs related to oslo.messaging it’ll be working over it.

@ Joshua - does something similar exists in TaskFlow already?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140314/b7ceb3db/attachment.html>