[openstack-dev] [TaskFlow] TaskFlow persistence

pn kk pnkk2016 at gmail.com
Wed Jan 27 17:50:44 UTC 2016


Hi,

Thanks for the responses. Putting it in a small example

def flow_factory(tmp):
    return lf.Flow('resume from backend example').add(
        TestTask(name='first', test=tmp),
        InterruptTask(name='boom'),
        TestTask(name='second', test="second task"))


class TestTask(task.Task):
    def __init__(self, name, provides=None, test, **kwargs):
        self.test=test
        super(TestTask, self).__init__(name, provides, **kwargs)
    def execute(self, *args, **kwargs):
        print('executing %s' % self)
        return 'ok'

class InterruptTask(task.Task):
    def execute(self, *args, **kwargs):
        # DO NOT TRY THIS AT HOME
        engine.suspend()

I was searching for a way in which I can reload the flow after crash
without passing the parameter "tmp" shown above
Looks like "load_from_factory" gives this provision.


engine = taskflow.engines.load_from_factory(flow_factory=flow_factory,
factory_kwargs={"tmp":"test_data"}, book=book, backend=backend)
engine.run()

Now it suspends after running interrupt task, I can now reload the flow
from the saved factory method without passing parameter again.
for flow_detail_2 in book:
    engine2 = taskflow.engines.load_from_detail(flow_detail_2,
backend=backend)

engine2.run()

Let me know if this is ok or is there a better approach to achieve this?

-Thanks


On Wed, Jan 27, 2016 at 12:03 AM, Joshua Harlow <harlowja at fastmail.com>
wrote:

> Hi there,
>
> Michał is correct, it should be saved.
>
> Do u have a small example of what u are trying to do because that will
> help determine if what u are doing will be saved or whether it will not be.
>
> Or even possibly explaining what is being done would be fine to (more
> data/info for me to reason about what should be stored in your case).
>
> Thanks,
>
> Josh
>
>
> Michał Dulko wrote:
>
>> On 01/26/2016 10:23 AM, pn kk wrote:
>>
>>> Hi,
>>>
>>> I use taskflow for job management and now trying to persist the state
>>> of flows/tasks in mysql to recover incase of process crashes.
>>>
>>> I could see the state and the task results stored in the database.
>>>
>>> Now I am looking for some way to store the input parameters of the tasks.
>>>
>>> Please share your inputs to achieve this.
>>>
>>> -Thanks
>>>
>>> I've played with that some time ago and if I recall correctly input
>> parameters should be available in the flow's storage, which means these
>> are also saved to the DB. Take a look on resume_workflows method on my
>> old PoC [1] (hopefully TaskFlow haven't changed much since then).
>>
>> [1] https://review.openstack.org/#/c/152200/4/cinder/scheduler/manager.py
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160127/4c918243/attachment.html>


More information about the OpenStack-dev mailing list