[openstack-dev] [TaskFlow] TaskFlow persistence
Joshua Harlow
harlowja at fastmail.com
Wed Jan 27 18:59:34 UTC 2016
pn kk wrote:
> Hi,
>
> Thanks for the responses. Putting it in a small example
>
> def flow_factory(tmp):
> return lf.Flow('resume from backend example').add(
> TestTask(name='first', test=tmp),
> InterruptTask(name='boom'),
> TestTask(name='second', test="second task"))
>
>
> class TestTask(task.Task):
> def __init__(self, name, provides=None, test, **kwargs):
> self.test=test
> super(TestTask, self).__init__(name, provides, **kwargs)
> def execute(self, *args, **kwargs):
> print('executing %s' % self)
> return 'ok'
>
> class InterruptTask(task.Task):
> def execute(self, *args, **kwargs):
> # DO NOT TRY THIS AT HOME
> engine.suspend()
>
> I was searching for a way in which I can reload the flow after crash
> without passing the parameter "tmp" shown above
> Looks like "load_from_factory" gives this provision.
Thanks for the example, ya, this is one such way to do this that u have
found, there are a few other ways, but that is the main one that people
should be using.
>
>
> engine = taskflow.engines.load_from_factory(flow_factory=flow_factory,
> factory_kwargs={"tmp":"test_data"}, book=book, backend=backend)
> engine.run()
>
> Now it suspends after running interrupt task, I can now reload the flow
> from the saved factory method without passing parameter again.
> for flow_detail_2 in book:
> engine2 = taskflow.engines.load_from_detail(flow_detail_2,
> backend=backend)
>
> engine2.run()
>
> Let me know if this is ok or is there a better approach to achieve this?
There are a few other ways, but this one should be the currently
recommended one.
An idea, do u want to maybe update (submit a review to update) the docs,
if u didn't find this very easy to figure out so that others can more
easily figure it out. I'm sure that would be appreciated by all.
>
> -Thanks
>
>
> On Wed, Jan 27, 2016 at 12:03 AM, Joshua Harlow <harlowja at fastmail.com
> <mailto:harlowja at fastmail.com>> wrote:
>
> Hi there,
>
> Michał is correct, it should be saved.
>
> Do u have a small example of what u are trying to do because that
> will help determine if what u are doing will be saved or whether it
> will not be.
>
> Or even possibly explaining what is being done would be fine to
> (more data/info for me to reason about what should be stored in your
> case).
>
> Thanks,
>
> Josh
>
>
> Michał Dulko wrote:
>
> On 01/26/2016 10:23 AM, pn kk wrote:
>
> Hi,
>
> I use taskflow for job management and now trying to persist
> the state
> of flows/tasks in mysql to recover incase of process crashes.
>
> I could see the state and the task results stored in the
> database.
>
> Now I am looking for some way to store the input parameters
> of the tasks.
>
> Please share your inputs to achieve this.
>
> -Thanks
>
> I've played with that some time ago and if I recall correctly input
> parameters should be available in the flow's storage, which
> means these
> are also saved to the DB. Take a look on resume_workflows method
> on my
> old PoC [1] (hopefully TaskFlow haven't changed much since then).
>
> [1]
> https://review.openstack.org/#/c/152200/4/cinder/scheduler/manager.py
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list