<div dir="ltr">Hi Joshua,<div><br></div><div>Thanks for the response.</div><div><br></div><div>As you mentioned, I went through below which has taskflow as celery front end:</div><div>
<p class=""><span class=""> <span class=""><a href="https://etherpad.openstack.org/p/TaskFlowWorkerBasedEngine">https://etherpad.openstack.org/p/TaskFlowWorkerBasedEngine</a></span></span></p><p class=""><span class="">
</span></p><p class=""><span class=""> <a href="https://blueprints.launchpad.net/taskflow/+spec/distributed-celery"><span class="">https://blueprints.launchpad.net/taskflow/+spec/distributed-celery</span></a></span></p></div><div><br></div><div>I guess if we have a job = task1 + task2 ; if we execute them through a taskflow parallel pattern, they will be executed in parallel.</div><div><br></div><div>Just curious to know that since this threading is built on eventlets/green threads/python threads they will be affected by GIL and we may not utilize multi-core capability of our systems.</div><div><br></div><div>It seems in celery, we can have these tasks run as multiple processes so that they are not affected by GIL.</div><div><br></div><div>Please correct me if my understanding is wrong ??</div><div><br></div><div>Thanks</div><div>Eswar</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Jan 9, 2016 at 3:06 AM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@fastmail.com" target="_blank">harlowja@fastmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I also updated <a href="https://wiki.openstack.org/wiki/DistributedTaskManagement" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/DistributedTaskManagement</a> to denote that said wiki is no longer active (it was an attempt to back a taskflow engine[1] with celery); although if u are interested in continuing down this path feel free.<br>
<br>
Hopefully that clears up some 'confusion' around that wiki.<br>
<br>
[1] <a href="http://docs.openstack.org/developer/taskflow/engines.html" rel="noreferrer" target="_blank">http://docs.openstack.org/developer/taskflow/engines.html</a><div class="HOEnZb"><div class="h5"><br>
<br>
Joshua Harlow wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
So actually they are quite different, (although similar at some level),<br>
<br>
Given that celery isn't really a replacement for taskflow although one<br>
could say, from what I've heard from others, that taskflow is a<br>
super-set of what celery is so taskflow likely can replace parts of<br>
celery (but not vice-versa).<br>
<br>
Feel free to jump on #openstack-state-management IRC channel if u want<br>
to chat in person more about why (it gets into details that might just<br>
be easier to explain in person).<br>
<br>
ESWAR RAO wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi All,<br>
<br>
Please let me know whether celery is replacement for taskflow.<br>
<br>
As per my understanding, task-flow can break jobs into tasks and execute<br>
them.<br>
<br>
>From celery wiki, it also does almost similar behaviour.<br>
<br>
I guess in most of openstack components taskflow is widely used.<br>
Any places where its being replaced with celery ??<br>
<br>
Celery: <a href="https://wiki.openstack.org/wiki/Celery" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/Celery</a><br>
Distributed: <a href="https://wiki.openstack.org/wiki/DistributedTaskManagement" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/DistributedTaskManagement</a><br>
TaskFlow: <a href="https://wiki.openstack.org/wiki/TaskFlow" rel="noreferrer" target="_blank">https://wiki.openstack.org/wiki/TaskFlow</a><br>
<br>
Thanks<br>
Eswar<br>
<br>
__________________________________________________________________________<br>
<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe:<br>
<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote>
<br>
_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
</blockquote>
<br>
_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
</div></div></blockquote></div><br></div>