Open Stack

Hi Joshua,

Thanks for the response.

As you mentioned, I went through below which has taskflow as celery front
end:

 https://etherpad.openstack.org/p/TaskFlowWorkerBasedEngine

 https://blueprints.launchpad.net/taskflow/+spec/distributed-celery

I guess if we have a job = task1 + task2 ; if we execute them through a
taskflow parallel pattern, they will be executed in parallel.

Just curious to know that since this threading is built on eventlets/green
threads/python threads they will be affected by GIL and we may not utilize
multi-core capability of our systems.

It seems in celery, we can have these tasks run as multiple processes so
that they are not affected by GIL.

Please correct me if my understanding is wrong ??

Thanks
Eswar


On Sat, Jan 9, 2016 at 3:06 AM, Joshua Harlow <harlowja at fastmail.com> wrote:

> I also updated https://wiki.openstack.org/wiki/DistributedTaskManagement
> to denote that said wiki is no longer active (it was an attempt to back a
> taskflow engine[1] with celery); although if u are interested in continuing
> down this path feel free.
>
> Hopefully that clears up some 'confusion' around that wiki.
>
> [1] http://docs.openstack.org/developer/taskflow/engines.html
>
>
> Joshua Harlow wrote:
>
>> So actually they are quite different, (although similar at some level),
>>
>> Given that celery isn't really a replacement for taskflow although one
>> could say, from what I've heard from others, that taskflow is a
>> super-set of what celery is so taskflow likely can replace parts of
>> celery (but not vice-versa).
>>
>> Feel free to jump on #openstack-state-management IRC channel if u want
>> to chat in person more about why (it gets into details that might just
>> be easier to explain in person).
>>
>> ESWAR RAO wrote:
>>
>>> Hi All,
>>>
>>> Please let me know whether celery is replacement for taskflow.
>>>
>>> As per my understanding, task-flow can break jobs into tasks and execute
>>> them.
>>>
>>> From celery wiki, it also does almost similar behaviour.
>>>
>>> I guess in most of openstack components taskflow is widely used.
>>> Any places where its being replaced with celery ??
>>>
>>> Celery: https://wiki.openstack.org/wiki/Celery
>>> Distributed: https://wiki.openstack.org/wiki/DistributedTaskManagement
>>> TaskFlow: https://wiki.openstack.org/wiki/TaskFlow
>>>
>>> Thanks
>>> Eswar
>>>
>>>
>>> __________________________________________________________________________
>>>
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> _______________________________________________
>> Mailing list:
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> Unsubscribe :
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160109/4fe4ccee/attachment.html>