Open Stack

Agree that the root cause is GIL.
I don't like the way we are struggling with pony threads (whatever you
may call eventlet) to stuff a whole bunch of work into _single_
process, which means we can only utilize one processor while we may
have several dozens of them.  It's era of many-core already!  That
doesn't make sense and eventually doesn't scale (think of a situation
when compute node have > 100 cores, which may to real in just a few
years).

Another service or fork new process for periodic task, they may solve
the problem for a while, but the ultimate solution is true
parallelism.  I'd like to take one more step forward: how about
working out a new framework for periodic tasks that can put each task
into its own *true* thread/process(worker?)?

On Tue, Nov 20, 2012 at 11:30 AM, Yun Mao <yunmao at gmail.com> wrote:
> It seems that the fundamental problem is the lack of true parallelism, due
> to eventlet and GIL, not lack of a new service. I personally feel like we
> have enough services already. :) For example, we might be able fork a new
> process and run all periodic tasks there. We'll probably hit some
> concurrency bugs in the beginning either way.
>
> Thanks,
>
> Yun
>
>
> On Mon, Nov 19, 2012 at 9:57 PM, Michael Still <michael.still at canonical.com>
> wrote:
>>
>> I don't much mind what we call it by the way.
>>
>> Josh raises a valid point in bug/1080921 -- nova-compute periodic tasks
>> block incoming API requests from being handled, which is probably
>> something that you'd never want to see happen in a production deployment.
>>
>> There's been some back and forth about this in terms of libvirt image
>> cache management, but it seems that many of the periodic tasks fall into
>> a category of "I want this to run, but I don't want to block servicing
>> API requests for it".
>>
>> What do people think about adding another nova service which runs beside
>> nova-compute and runs periodic tasks which don't need any sort of shared
>> global state with nova-compute? We could do it with a flag so that the
>> default remains to use nova-compute for these periodic tasks, but if you
>> "opt in" they're done with a nova-periodic instead.
>>
>> Thoughts?
>>
>> Michael
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Regards
Huang Zhiteng