Open Stack

Possibly then this could work out by creating a specific class that is started by nova-compute as a 'helper' process where all of these periodic actions happen.

Using the multiprocessing library could be the way to tie these together (using a daemonized subprocess), but not using the queue to interchange anything (unless needed).

Then when nova-compute fires up, it can fire up its 'helper' process that does background tasks that nova-compute 'likes' but will not block nova-compute from running. This 'helper' process has the benefit that it will be killed when nova-compute is killed and/or stopped (due to the mentioned daemoinizing), they are somewhat tied together (and easier to find who created who), and there doesn't need to be YAS (yet another process/service) to remember to start.

An idea at least :-P

-josh

From: Yun Mao <yunmao at gmail.com<mailto:yunmao at gmail.com>>
Reply-To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, November 19, 2012 7:30 PM
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] A new nova service? nova-periodic?

It seems that the fundamental problem is the lack of true parallelism, due to eventlet and GIL, not lack of a new service. I personally feel like we have enough services already. :) For example, we might be able fork a new process and run all periodic tasks there. We'll probably hit some concurrency bugs in the beginning either way.

Thanks,

Yun


On Mon, Nov 19, 2012 at 9:57 PM, Michael Still <michael.still at canonical.com<mailto:michael.still at canonical.com>> wrote:
I don't much mind what we call it by the way.

Josh raises a valid point in bug/1080921 -- nova-compute periodic tasks
block incoming API requests from being handled, which is probably
something that you'd never want to see happen in a production deployment.

There's been some back and forth about this in terms of libvirt image
cache management, but it seems that many of the periodic tasks fall into
a category of "I want this to run, but I don't want to block servicing
API requests for it".

What do people think about adding another nova service which runs beside
nova-compute and runs periodic tasks which don't need any sort of shared
global state with nova-compute? We could do it with a flag so that the
default remains to use nova-compute for these periodic tasks, but if you
"opt in" they're done with a nova-periodic instead.

Thoughts?

Michael

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121119/1b39fd55/attachment.html>