Open Stack

On Mon, Jan 14, 2013 at 10:39 PM, Mark McLoughlin <markmc at redhat.com> wrote:

> Or perhaps even simpler than native threads, have the periodic task
> spawn off a new process for the long-running operation and reap it on
> the next task iteration.

This is an interesting approach. It is however harder for the
developer adding a periodic task to implement, which means they're
more likely to get it wrong.

I find the simplicity of being able to just mark a periodic task as
being external very attractive. Its "no" additional code for the
person making something external, its a mechanism that developers are
already familiar with, and it keeps the code for the periodic element
with the rest of the code for that functionality. In the same way that
a developer doesn't have to worry about the internal mechanisms of how
periodic tasks get run, I would like to have something where the
majority of developers don't have to spend significant time
implementing a new mechanism for just their periodic task, and then
adding tests for that mechanism.

I can see a few approaches now:

 - we keep nova-periodic as a separate process: simpler to code,
harder to deploy

 - we move the existing nova-periodic code into a separate process
fork()ed by nova-compute on startup: separates out slow periodic tasks
from the main nova-compute, easier to deploy. One very slow periodic
task still blocks other less slow periodic tasks.

 - we change the periodic task running code to fork() a process every
time it needs to run an "external" periodic task, it then wait()s for
those processes later in its loop: means you can have as many slow
periodic tasks as you want without them blocking each other, makes the
periodic task running more complex and therefore more likely to have
bugs, easy to deploy, transparent to users.

What do people think?

We should note that we need the externalizable flag mostly for the
benefit of shared data structures and locking. The flag indicates that
the developer has ensured that she has no shared state with the inner
workings of nova-compute.

Michael