Open Stack

On Mon, Jan 14, 2013 at 09:55:12PM +1100, Michael Still wrote:
> On Mon, Jan 14, 2013 at 9:44 PM, Daniel P. Berrange <berrange at redhat.com> wrote:
> 
> [snip]
> 
> > For example in the libvirt driver we use a threadpool for running the
> > native API calls to avoid blocking eventlet. This doc
> >
> >   doc/source/devref/threading.rst
> >
> > indicates that DB calls used to do the same thing, but apparently don't
> > anymore. That said, this may be solved with most of the DB calls being
> > removed due to nodb-compute blueprint. There could be other areas that
> > would benefit from a threadpool though
> 
> So the current big use case for this is in the libvirt driver. The
> image cache manager needs to do a lot of file IO, and it locks up the
> main thread while it does that. If your images are really large, then
> it locks the world up for a really long time. Now, the initial
> proposal for that was to put a heap of yields into the code that did
> the IO, but we didn't like that, so this code started out as a way of
> exploring how to solve the problem without yielding. It should be
> noted this all came about at the request of Yahoo! who had performance
> problems because of this IO load. In fact, I'm pretty sure that the
> image cache manager is the only "external capable" periodic task at
> the moment, but I am sure there could be more.

Looking at the code that is run for the image cache manager, I can't
help thinking this would easily be handled by a native thread without
introducing any concurrency/locking problems wrt the rest of the Nova
libvirt driver. So IMHO this doesn't really justify a new standalone
service.

> > If threadpools don't suffice, rather than introducing explicit new user
> > visible binaries, it may be preferrable to just have the main binary
> > fork() some dedicated processes for periodic tasks. Alternatively it
> > may actually be preferrable to fork() 'n' complete nova-compute services,
> > one for each logical CPU in the host to maximise scaling across the
> > entire service, not merely periodic tasks.
> 
> That's a reasonable plan, but its not incompatible with the current
> review. forking() at the start of nova-compute would be a relatively
> small change from the current implementation. However, I'd rather this
> discussion include concrete alternatives. So -- do people prefer that
> approach to another binary? I don't really mind if they do, but why?
> 
> > Or maybe nova-periodic is the right idea after all - I'm just not sure
> > based on the limit info presented thus far.
> 
> What other information would you like to see? This thread has
> contained the entire debate so far.

Well the information you provided above about the image cache manager
problems is the sort of thing I was interested in :-)

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|