Open Stack

On Wed, 2014-04-23 at 07:25 +0100, Mark McLoughlin wrote:
> On Tue, 2014-04-22 at 15:54 -0700, Devananda van der Veen wrote:
> > Hi!
> > 
> > When a project is using oslo.messaging, how can we change our default
> > rpc_thread_pool_size?
> > 
> > -----------------------
> > Background
> > 
> > Ironic has hit a bug where a flood of API requests can deplete the RPC
> > worker pool on the other end and cause things to break in very bad ways.
> > Apparently, nova-conductor hit something similar a while back too. There've
> > been a few long discussions on IRC about it, tracked partially here:
> >   https://bugs.launchpad.net/ironic/+bug/1308680
> > 
> > tldr; a way we can fix this is to set the rpc_thread_pool_size very small
> > (eg, 4) and keep our conductor.worker_pool size near its current value (eg,
> > 64). I'd like these to be the default option values, rather than require
> > every user to change the rpc_thread_pool_size in their local ironic.conf
> > file.
> > 
> > We're also about to switch from the RPC module in oslo-incubator to using
> > the oslo.messaging library.
> > 
> > Why are these related? Because it looks impossible for us to change the
> > default for this option from within Ironic, because the option is
> > registered when EventletExecutor is instantaited (rather than loaded).
> > 
> > https://github.com/openstack/oslo.messaging/blob/master/oslo/messaging/_executors/impl_eventlet.py#L76
> 
> It may have been possible for Ironic to set its own default before
> oslo.messaging, but it wouldn't have been recommended because there's no
> explicit API for doing so.
> 
> With oslo.messaging, we have a set_transport_defaults() which shows how
> we'd approach adding this capability.
> 
> The question comes down to whether this really is a situation where we
> need per-application defaults or just that the current defaults are
> screwed up. If the latter, I'd much rather just change the defaults.

History is always useful :)

Soren added the threadpool with a default size of 1024:

  https://code.launchpad.net/~soren/nova/rpc-threadpool/+merge/49896

Johannes changed it back to 64:

  https://review.openstack.org/6792

Mark.