[openstack-dev] RabbitMQ Scaling

Chris Behrens cbehrens at codestud.com
Wed Nov 28 02:23:58 UTC 2012


On Nov 26, 2012, at 11:00 AM, Ray Pekowski <pekowski at gmail.com> wrote:

> Chris,
> 
> Thanks for the input.
> 
> On Mon, Nov 26, 2012 at 11:36 AM, Chris Behrens <cbehrens at codestud.com> wrote:
> This is what cells does, and I was planning on porting this to openstack-common for general RPC stuff, too.. so this is great.
> 
> If you were already planning on doing this, then isn't my work duplicate and I should just wait for yours?  Or should we compare notes and pick some merged version of our two solutions?  And if so, the "cells" code and the RPC code should be as similar as possible, right?

Keep going… Don't wait for me. :)   I hadn't started on anything yet for the openstack-common code.  The code should be pretty similar, yes.  I'd hope that nova-cells could use the code from openstack-common so there's not too much specific within the cells rpc communication driver itself.

It sounded like our basic idea was the same.  I'll drop you a note off-list.

>  
> However, I'd like to see the # of greenthreads for receiving responses be configurable…. definitely not locked to 1.
> 
> I'm open to discussing this, but it seems to me that if a service gets by with a single thread for receiving all requests, that the callers should be able to get by with a single thread for receiving all responses.  Perhaps there is something in your solution that could block the receiving thread.  Or there is something I have not thought of in my solution.  In any case, we do need more discussion.  Is this mailing list the place for further discussion or is there some other mechanism?

Ah, right.  Well, we do consume from the queue in a single thread, but then the message is actually worked on via a pool of greenthreads.  I guess that pool of greenthreads is really not needed for what amounts to just putting a response into an eventlet Queue… so nevermind!  A single thread would be fine.

>  
> 
> Might be a big enough change to how rpc.calls work to warrant a blueprint.
> 
> Sounds like a blueprint is in order.  Do you want me to open the blueprint?  And would a single blueprint be OK?  I've heard that it is common to open multiple blueprints for a single general idea.  For example, one for adding a reply queue ID, one for returning the msg_id to the caller, and one for offloading the callback to a single receiver (or small set of receivers).  And perhaps one for adding an option for choosing backward compatible RPC, since this new RPC model will clearly not be backward compatible with the dynamic queue/exchange model.  I'm all for a single blueprint, but just want to follow the process.

Go for it.  I think a single blueprint for this is fine.  I'm not sure it's really large enough to warrant breaking it up.  But others can speak up if they have a different opinion...

- Chris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20121127/abbe1280/attachment.html>


More information about the OpenStack-dev mailing list