Open Stack

On Wed, Nov 2, 2016 at 8:11 PM, Joshua Harlow <harlowja at fastmail.com> wrote:
> Hi folks,
>
> There was a bunch of chatter at the summit about how there are really two
> different types of (oslo) messaging usage that exist in openstack and how
> they need not be backed by the same solution type (rabbitmq, qpid,
> kafka...).
>
> For those that were not at the oslo sessions:
>
> https://wiki.openstack.org/wiki/Design_Summit/Ocata/Etherpads#Oslo
>
> The general gist was though that we need to make sure people really do know
> that there are two very different types of messaging usage in openstack and
> to ensure that operators (and developers) are picking the right backing
> technology for each type.
>
> So some questions naturally arise out of this.
>
> * Where are the best practices with regard to selection of the best backend
> type for rpc (and one for notifications); is this something oslo.messaging
> should work through (or can the docs team and operator group also help in
> making this)?
>
> * What are the tradeoffs in using the same (or different) technology for rpc
> and notifications?
>

I think the olso.messaging team should take the lead here and educate
users as to what the options are, and  how the two supported messaging
services (RPC and Notifications) differ with respect to backend
requirements.   These topics really should be part of the
oslo.messaging 'Theory of Operations' documentation that was discussed
during the Arch WG summit meeting.

Currently the biggest functional difference between the backends is
the support of store-and-forward (e.g. queueing) verses point-to-point
message transfer.  Oslo could at least explain the pros and cons of
each approach with respect to the RPC and Notification services so
that folks understand what the tradeoffs and advantages are in the
first place.

Furthermore the team should also document the functional differences
between the various choices of backends.  For instance it would be
useful to understand how the two supported point-to-point backends
(zmq and dispatch router) differ in both behavior and recommended
deployment.


> * Is it even possible for all oslo.messaging consuming projects to be able
> to choose 2 different backends, are consuming projects consuming the library
> correctly so that they can use 2 different backends?
>
> * Is devstack able to run with say kafka for notifications and rabbitmq for
> rpc (if not, is there any work item the oslo group can help with to make
> this possible) so that we can ensure and test that all projects can work
> correctly with appropriate (and possibly different) backends?
>
> * Any other messaging, arch-wg work that we (oslo or others) can help out
> with to make sure that projects (and operators) are using the right
> technology for the right use (and not just defaulting to RPC over rabbitmq
> because it exists, when in reality something else might be a better choice)?
>

Ultimately there should be recommendations for which backends are
optimal for a range of different deployment scenarios, but at this
point we really don't have enough data and experience with these
backends to create such recommendations.

> * More(?)
>
> Just wanted to get this conversation started, because afaik it's one that
> has not been widely circulated (and operators have been setting up rabbitmq
> in various HA and clustered and ... modes, when in reality thinking through
> what and how it is used may be more appropriate); this also applies to
> developers since some technical solutions in openstack seem to be created
> due to (in-part) rabbitmq shortcomings (cells v1 afaik was *in* part created
> due to scaling issues).
>
> -Josh
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Ken Giusti  (kgiusti at gmail.com)