[openstack-dev] [all][massively distributed][architecture]Coordination between actions/WGs

Flavio Percoco flavio at redhat.com
Mon Sep 5 10:08:04 UTC 2016


On 02/09/16 10:56 -0700, Clint Byrum wrote:
>Excerpts from Ken Giusti's message of 2016-09-02 11:05:51 -0400:
>> On Thu, Sep 1, 2016 at 4:53 PM, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>> > On 1 September 2016 at 06:52, Ken Giusti <kgiusti at gmail.com> wrote:
>> >>
>> >> On Wed, Aug 31, 2016 at 3:30 PM, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>> >
>> >> > I have opinions about other patterns we could use, but I don't want to
>> >> > push
>> >>
>> >> > my solutions here, I want to see if this is really as much of a problem
>> >> > as
>> >> > it looks and if people concur with my summary above.  However, the right
>> >> > approach is most definitely to create a new and more fitting set of oslo
>> >> > interfaces for communication patterns, and then to encourage people to
>> >> > move
>> >> > to the new ones from the old.  (Whether RabbitMQ is involved is neither
>> >> > here
>> >> > nor there, as this is really a question of Oslo APIs, not their
>> >> > implementation.)
>> >> >
>> >>
>> >> Hmmmmmm... maybe.   Message bus technology is varied, and so is it's
>> >> behavior.  There are brokerless, point-to-point backends supported by
>> >> oslo.messaging [1],[2] which will exhibit different
>> >> capabilities/behaviors from the traditional broker-based
>> >> store-and-forward backend (e.g. message acking end-to-end vs to the
>> >> intermediary).
>> >
>> >
>> > The important thing is that you shouldn't have to look behind the curtain.
>> > We can offer APIs that are driven by the implementation (designed for test,
>> > and trivial to implement correctly given handy open source projects we know
>> > and trust) and the choice of design will therefore be dependent on the
>> > backend mechanisms we consider for use to implement the API.  APIs are
>> > always a point of negotiation between what the caller needs and what can be
>> > implemented in a practical amount of time.  But *I do not care* whether
>> > you're using rabbits or carrier pigeons just so long as what you have
>> > documented that the API promises me is actually true.  I *do not expect* to
>> > have to read RabbitMQ ior AMQP documentation to work out what behaviour I
>> > should expect for my messaging.  And its behaviour should be consistent if I
>> > have a choice of messaging backends.
>> >
>>
>> And I agree totally - this is the way it _should_ be.  And to get
>> there we do have to address the ambiguities in the existing API, as
>> well as extend it so applications can explicitly state their service
>> needs.
>>
>> My point is that the API also has to be _backend_ agnostic.  That
>> really hasn't been the priority it should be IMHO.  The current API as
>> it stands leaks too much of the backend behavior past the API.
>>
>> For example here's where we are with the current API: a majority of
>> deployments are broker based - applications using oslo.messaging  have
>> come to rely _indirectly_ on the behavioral side effects of using a
>> broker backend.  In fact RabbitMQ's operational characteristics have
>> become the de-facto "correct" behavior.  Any other backend that
>> doesn't exhibit exactly the same behavior as RabbitMQ is considered
>> buggy.   Consider qpidd for example - simple differences in default
>> queue lifecycle and default flow control settings resulted in
>> messaging behavior different from RabbitMQ.  These were largely
>> considered bugs in qpidd.  I think this played a large part in the
>> lack of adoption of qpidd.
>>
>> And qpidd is the same type of messaging backend as rabbitmq - a
>> broker.  Imagine what deployer's are going to hit when they attempt to
>> use a completely different technology - non-brokered backends like
>> Zeromq or message routing.
>>
>> Improving the API as you describe will go a long way to solving this
>> situation.  And believe me I agree 100% that this API work needs to be
>> done.
>>
>> But the API redesign should be done in a backend-agnostic manner.  We
>> (the oslo.messaging devs) have to ensure that _required_ API features
>> cannot be tied to any one backend implementation.  For example things
>> like notification pools are trivial to support for broker backends,
>> but hard/impossible for point to point distributed technologies.  It
>> must be clear to the application devs that using those optional
>> features that cannot be effectively implemented for a given backend
>> basically forces the deployer's hand.
>>
>> My point is yes we need to improve that API but it should be done in a
>> backend agnostic way. There are currently features/behaviors that
>> essentially require a broker back end.  We should avoid making such
>> features mandatory elements of the API and ensure that the API users
>> are well aware of the consequences for deployers when using such
>> features.
>>
>
>All of what you say is true.
>
>However, I want us to also consider the cost of being so modular at the
>RPC level.
>
>Yes it's nice that we have RabbitMQ and ZeroMQ as options, But do we
>actually need these options? Could we just migrate to ZeroMQ, or HTTP/2,
>gRPC, thrift, etc.? Then we could tell deployers "good news, you don't
>need that component anymore, we factored it out" rather than "hey look
>here, more deployment choices, good luck!"

Based on the last OPs midcycle's feedback, the answer to the above question
would probably be "just keep rabbitmq". Turns out that the complains about
Rabbitmq dropped significantely for whatever reason (Either it becaome more
stable or OPs just learned how to cope with Rabbits craziness).

We should probably start by asking ourselves who's really being bitten by the
messaging bus right now? Large (and please, let's not bikeshed on what a Large
Cloud is) Clouds? Small Clouds? New Clouds? Everyone? 

The we can start asking ourselves things like: Would a change of the
API/underlying technology help them? Why? How? What technology exactly and why?
What technology would make their lives simpler and why?

I'm not saying this research/work is not useful/important (in fact, I've been
advocating for it for almost 2 years now) but I do want us to be more careful
and certainly I don't think this change should be anything but transparent for
every deployment out there.

To answer this topic more directly. As much as being opinionated would help
driving focus and providing a better result here, I believe we are not there yet
and I also believe a backend agnostic API would be more benefitial to begin
with. We're not going to move 98% of the OpenStack deployments out there off of
rabbitmq just like that.

Hope I was clear enough :D
Flavio

>My point isn't to discount any of the options, or to argue against
>having a strong, opaque messaging API that allows us to replace the
>implementation underneath. My point is to say, we're trying to change
>_both_ things at once, the modules underneath, and the API, and that
>feels like it might be leading to a deadlock on both efforts.
>
>On a meta-point, this discussion has been excellent, and the idea to
>have a working group just for largely distributed efforts seems to
>fit well with this. But I think this is more fundamental, and belongs
>in a design conversation with like-minded individuals. I'd like to get
>these conversations started, and start work to move forward with actual
>improvements. If you want to do that as well, please review this:
>
>https://review.openstack.org/335141
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-- 
@flaper87
Flavio Percoco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 847 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160905/9a359d1f/attachment.pgp>


More information about the OpenStack-dev mailing list