Open Stack

On Thu, Sep 1, 2016 at 4:53 PM, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
> On 1 September 2016 at 06:52, Ken Giusti <kgiusti at gmail.com> wrote:
>>
>> On Wed, Aug 31, 2016 at 3:30 PM, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>
>> > I have opinions about other patterns we could use, but I don't want to
>> > push
>>
>> > my solutions here, I want to see if this is really as much of a problem
>> > as
>> > it looks and if people concur with my summary above.  However, the right
>> > approach is most definitely to create a new and more fitting set of oslo
>> > interfaces for communication patterns, and then to encourage people to
>> > move
>> > to the new ones from the old.  (Whether RabbitMQ is involved is neither
>> > here
>> > nor there, as this is really a question of Oslo APIs, not their
>> > implementation.)
>> >
>>
>> Hmmmmmm... maybe.   Message bus technology is varied, and so is it's
>> behavior.  There are brokerless, point-to-point backends supported by
>> oslo.messaging [1],[2] which will exhibit different
>> capabilities/behaviors from the traditional broker-based
>> store-and-forward backend (e.g. message acking end-to-end vs to the
>> intermediary).
>
>
> The important thing is that you shouldn't have to look behind the curtain.
> We can offer APIs that are driven by the implementation (designed for test,
> and trivial to implement correctly given handy open source projects we know
> and trust) and the choice of design will therefore be dependent on the
> backend mechanisms we consider for use to implement the API.  APIs are
> always a point of negotiation between what the caller needs and what can be
> implemented in a practical amount of time.  But *I do not care* whether
> you're using rabbits or carrier pigeons just so long as what you have
> documented that the API promises me is actually true.  I *do not expect* to
> have to read RabbitMQ ior AMQP documentation to work out what behaviour I
> should expect for my messaging.  And its behaviour should be consistent if I
> have a choice of messaging backends.
>

And I agree totally - this is the way it _should_ be.  And to get
there we do have to address the ambiguities in the existing API, as
well as extend it so applications can explicitly state their service
needs.

My point is that the API also has to be _backend_ agnostic.  That
really hasn't been the priority it should be IMHO.  The current API as
it stands leaks too much of the backend behavior past the API.

For example here's where we are with the current API: a majority of
deployments are broker based - applications using oslo.messaging  have
come to rely _indirectly_ on the behavioral side effects of using a
broker backend.  In fact RabbitMQ's operational characteristics have
become the de-facto "correct" behavior.  Any other backend that
doesn't exhibit exactly the same behavior as RabbitMQ is considered
buggy.   Consider qpidd for example - simple differences in default
queue lifecycle and default flow control settings resulted in
messaging behavior different from RabbitMQ.  These were largely
considered bugs in qpidd.  I think this played a large part in the
lack of adoption of qpidd.

And qpidd is the same type of messaging backend as rabbitmq - a
broker.  Imagine what deployer's are going to hit when they attempt to
use a completely different technology - non-brokered backends like
Zeromq or message routing.

Improving the API as you describe will go a long way to solving this
situation.  And believe me I agree 100% that this API work needs to be
done.

But the API redesign should be done in a backend-agnostic manner.  We
(the oslo.messaging devs) have to ensure that _required_ API features
cannot be tied to any one backend implementation.  For example things
like notification pools are trivial to support for broker backends,
but hard/impossible for point to point distributed technologies.  It
must be clear to the application devs that using those optional
features that cannot be effectively implemented for a given backend
basically forces the deployer's hand.

My point is yes we need to improve that API but it should be done in a
backend agnostic way. There are currently features/behaviors that
essentially require a broker back end.  We should avoid making such
features mandatory elements of the API and ensure that the API users
are well aware of the consequences for deployers when using such
features.


>> All the more reason to have explicit delivery guarantees and well
>> understood failure scenarios defined by the API.
>
> And on this point we totally agree.
>
> I think the point of an API is to subdivide who carries which
> responsibilities - the caller for handling exceptional cases and the
> implementer for having predictable behaviour.  Documentation is the means of
> agreement.
>
> Sorry to state basic good practice - I'm sure we do all accept that this is
> good behaviour - but with a component that's this central to what we do and
> so frequently used by so many people I think it's worth reiterating.
> --
> Ian.
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Ken Giusti  (kgiusti at gmail.com)