Open Stack

On 07/07/2014 09:53 PM, Victoria Martínez de la Cruz wrote:
> During the last couple of weeks I've been working on adding support for
> AMQP 1.0 in Marconi transport. For that, I've based my research on
> current Marconi WSGI API, Apache Proton API [0] and Azure Service Bus
> docs [1].
>
> There are several things to decide here in order to take full profit of
> AMQP's performance while keeping things consistent.
>
> I'd like to know your opinions about the following.
>
>  > Marconi and AMQP operations mapping
>
> Marconi doesn't follow a strict queue semantic. It provides some more
> operations that are not supported in other queues implementations, as
> messages listing, message/s retrieval by id and message/s deletion by id.

Several messaging system provide for queue browsing, which is 
effectively the same as Marconi's 'message listing'.

The fact that deletion is done by id using the HTTP interface to Marconi 
doesn't mean that the same approach needs to be taken using a different 
protocol.

> Also, we don't have different entities to provide publish/subscribe and
> producer/consumer as other messaging implementations (e.g. in Azure
> topics are for pub/sub and queues are for prod/cons)

That needn't be an issue. In AMQP the receiving link (i.e. the 
subscriber/consumer) can specify a 'distribution mode'. If the mode 
specified is 'copy' then the link is essentially a non-destructive 
reader of messages, (i.e. a sub in the pub/sub pattern). If it is move 
then messages allocated to that link should not be sent to other similar 
receiving links (i.e. competing consumers, in the producer/consumer 
pattern). For a distribution mode of 'move' the link claims the message 
sent to it. It can then also be required to acknowledge those messages 
(at which point they are permanently removed).

I.e. the distribution mode specified by the client when establishing the 
receiving link allows them to chose between the two behaviours, exactly 
as users of the HTTP interface do now.

> Marconi support prod/cons through claims. We rely on clients to send an
> ack of the message being consumed in order to delete the message and
> avoid other clients to consume it later.

AMQP is not that different here. The main difference being that in AMQP 
the message are identified through session scoped identifiers rather 
than global identifiers.

> So, the thing is... *should we add support for those additional
> features?

Assuming I understand the question - i.e. should Marconi try and support 
retrieval and deletion of messages by id over AMQP - I would argue, no, 
it should not.

Using AMQP as the transport, it becomes an alternate interface to 
Marconi. The fact that global ids may better suit the HTTP interface 
doesn't mean the same applies to other interfaces.

Someone using AMQP as the interface would, in my opinion, want the 
'normal' behaviour with that protocol.

> how we can do that in order to be consistent with AMQP and
> avoid confusion?*
>
> I drafted two different possibilities
>
> e.g. Delete a message
>
> - Specify the operation in the URI
>
> ./client.py amqp://127.0.0.1:8888/myqueue/messages/id/delete
> <http://127.0.0.1:8888/myqueue/messages/id/delete>
>
> - Specify the operation in the message suject
>
> ./client.py amqp://127.0.0.1:8888/myqueue
> <http://127.0.0.1:8888/myqueue> {subject: 'DELETE', id: 'id1'}

As above, I would use AMQP directly and naturally. To acknowledge a 
message you must have had it delivered to you (with a distribution mode 
of 'move'), and you must the accept it (i.e. acknowledge it). There is 
no need to layer a special purpose command to the Marconi system on top 
of an AMQP message.

--Gordon.