Open Stack

Excerpts from Zane Bitter's message of 2015-05-27 10:40:48 -0700:
> On 27/05/15 12:42, Clint Byrum wrote:
> >
> > == Crazy idea section ==
> >
> > One thing I never had a chance to discuss with any of the Zaqar devs that
> > I would find interesting is an email-only backend for Zaqar. Basically
> > make Zaqar an HTTP-to-email gateway. There are quite a few hyper-scale
> > options for SMTP and IMAP, and they're inherently multi-tenant, so I'd
> > find it interesting to see if the full Zaqar API could be mapped onto
> > that. This would probably be more comfortable to scale for some deployers
> > than Redis or MongoDB, and might have the nice side-effect that a deployer
> > could expose IMAP IDLE for efficient end-user subscription,
> 
> Can you guarantee delivery end-to-end (and still get the scaling 
> benefits)? Because AIUI SMTP is only best effort, and that makes this 
> idea a non-starter IMHO.
> 

So yes and no. If you are going to set your Zaqar backend up to forward
messages across the internet, then no, of course you cannot guarantee
delivery. That is not what I am suggesting as the default configuration.

However, in a closed system like a Zaqar backend SMTP would be, you
have control over things like retries and bandwidth, so you can at
least guarantee that if the intended destination is available, the
message will make it there. If you have 100 SMTP servers enqueing for
1000 destination queue servers, all with 1Gbit network between them,
you'd simply set your retry interval much lower than for the internet,
and allow infinite or very very high retries before giving up on messages.

The scaling comes from the asynchronous queueing. One can have
guaranteed delivery with asynchronous queueing.