Open Stack

Guys, I was reading this article[4] on the flight home from Portland and
had a brainstorm that we could use this in Marconi to optimize inserts
into MongoDB

We have the following constraints:

1. FIFO must be guaranteed for a given queue (based on feedback from the
summit).
2. Markers must be stable (no duplicates, no missed messages).

Given that ObjectIds do NOT represent strict insertion order[1], we must
generate our own monotonic message IDs, ensuring that the documents are
inserted in the same order the IDs are generated. Note that a simple
side-counter will also not suffice:

T+0100 ms - Producer A posts message M1
T+0100 ms - Producer B posts message M2
T+0101 ms - Server A increments side counter and gets ID = 7
T+0101 ms - Server B increments side counter and gets ID = 8
T+0101 ms - Server B inserts M2 with ID = 8
T+0100 ms - Observer X lists messages with marker = 0, and gets new marker
= 8
T+0101 ms - Server A inserts M1 with ID = 7
T+0100 ms - Observer X lists messages with marker = 8 and never sees M1!

Using the Active Object model, but at a macro level (see also the
whiteboard diagram[3]):

1. Client posts message to Service
2. Service hashes/places queue to a specific Buffer instance (Redis). BTW
­ we need to talk about placement (I'll send a different email).
3. Service inserts message in Buffer
4. Buffer pulls messages off Redis, assigns ID, and inserts in DB (MongoDB)

This avoids the aforementioned race condition; we don't have to detect
that condition and retry. There is only a single thread pulling messages
off the buffer and inserting into the DB. Note that inserts into MongoDB
can't go any faster than that, anyway, unless we shard, and that brings up
a whole new set of issuesŠ

Is there a better way to do this?

@kgriffs


[1] http://docs.mongodb.org/manual/reference/object-id/
[2] 
https://wiki.openstack.org/wiki/Marconi/specs/storage/message-pagination#3.
_Implications
[3] http://i.imgur.com/prsBGlB.jpg
[4] 
http://www.drdobbs.com/architecture-and-design/know-when-to-use-an-active-o
bject-instea/227500074