Open Stack

On Apr 28, 2016, at 1:09 PM, Jay Pipes <jaypipes at gmail.com> wrote:

> nova list as an admin (or any user frankly) should be a proxy call to Project Searchlight and elasticsearch.
> 
> elasticsearch is a great interface for this kind of operation. We should use it.

Oh, that’s great! A Java-based dependency! I’m *sure* that there will be no objection to that.

> The Cells architecture, which allows the operator to both scale the message queue *and* limit the scope of failure domains, is a good one. Having a database that stores only the local (to the cell) information is perfectly fine given the top-level API database's index/mapping tables. Where this design has short-comings, as Ed and others point out, are things like doing list operations across dozens of separate cells. So, let's not use the Nova database for that and instead use a solution that works very well for it: Project Searchlight.

What I’m hearing is: let’s shard the database along a random line that seriously impacts performance, and then add another layer to help mitigate the performance impact of that decision.

> And for those that have small clouds and/or don't want/need to install elasticsearch, OK, cool, stick with a single cell and a single RDBMS instance.

Your own tests showed that a single RDBMS instance doesn’t even break a sweat under your test loads. I don’t see why we need to shard it in the first place, especially if in doing so we add another layer of complexity and another dependency in order to compensate for that choice. Cells are a useful concept, but this proposed implementation is adding way too much complexity and debt to make it worthwhile.


-- Ed Leafe