Open Stack

On 11/19/2013 12:27 PM, Joshua Harlow wrote:
> Personally I would prefer #3 from the below. #2 I think will still have to
> deal with consistency issues, just switching away from a DB doesn't make
> magical ponies and unicorns appear (in-fact it can potentially make the
> problem worse if its done incorrectly - and its pretty easy to get it
> wrong IMHO). #1 could also work, but then u hit a vertical scaling limit
> (works if u paid oracle for there DB or IBM for DB2 I suppose). I prefer
> #3 since I think it is honestly needed under all solutions.

Personally I think we need a combination of #3 (resource reservation) 
with something else to speed up scheduling.

We have multiple filters that currently loop over all the compute nodes, 
gathering a bunch of data from the DB and then ignoring most of that 
data while doing some simple logic in python.

There is really no need for the bulk of the resource information to be 
stored in the DB.  The compute nodes could broadcast their current state 
to all scheduler nodes, and the scheduler nodes could reserve resources 
directly from the compute nodes (triggering an update of all the other 
scheduler nodes).

Failing that, it should be possible to push at least some of the 
filtering down into the DB itself. Stuff like ramfilter or cpufilter 
would be trival (and fast) as an SQL query.

Chris