Open Stack

Thanks for these notes, Matt. Couple of comments inline.





On 5/2/16, 7:32 PM, "Matt Riedemann" <mriedem at linux.vnet.ibm.com> wrote:

>Andrew Laski led a double session for cells v2 on Wednesday afternoon. 
>The full session etherpad is here [1].
>
>Andrew started with an overview of what's done and what's in progress. 
>Note that some of the background on cells, what's been completed for 
>cells v2 and what's being worked on is also in a summit video from a 
>user conference talk that Andrew gave [2].
>
>Pagination
>----------
>
>This took a significant portion of the second cells v2 session and is 
>one of the more complicated problems to sort out. There are problems 
>with listing all instances across all cells especially when we support 
>sorting. And we really have a latent bug in the API since we never 
>restricted the list of valid sort keys for listing instances, so you can 
>literally sort on anything in the instances table in the DB.
>
>There were some ideas about how to handle this:
>
>1. Don't support sorting in the API if you have multiple cells. Leave it 
>up to the caller to sort the results on their own. Obviously this isn't 
>a great solution for existing users that rely on this in the API.
>
>2. Each cell sorts the results individually, and the API merge sorts the 
>results from the cells. There is still overhead here.
>
>3. Don't split the database, or use a distributed database like Redis. 
>Since this wasn't brought up in person in the session, or on Friday, it 
>wasn't discussed. There is another thread about this though [4].
>
>4. Use the OpenStack Searchlight project for doing massive queries like 
>this. This would be optional for a cell of one but recommended/required 
>for anyone running multiple cells. The downside to this is it's a new 
>dependency, and requires Elasticsearch (but many deployments are 
>probably already running an ELK stack for monitoring their logs). It's 
>also unclear at an early stage how easy this would be to integrate into 
>Nova. Plus deployers would need to setup Searchlight to listen to 
>notifications emitted from Nova so the indexes are updated in ES. It is, 
>however, arguably a better tool for the job than Nova trying to deal 
>with filtering and sorting with python. There is general agreement 
>within the core team that this is the path forward, but it's going to 
>require investigation and testing before we get a better idea of how 
>feasible this is.


I’ve spoken about this with some members of the Searchlight team and we
would like to do whatever we can to support this effort, so please let us
know how we can best help out here.

From our standpoint, versioned notifications with enough data that we
don’t have to do any callbacks to fill in the gaps at indexing time has
been placed among our top priorities for Newton. We had a fishbowl
at the summit on notifications and Jay introduced a couple of us to
Gibi. So, we’ll be getting involved as consumers of that effort very
shortly.


>
>Related to paging, we also have an existing problem with the marker that 
>will need to be sorted out before we can support multiple cells with v2. 
>Flavors are now in the API DB, and we return a marker for paging, but it 
>doesn't have the cell context, so we have to work that in. The good news 
>is we control the marker and we never documented anywhere that it's a 
>specific resource uuid (although for instances it is the last instance 
>uuid processed). So this is fixable, but is a known issue right now.
>