Open Stack

Hi,

The switch to Elasticsearch 2.x for the gate functional tests revealed a couple of issues that need resolving fairly quickly [1] [2] [3]. Of them, [2] raises a problem regarding routing for documents belonging to child plugins.

Once we define a parent/child mapping, index/update/delete operations on child documents need to be accompanied with a routing hint since parent & child documents must be located on the same shard. Elasticsearch 1.x helpfully broadcasts deletes to all shards if no routing is given; Elasticsearch 2.x does not. We currently route on the parent id, which is fine in most cases but presents a problem for deletes. A neutron port delete notification contains only the port_id, not the network_id to which it belongs. To get the network_id we'd need to first query Elasticsearch, and again, without the network_id we can't issue a GET for the port but instead would need to run a search. This then raises some consistency problems.

My suggestion is that we route by tenant id instead unless a plugin specifically overrides it. This has potentially some advantages in that some searches will only run against information belonging to one tenant (although unfortunately glance and neutron both have the concept of public and shared resources), but more importantly we always have the project id for every notification. The downside is the potential for hot spotting; one big user may create load on one shard. A compromise would be to route by tenant id but only for child/parent plugins that are affected by this (currently, neutron). A further compromise would be a configuration level setting to route all resources by project id.

I intend to implement the second option (routing by project id where necessary) for now, but any thoughts welcome.

Steve

[1] https://bugs.launchpad.net/searchlight/+bug/1588933 - patch at https://review.openstack.org/325487 to fix it and disable tests affected by [2] and [3]
[2] https://bugs.launchpad.net/searchlight/+bug/1588540 - routing missing for some calls on child plugins, particularly delete
[3] https://bugs.launchpad.net/searchlight/+bug/1589319 - some reindex functional tests fail