Open Stack

---------- Forwarded message ----------
From: Matt Riedemann <mriedemos at gmail.com>
Date: Thu, May 18, 2017 at 4:21 PM
Subject: [openstack-dev] [nova] Boston Forum session recap - searchlight
integration
To: openstack-dev at lists.openstack.org


Hi everyone,

After previous summits where we had vertical tracks for Nova sessions I
would provide a recap for each session.

The Forum in Boston was a bit different, so here I'm only attempting to
recap the Forum sessions that I ran. Dan Smith led a session on Cells v2,
John Garbutt led several sessions on the VM and Baremetal platform concept,
and Sean Dague led sessions on hierarchical quotas and API microversions,
and I'm going to leave recaps for those sessions to them.

I'll do these one at a time in separate emails.


Using Searchlight to list instances across cells in nova-api
------------------------------------------------------------

The etherpad for this session is here [1]. The goal for this session was to
explain the problem and proposed plan from the spec [2] to the operators in
the room and get feedback.

Polling the room we found that not many people are deploying Searchlight
but most everyone was using ElasticSearch.

An immediate concern that came up was the complexity involved with
integrating Searchlight, especially around issues with latency for state
changes and questioning how this does not redo the top-level cells v1 sync
issue. It admittedly does to an extent, but we don't have all of the weird
side code paths with cells v1 and it should be self-healing. Kris Lindgren
noted that the instance.usage.exists periodic notification from the
computes hammers their notification bus; we suggested he report a bug so we
can fix that.

It was also noted that if data is corrupted in ElasticSearch or is out of
sync, you could re-sync that from nova to searchlight, however, searchlight
syncs up with nova via the compute REST API, which if the compute REST API
is using searchlight in the backend, you end up getting into an infinite
loop of broken. This could probably be fixed with bypass query options in
the compute API, but it's not a fun problem.

It was also suggested that we store a minimal set of data about instances
in the top-level nova API database's instance_mappings table, where all we
have today is the uuid. Anything that is set in the API would probably be
OK for this, but operators in the room noted that they frequently need to
filter instances by an IP, which is set in the compute. So this option
turns into a slippery slope, and is potentially not inter-operable across
clouds.

Matt Booth is also skeptical that we can't have a multi-cell query perform
well, and he's proposed a POC here [3]. If that works out, then it defeats
the main purpose for using Searchlight for listing instances in the compute
API.

Since sorting instances across cells is the main issue, it was also
suggested that we allow a config option to disable sorting in the API. It
was stated this would be without a microversion, and filtering/paging would
still be supported. I'm personally skeptical about how this could be
consider inter-operable or discoverable for API users, and would need more
thought and input from users like Monty Taylor and Clark Boylan.

Next steps are going to be fleshing out Matt Booth's POC for efficiently
listing instances across cells. I think we can still continue working on
the versioned notifications changes we're making for searchlight as those
are useful on their own. And we should still work on enabling searchlight
in the nova-next CI job so we can get an idea for how the versioned
notifications are working by a consumer. However, any major development for
actually integrating searchlight into Nova is probably on hold at the
moment until we know how Matt's POC works.

[1] https://etherpad.openstack.org/p/BOS-forum-using-searchlight
-to-list-instances
[2] https://specs.openstack.org/openstack/nova-specs/specs/pike/
approved/list-instances-using-searchlight.html
[3] https://review.openstack.org/#/c/463618/

-- 

Thanks,

Matt

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
-- 
Kind regards,

Melvin Hillsman
mrhillsman at gmail.com
mobile: (832) 264-2646

Learner | Ideation | Belief | Responsibility | Command
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170518/69dc7d0e/attachment.html>