Open Stack

This discussion is actually rather timely. In Kilo, we just added a new
experimental API in Glance called the Catalog Index Service [1].
Basically, it provides multi-tenant elastic search based caching and
indexing for various resources using a plugin mechanism.

We proposed it and built it initially to be targeted specifically at
Glance resources for a number of reasons including constraining the scope
and ensuring that we had proper guidance over some RBAC aspects.

Not surprisingly, quite a few people expressed a very strong interest in
this service providing search and indexing for many OpenStack services
including Nova. There is also debate on where it should reside.
Ultimately, the decision was to address this during liberty and to have at
least one session at the Vancouver summit on it in the Glance track. I
also have proposed it as a topic on the Horizon summit planning ether pad
and plan to provide more details shortly (finishing out Kilo was the top
priority).

The current experimental service was built with a resource plugin
architecture so that the actual indexing (with notification listening) and
RBAC handling for different resource types is handled via plugins. In
addition, when deployed it is registered in Keystone as a service of type
³search² with its own endpoint.

The basic idea for Horizon has been that we can check to see if the search
service is enabled for the current user¹s logged in region. If so, we will
add support in the API layers to leverage the search service for resources
indexed by the service when appropriate.
 
We also will be able to take better advantage of advanced search faceting
with near real-time results and autocompletion using an angular UI
component called Magic Search [2] that Eucalyptus open sourced and which
IBM / HP have been working on to enable in Horizon [3]. We want to also
discuss this at the summit in the Horizon track.

We are currently investigating and prototyping out a number of things
related to all of the above and will provide more info and results very
shortly. We¹ll definitely be looking for feedback and help on how to
approach a number of topics!

Thank you,
Travis

[1] 
http://docs-draft.openstack.org/51/138051/9/check/gate-glance-specs-docs/d9
ad1b8//doc/build/html/specs/kilo/catalog-index-service.html

[2] https://www.youtube.com/watch?v=FTjqFddBJBA

[3] https://review.openstack.org/#/c/157481/



On 4/10/15, 11:30 AM, "Clint Byrum" <clint at fewbar.com> wrote:

>Excerpts from Attila Fazekas's message of 2015-04-10 06:20:02 -0700:
>> I think the Nova API (list) needs to be extended by custom attribute
>> filtering and with multiple `server` get by uuid (up to 128+).
>>  
>> The basic list usually does not shows what I need.
>> nova processing more data than it is really needs just for a basic list.
>> 
>> The detailed list is very slow.
>> 
>> Maybe the attribute filtering or the multi item get(/POST) is not too
>>RESTish thing,
>> but it can be very efficient!
>> 
>
>Multi-get is a good idea anyway, as it will help with workloads that
>need it. However, it won't really be much better than the detail list for
>things like Horizon. It will just break the problem up into a less-slow
>list, and a less-slow fetch, instead of one slow list.
>
>This is related to the discussion about how OpenStack should talk
>to users. It would make a lot more sense for Horizon to poll a high
>speed but not 100% consistent Atom feed that is updated by a daemon
>listening to notifications, than always asking for the details from the
>realtime-accurate API. I believe rackspace has something in place called
>'yagi'[1] that does this for them, but I don't know what UI's they have
>that use it.
>
>Another option is for the listing API to just cache aggressively on its
>backend, with users being given the option to add a request option that
>means "go slow and get me up to date information." However, one must be
>careful to avoid thundering herds and other problems that backend caches
>tend to mask until the worst moment. For instance, if the cache starts
>getting invalidated faster than users are querying it because there are
>too many users asking for the slow version, then you are suddenly back
>in the sinking boat without a bucket to bail water anymore.
>
>[1] https://github.com/rackerlabs/yagi
>
>__________________________________________________________________________
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev