Open Stack

We are eager to announce the release of:

searchlight 0.2.0: OpenStack Search Service

This release is part of the mitaka release series.

For more details, please see below.

0.2.0
^^^^^

Adds cinder plugins for volumes and snapshots.

Added separation of administrator-only data at an index level, which
allows full faceting and searching of sensitive data by those with
access to it.

In order to allow policy files to more be useful in Searchlight, we
now support limited control over individual resource types via oslo
policy control.

Remove the 'index' API. It is currently a passthrough to
elasticsearch, allowing direct index access and bypassing any document
transforms. As such it is dangerous, and only marginally useful.
Administrators wishing to do this should do so directly in
Elasticsearch.

Swift plugin for searchlight

Ability added to reindex data while allowing indexing of
notifications, and therefore returning accurate search results during
the indexing operation.

A fix for a couple of parent-child plugin relationship issues related
to initial indexing. Child plugins can no longer be indexed separately
from their parents.

Config options for searchlight-api and searchlight-listener are
grouped under config groups [api] and [listener].

The default config file for searchlight was searchlight-api.conf
despite the fact that this config file was used by both the API and
the listener processes. Therefore, we decided to rename the default
config file to searchlight.conf. This change also prompted us to to
follow the convention used by many other projects and simplify
searchlight-api-paste.ini to api-paste.ini.


New Features
************

* Cinder plugins for volumes and snapshots.

* Plugins with sensitive, administrator-only data are indexed twice;
  once with that data and once without. Searches are directed to the
  appropriate version depending on the context of the user.

* Adds neutron plugins for networks, subnets, ports and routers.

* Policy rules can now be specified per resource type and per
  operation per resource type.

* Removes the /index api and related tests. If an admin needs to
  directly modify data in Elasticsearch they should do so in
  Elasticsearch.

* Three resource types are introduced for swift plugin.
  OS::Swift:Account ->OS::Swift::Container -->OS::Swift::Object

* Reindexing of data now takes place in a separate Elasticsearch
  index. During indexing, incoming events are processed into both the
  old and new indices. Once indexing is complete, an Elasticsearch
  alias is used to switch searches seamlessly onto the new index, at
  which time the old one is removed.

* Plugin configuration refers to *resource_group_name* rather than
  *index_name* to represent groups of resources separate from the
  logical Elasticsearch index. This setting is used to determine index
  and alias names.


Known Issues
************

* Backups cannot currently be implemented because the cinder API
  does not provide enough information for RBAC. They will be added
  when that is resolved.

* Neutron resources do not provide dates (created_at or updated_at).
  created_at is left empty; updated_at is taken from the server (and
  will be adjusted to use the notification timestamp at a later time)

* RBAC is still separate from policy restriction; this is targeted
  for the Newton release.

* The Swift service currently doesn't send notifications. Follow the
  swift plugin documentation for current solutions.

* Reindexing must now take place across all resource types that
  share the index (it is not possible to re-index a specific resource
  type). This is a limitation that Elasticsearch 2.x would likely have
  forced upon us anyway. A subsequent patch will allow more efficient
  reindexing for associated resource types.


Upgrade Notes
*************

* Options 'bind_host', 'bind_port', 'backlog', 'tcp_keepidle',
  'ca_file', 'cert_file', 'key_file', 'workers', 'max_header_line',
  'public_endpoint', and 'http_keepalive' are moved to [api] group.
  Options 'listener_workers' is renamed to 'workers' and moved to
  [listener] group. Options 'host' and 'http_timeout' are removed
  since they are not currently used anywhere.

* During an upgrade to Mitaka, operators must rename the previous
  configured searchlight-api.conf to searchlight.conf and searchlight-
  api-paste.ini to api-paste.ini.


Security Issues
***************

* Removes the admin-only '/index' api, which allowed an unnecessary
  amount of access to Elasticsearch.


Bug Fixes
*********

* Child plugins cannot be indexed on their own, only as a result of
  indexing their parent. This fixes an issue whereby indexing only a
  parent plugin could leave orphaned children (or vice versa).

* Applies mappings for parent/child plugins simultaneously. Under
  elasticsearch 2.x, child mappings must be created at the same time
  or before parents, not the other way round.

* The workers for listener was always 1, so config listener_workers
  was not work as expected. Now operators can set 'workers' under
  group [listener] to running multiple listener processes.

Changes in searchlight 0.2.0.0rc1..0.2.0
----------------------------------------

2a44b63 Backward compatibility with designate v1 api
4abf961 Disable oslo_config file discovery in tests
bcc8243 Increase heap for elasticsearch in func tests
1e80553 Fix _is_multiple_alias_exception signature
574a095 Add missing zero-downtime indexing documentation
bb5ff07 Re-indexing optimization for doc_type
25ca131 Add missing mitaka release notes
531447d Zero Downtime Re-indexing Error Handling.
b1d339b Don't index DHCP ports
a5ccab8 Update .gitreview for stable/mitaka

Diffstat (except docs and test files)
-------------------------------------

.gitreview                                         |   1 +
.../bp-index-role-separation-535e01ba3bb2252e.yaml |  11 +
...-zero-downtime-reindexing-a607d52b8711d845.yaml |  24 +
searchlight/cmd/manage.py                          | 116 ++--
searchlight/elasticsearch/plugins/base.py          |   5 +
.../plugins/designate/notification_handlers.py     | 119 +++-
.../elasticsearch/plugins/designate/recordsets.py  |   1 -
.../elasticsearch/plugins/designate/zones.py       |   2 +-
.../plugins/neutron/notification_handlers.py       |  12 +
searchlight/elasticsearch/plugins/neutron/ports.py |   6 +
searchlight/elasticsearch/plugins/utils.py         | 210 +++++--
22 files changed, 2186 insertions(+), 118 deletions(-)