We are eager to announce the release of: searchlight 0.2.0: OpenStack Search Service This release is part of the mitaka release series. For more details, please see below. 0.2.0 ^^^^^ Adds cinder plugins for volumes and snapshots. Added separation of administrator-only data at an index level, which allows full faceting and searching of sensitive data by those with access to it. In order to allow policy files to more be useful in Searchlight, we now support limited control over individual resource types via oslo policy control. Remove the 'index' API. It is currently a passthrough to elasticsearch, allowing direct index access and bypassing any document transforms. As such it is dangerous, and only marginally useful. Administrators wishing to do this should do so directly in Elasticsearch. Swift plugin for searchlight Ability added to reindex data while allowing indexing of notifications, and therefore returning accurate search results during the indexing operation. A fix for a couple of parent-child plugin relationship issues related to initial indexing. Child plugins can no longer be indexed separately from their parents. Config options for searchlight-api and searchlight-listener are grouped under config groups [api] and [listener]. The default config file for searchlight was searchlight-api.conf despite the fact that this config file was used by both the API and the listener processes. Therefore, we decided to rename the default config file to searchlight.conf. This change also prompted us to to follow the convention used by many other projects and simplify searchlight-api-paste.ini to api-paste.ini. New Features ************ * Cinder plugins for volumes and snapshots. * Plugins with sensitive, administrator-only data are indexed twice; once with that data and once without. Searches are directed to the appropriate version depending on the context of the user. * Adds neutron plugins for networks, subnets, ports and routers. * Policy rules can now be specified per resource type and per operation per resource type. * Removes the /index api and related tests. If an admin needs to directly modify data in Elasticsearch they should do so in Elasticsearch. * Three resource types are introduced for swift plugin. OS::Swift:Account ->OS::Swift::Container -->OS::Swift::Object * Reindexing of data now takes place in a separate Elasticsearch index. During indexing, incoming events are processed into both the old and new indices. Once indexing is complete, an Elasticsearch alias is used to switch searches seamlessly onto the new index, at which time the old one is removed. * Plugin configuration refers to *resource_group_name* rather than *index_name* to represent groups of resources separate from the logical Elasticsearch index. This setting is used to determine index and alias names. Known Issues ************ * Backups cannot currently be implemented because the cinder API does not provide enough information for RBAC. They will be added when that is resolved. * Neutron resources do not provide dates (created_at or updated_at). created_at is left empty; updated_at is taken from the server (and will be adjusted to use the notification timestamp at a later time) * RBAC is still separate from policy restriction; this is targeted for the Newton release. * The Swift service currently doesn't send notifications. Follow the swift plugin documentation for current solutions. * Reindexing must now take place across all resource types that share the index (it is not possible to re-index a specific resource type). This is a limitation that Elasticsearch 2.x would likely have forced upon us anyway. A subsequent patch will allow more efficient reindexing for associated resource types. Upgrade Notes ************* * Options 'bind_host', 'bind_port', 'backlog', 'tcp_keepidle', 'ca_file', 'cert_file', 'key_file', 'workers', 'max_header_line', 'public_endpoint', and 'http_keepalive' are moved to [api] group. Options 'listener_workers' is renamed to 'workers' and moved to [listener] group. Options 'host' and 'http_timeout' are removed since they are not currently used anywhere. * During an upgrade to Mitaka, operators must rename the previous configured searchlight-api.conf to searchlight.conf and searchlight- api-paste.ini to api-paste.ini. Security Issues *************** * Removes the admin-only '/index' api, which allowed an unnecessary amount of access to Elasticsearch. Bug Fixes ********* * Child plugins cannot be indexed on their own, only as a result of indexing their parent. This fixes an issue whereby indexing only a parent plugin could leave orphaned children (or vice versa). * Applies mappings for parent/child plugins simultaneously. Under elasticsearch 2.x, child mappings must be created at the same time or before parents, not the other way round. * The workers for listener was always 1, so config listener_workers was not work as expected. Now operators can set 'workers' under group [listener] to running multiple listener processes. Changes in searchlight 0.2.0.0rc1..0.2.0 ---------------------------------------- 2a44b63 Backward compatibility with designate v1 api 4abf961 Disable oslo_config file discovery in tests bcc8243 Increase heap for elasticsearch in func tests 1e80553 Fix _is_multiple_alias_exception signature 574a095 Add missing zero-downtime indexing documentation bb5ff07 Re-indexing optimization for doc_type 25ca131 Add missing mitaka release notes 531447d Zero Downtime Re-indexing Error Handling. b1d339b Don't index DHCP ports a5ccab8 Update .gitreview for stable/mitaka Diffstat (except docs and test files) ------------------------------------- .gitreview | 1 + .../bp-index-role-separation-535e01ba3bb2252e.yaml | 11 + ...-zero-downtime-reindexing-a607d52b8711d845.yaml | 24 + searchlight/cmd/manage.py | 116 ++-- searchlight/elasticsearch/plugins/base.py | 5 + .../plugins/designate/notification_handlers.py | 119 +++- .../elasticsearch/plugins/designate/recordsets.py | 1 - .../elasticsearch/plugins/designate/zones.py | 2 +- .../plugins/neutron/notification_handlers.py | 12 + searchlight/elasticsearch/plugins/neutron/ports.py | 6 + searchlight/elasticsearch/plugins/utils.py | 210 +++++-- 22 files changed, 2186 insertions(+), 118 deletions(-)