OpenStack-announce
Threads by month
- ----- 2024 -----
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2023 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2022 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2021 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2020 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2019 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2018 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2017 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2016 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2015 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2014 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2013 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2012 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2011 -----
- December
- November
- October
- September
- August
- July
- June
- May
- April
- March
- February
- January
- ----- 2010 -----
- December
- November
October 2016
- 4 participants
- 153 discussions
We are stoked to announce the release of:
panko 1.0.0: Event storage dispatcher for Ceilometer
This release is part of the newton release series.
For more details, please see below.
Changes in panko 1.0.0.0b3..1.0.0
---------------------------------
6c7d12e Pass oslo_config_project to keystone conf
d8dbf58 Clean deprecated "rpc_backend" in test_bin
64ca981 Remove unused logging import
910189c Remove unused import logging and LOG object
Diffstat (except docs and test files)
-------------------------------------
etc/panko/api_paste.ini | 1 +
panko/api/controllers/v2/utils.py | 3 ---
4 files changed, 3 insertions(+), 12 deletions(-)
1
0
We are pumped to announce the release of:
zaqar 3.0.0: OpenStack Queuing and Notification Service
This release is part of the newton release series.
For more details, please see below.
3.0.0
^^^^^
New Features
************
* Add a new webhook notifier using trust authentication. When using
the 'trust+' URL prefix, Zaqar will create a Keystone trust for the
user, and then use it when a notification happens to authenticate
against Keystone and send the token to the endpoint.
* Support 'post_data' and 'post_headers' options on subscribers,
allowing customization of the payload when having a webhook
subscriber. The 'post_data' option supports the '$zaqar_message$'
string template, which will be replaced by the serialized JSON
message if specified.
* Queues now behave lazy in subscriptions also. So there is no need
for the user to pre-create a queue before creating a subscription
for this queue. Zaqar will create the queue automatically on the
subscription creation request. As before, all subscriptions will
continue to stay active even if the corresponding queue was deleted.
* Currently Zaqar can support more built-in/reserved attributes in
queue. For now there are two important attributes
'max_messages_post_size' and 'max_message_ttl'. With this feature,
when user query queues Zaqar will show those two attributes (read
from config file if there is no customized value from user) in queue
metadata so that user can know what value it is.
* Currently, the v1 API is still accessible though it has been
deprecated for a while. And we're going to deprecate v1.1 soon. To
keep the backward compatibility, a new config option -
"enable_deprecated_api_versions" is added so that operator can
totally turn off an API version or still support it by adding the
API version to the list of the new config option.
* Now before users send messages to subscribers through a queue, the
subscribers should be confirmed first. Zaqar only sends messages to
the confirmed subscribers. This feature supports "webhook" and
"mailto" subscribers with mongoDB or redis backend. The "mailto"
part will be done in O cycle. Set "require_confirmation = True" to
enable this feature. The default value is "False" now and we will
enable it by default after one or two cycles.
Deprecation Notes
*****************
* Zaqar API v2 has been released for several cycles and it is
integrated as the default API version by most of the OpenStack
services. So it is time to deprecated v1.1 in favor of v2. Now in
Newton cycle, Zaqar API v1.1 is officially deprecated.
Bug Fixes
*********
* When access the root path of Zaqar service, for example: curl GET
http://127.0.0.1:8888/, user will see 401 error. Which will cause
some front end proxy (like HAProxy) to complain. Now this issue has
been fixed.
* Query for all subscriptions on a given queue by taking into
account the returned marker, if any. Without this fix, only 10
subscriptions can be extracted from database to send notification.
* In IPv6 management network environment, starting Zaqar server will
run into 'Address family for hostname not support' error when use
WSGI simple server. The root cause is that Python's TCPServer
implementation is hard-coded to use IPv4, even in IPv6 environments.
Now this issue has been fixed.
Changes in zaqar 2.0.0.0rc1..3.0.0
----------------------------------
1c77c6e Don't check key format in mongodb
77d7b1a Update UPPER_CONSTRAINTS_FILE for stable/newton
70a72bb Update .gitreview for stable/newton
49d8dd1 Remove small duplication from message storage
3701257 Consolidate constants to a module
b745146 Officially deprecate API v1.1
6767092 Fix links in devstack/README.rst
f38cecf Disable DEPRECATED API versions by default
da130bf Updated from global requirements
4bae6c7 Config logABug feature for zaqar api-ref
9f5b518 Clean imports in code
96f9e8b Remove openstack-doc-tools
3b6932f Use upper constraints for all jobs in tox.ini
6ff9be9 Get ready for os-api-ref sphinx theme change
69c7997 Subscription Confirmation Support-1
96aae7c Fix a minor error in API reference
fb775a9 Remove the useless function
cf6210d Don't check for project-id if accessing the / route
77d0567 Adding Negative tests cases to Zaqar v2
8ef5fc9 Start accepting **kwargs in service clients
c7fa056 Fix errors in API reference for message resource
ef50510 modify the home-page info with the developer documentation
8bdc215 Add flavors api ref
930a075 Support `messages` API ref
43765a0 Cleanup i18n marker functions to match Oslo usage
4083856 Updated from global requirements
28cc171 This patch removes extra spaces from files.
9cfa5bc Fix falcon.HTTPConflict initialization getting error
8703ead Add Python 3.5 classifier and venv
8480440 Remove docs,comments,references to Oslo Incubator
37cbe90 Fixes: Typo in comments
51604b4 Add a trust notifier task
0966c7a Ensure queue exists before get/update the claim
fabc0b1 Add pools api ref
62e92fc Disable warnerrors=true in setup.cfg
18fc577 Add `Pre-signed queue' api ref
329b2d9 Config: no need to set default=None
cc16004 Updated from global requirements
31d6bb6 Fix issue with subscription list in Redis
349839d Remove tempest smoke tests run
d0a129f (redis)Fix the 503 error when get/delete messages
ef0f8ba Add `Health` api ref
1772e66 Fix the wrong check of X-PROJECT-ID
65bf61d Add subscription api ref
87af703 Cleanup integration tests setup
228f4a6 Clean the auth app after authentication failure
721925c Add Claims for api ref
53b5954 Let v2 tempest base on the base tests
95c9c5d Fix debug log missing issue
35cc77f Remove leftovers on websocket disconnections
7c6eecd Fix config parsing in zaqar-server
18c169b Updated from global requirements
d8cde2a Fix post install jobs
e439b2e Relocates cli options registration from bootstrap to server
6cb3855 Delete in tree config ref
b8a70e4 Add release note for bug 1590173
9a3b019 Add Queues for api ref
5134d30 Add Zaqar installation guide
069bc54 Fix incorrect message when getting non-existing claim
4c2b7e0 Make queues lazy in subscriptions
e9dbb19 Add release notes
989f477 Use is_valid_ipv6 from oslo.utils
d30ceae Use debug instead of warning for pipeline implement error
61100fe Fix subscription limit per queue
e72dc08 Fix fail of getting claim after claim expire(Redis)
c1f0ac1 Make the notifier max_workers configurable
897424a Show 'age' field in subscriptions (redis)
3b32dfb Show 'age' field in subscriptions (mongodb)
682eae0 Add reserved metadata check
c5640b8 Support Tempest test as gate job
8e5be21 Avoid multi-threading problem with sql backend
48f3e9f Fix fail of claim after some message expire(Redis)
437c26a Always use pip constraints
da4009d Init commit to add API ref in tree
4f1bff4 Updated from global requirements
c3af668 Define context.roles with base class
e007e47 Use run() wrapper instead of connection.execute()
6250cde Improve the log to avoid unexpected ERROR
ebc6721 Rename notification/task to notification/tasks
c3ebf0d Replace tempest-lib with tempest.lib
437c2fe Refactor zaqar/storage/utils.py to respect DRY
7e87ade remove verbose option in devstack plugin
e6d975f Fix falcon confilct in requirements.txt
58e8429 Fix py34 gate job
3f15c0b Move to falcon middleware
d744f81 Changed assertTrue(isinstance()) to assertIsInstance.
592304e Trivial: remove openstack/common in flake8 exclude list
a066bca Fix cache decorator tests
4a9f313 Imported Translations from Zanata
4f6ac58 Updated from global requirements
cae19a2 Fix wrong claim error message
fc612ce Default message ttl is needed to change correctly
b59c304 Fix auth issue when accessing root path "/"
8237744 Make sure use IPv6 sockets for Zaqar in IPv6 environment
e4a5fd4 Show default attributes for queue
e78c257 Updated from global requirements
0810e12 Added samples in multiple languages
791cc6f Updated from global requirements
82e02a1 Fix tempest tests list
d5286f9 Fix service_available opt registration and extra comma
b645feb expires should be checked when using pre-signed url
da85355 Validate PUT of reserved queue attributes metadata
400fc6e Validate PUT of reserved queue attributes metadata
89c20fb Warn on upcoming deprecations for v1.0 and v1.1
9671738 Generate oslo.cache options via tox -e genconfig
eb3dcee Warn on upcoming deprecations for v1.0 and v1.1
0a191c2 Update reno for stable/mitaka
05efac8 Update .gitreview for stable/mitaka
7f91ccc Use assertEqual/GreaterEqual/LessEqual
Diffstat (except docs and test files)
-------------------------------------
.gitreview | 1 +
api-ref/source/claims.inc | 216 +++++++
api-ref/source/conf.py | 250 ++++++++
api-ref/source/flavors.inc | 205 +++++++
api-ref/source/health.inc | 66 +++
api-ref/source/index.rst | 27 +
api-ref/source/messages.inc | 328 +++++++++++
api-ref/source/parameters.yaml | 434 ++++++++++++++
api-ref/source/pools.inc | 205 +++++++
api-ref/source/queues.inc | 311 ++++++++++
api-ref/source/samples/claim_messages_request.json | 4 +
.../source/samples/claim_messages_response.json | 12 +
api-ref/source/samples/claim_query_response.json | 15 +
api-ref/source/samples/claim_update_request.json | 4 +
api-ref/source/samples/flavor-create-request.json | 3 +
api-ref/source/samples/flavor-list-response.json | 22 +
api-ref/source/samples/flavor-show-response.json | 12 +
api-ref/source/samples/flavor-update-request.json | 3 +
api-ref/source/samples/flavor-update-response.json | 12 +
api-ref/source/samples/health-response.json | 48 ++
.../samples/messages-get-byids-response.json | 15 +
api-ref/source/samples/messages-get-response.json | 11 +
api-ref/source/samples/messages-list-response.json | 32 +
api-ref/source/samples/messages-post-request.json | 18 +
api-ref/source/samples/messages-post-response.json | 6 +
api-ref/source/samples/pool-create-request.json | 8 +
api-ref/source/samples/pool-list-response.json | 24 +
api-ref/source/samples/pool-show-response.json | 7 +
api-ref/source/samples/pool-update-request.json | 8 +
api-ref/source/samples/pool-update-response.json | 7 +
api-ref/source/samples/queue-create-request.json | 5 +
.../source/samples/queue-pre-signed-request.json | 5 +
.../source/samples/queue-pre-signed-response.json | 16 +
api-ref/source/samples/queue-show-response.json | 5 +
api-ref/source/samples/queue-stats-response.json | 7 +
api-ref/source/samples/queue-update-request.json | 7 +
api-ref/source/samples/queue-update-response.json | 3 +
api-ref/source/samples/queues-list-response.json | 22 +
.../samples/subscription-create-request-http.json | 5 +
.../samples/subscription-create-request-mail.json | 8 +
.../samples/subscription-create-response.json | 3 +
.../source/samples/subscription-show-response.json | 10 +
.../samples/subscription-update-request.json | 7 +
.../samples/subscriptions-list-response.json | 26 +
api-ref/source/samples/versions-list-response.json | 55 ++
api-ref/source/subscription.inc | 225 +++++++
api-ref/source/versions.inc | 38 ++
devstack/README.rst | 12 +-
devstack/gate/gate_hook.sh | 19 +-
devstack/plugin.sh | 16 +-
devstack/settings | 5 +
.../api/zaqar.notification.tasks.webhook.rst | 7 +
...qar.openstack.common.cache._backends.memory.rst | 7 -
.../api/zaqar.openstack.common.cache.backends.rst | 7 -
.../api/zaqar.openstack.common.cache.cache.rst | 7 -
.../api/zaqar.openstack.common.fileutils.rst | 7 -
.../api/zaqar.openstack.common.gettextutils.rst | 7 -
.../api/zaqar.openstack.common.lockutils.rst | 7 -
.../api/zaqar.openstack.common.timeutils.rst | 7 -
.../zaqar-config-ref/autogen/zaqar.flagmappings | 86 ---
.../zaqar-config-ref/autogen/zaqar.headers | 10 -
.../zaqar-config-ref/common/tables/zaqar-api.xml | 38 --
.../common/tables/zaqar-auth_token.xml | 145 -----
.../common/tables/zaqar-authentication.xml | 26 -
.../common/tables/zaqar-drivers.xml | 30 -
.../common/tables/zaqar-logging.xml | 102 ----
.../common/tables/zaqar-mongodb.xml | 70 ---
.../common/tables/zaqar-pooling.xml | 26 -
.../common/tables/zaqar-storage.xml | 34 --
.../common/tables/zaqar-transport.xml | 66 ---
.../zaqar-config-ref/common/tables/zaqar-wsgi.xml | 30 -
.../zaqar-config-ref/os-zaqar-configRef.xml | 250 --------
etc/oslo-config-generator/zaqar.conf | 2 +
etc/policy.json.sample | 1 +
examples/websocket.html | 309 ----------
install-guide/source/conf.py | 301 ++++++++++
install-guide/source/get_started.rst | 65 +++
install-guide/source/index.rst | 22 +
install-guide/source/install-rdo.rst | 545 +++++++++++++++++
install-guide/source/install.rst | 33 ++
install-guide/source/next-steps.rst | 9 +
install-guide/source/verify.rst | 36 ++
...dd-a-notifier-using-trust-271d9cd1d2b4cdeb.yaml | 10 +
.../notes/deprecate-v11-976cccc1b56a28e7.yaml | 6 +
..._auth_issue_for_root_path-b15e1c4e92e4e8b1.yaml | 7 +
.../fix_subscription_limit-c3cdc9385825285a.yaml | 5 +
...y-queues-in-subscriptions-6bade4a1b8eca3e5.yaml | 8 +
...ault_attributes_for_queue-3d87333752484c87.yaml | 8 +
...rnoff-deprecated-versions-44656aeb8ebb8881.yaml | 7 +
.../notes/user_ipv6_sockets-1e1b436de6b81ae3.yaml | 7 +
...subscription_confirmation-883cb7f325885ef0.yaml | 9 +
releasenotes/source/conf.py | 12 +-
releasenotes/source/index.rst | 3 +-
releasenotes/source/mitaka.rst | 6 +
requirements.txt | 20 +-
.../receive_message/JsonDecoder.java | 43 ++
.../receive_message/SampleZaqarEndpoint.java | 57 ++
.../send_message/SampleZaqarEndpoint.java | 45 ++
samples/javascript/receive_message/zaqar_sample.js | 32 +
samples/javascript/send_message/zaqar_sample.js | 25 +
samples/javascript/websocket.html | 309 ++++++++++
.../jaxrs/receive_message/SampleZaqarServlet.java | 55 ++
samples/jaxrs/send_message/SampleZaqarServlet.java | 52 ++
samples/nodejs/receive_message/zaqar_sample.js | 34 ++
samples/nodejs/send_message/zaqar_sample.js | 27 +
.../receive_message/zaqar_sample.py | 30 +
.../send_message/zaqar_sample.py | 27 +
samples/zaqar/subscriber_service_sample.py | 79 +++
setup.cfg | 20 +-
test-requirements.txt | 15 +-
tox.ini | 27 +-
zaqar/api/handler.py | 27 +-
zaqar/api/v1/request.py | 61 +-
zaqar/api/v1/response.py | 15 +-
zaqar/api/v1_1/request.py | 41 +-
zaqar/api/v1_1/response.py | 19 +-
zaqar/api/v2/endpoints.py | 23 +-
zaqar/api/v2/request.py | 17 +-
zaqar/api/v2/response.py | 19 +-
zaqar/bootstrap.py | 20 +-
zaqar/cmd/gc.py | 4 +-
zaqar/cmd/server.py | 20 +-
zaqar/common/api/request.py | 15 +-
zaqar/common/auth.py | 95 +++
zaqar/common/configs.py | 11 +-
zaqar/common/consts.py | 115 ++++
zaqar/common/decorators.py | 35 ++
zaqar/common/pipeline.py | 2 +-
zaqar/common/transport/wsgi/helpers.py | 17 +-
zaqar/common/urls.py | 14 +-
zaqar/context.py | 8 +-
zaqar/locale/es/LC_MESSAGES/zaqar.po | 22 +-
zaqar/locale/fr/LC_MESSAGES/zaqar-log-error.po | 15 +-
zaqar/locale/zaqar-log-error.pot | 54 --
zaqar/locale/zaqar.pot | 604 -------------------
zaqar/notification/notifier.py | 113 +++-
zaqar/notification/task/__init__.py | 0
zaqar/notification/task/mailto.py | 53 --
zaqar/notification/task/webhook.py | 38 --
zaqar/notification/tasks/__init__.py | 0
zaqar/notification/tasks/mailto.py | 56 ++
zaqar/notification/tasks/trust.py | 63 ++
zaqar/notification/tasks/webhook.py | 49 ++
zaqar/storage/configuration.py | 48 ++
zaqar/storage/mongodb/claims.py | 2 +-
zaqar/storage/mongodb/messages.py | 19 +-
zaqar/storage/mongodb/subscriptions.py | 59 +-
zaqar/storage/pipeline.py | 6 +-
zaqar/storage/pooling.py | 11 +
zaqar/storage/redis/claims.py | 15 +-
zaqar/storage/redis/messages.py | 11 +-
zaqar/storage/redis/models.py | 10 +-
zaqar/storage/redis/scripts/claim_messages.lua | 45 +-
zaqar/storage/redis/subscriptions.py | 22 +-
zaqar/storage/redis/utils.py | 2 +-
zaqar/storage/sqlalchemy/catalogue.py | 12 +-
zaqar/storage/sqlalchemy/driver.py | 8 +-
zaqar/storage/sqlalchemy/flavors.py | 14 +-
zaqar/storage/sqlalchemy/pools.py | 20 +-
zaqar/storage/utils.py | 7 +-
.../api_schema/response/v2/queues.py | 4 +-
.../services/messaging/json/messaging_client.py | 64 +-
.../unit/transport/websocket/v2/test_claims.py | 51 +-
.../unit/transport/websocket/v2/test_messages.py | 70 +--
.../transport/websocket/v2/test_queue_lifecycle.py | 47 +-
.../transport/websocket/v2/test_subscriptions.py | 126 +++-
.../unit/transport/wsgi/v1/test_validation.py | 16 +
.../unit/transport/wsgi/v1_1/test_validation.py | 39 ++
.../transport/wsgi/v2_0/test_queue_lifecycle.py | 67 ++-
.../unit/transport/wsgi/v2_0/test_subscriptions.py | 98 +++-
zaqar/transport/auth.py | 5 +
zaqar/transport/validation.py | 18 +-
zaqar/transport/websocket/driver.py | 2 +-
zaqar/transport/websocket/factory.py | 3 +
zaqar/transport/websocket/protocol.py | 24 +-
zaqar/transport/wsgi/app.py | 4 +-
zaqar/transport/wsgi/driver.py | 46 +-
zaqar/transport/wsgi/v1_0/__init__.py | 9 +
zaqar/transport/wsgi/v1_0/metadata.py | 14 +-
zaqar/transport/wsgi/v1_1/__init__.py | 17 +-
zaqar/transport/wsgi/v1_1/queues.py | 16 +-
zaqar/transport/wsgi/v2_0/__init__.py | 11 +-
zaqar/transport/wsgi/v2_0/pools.py | 2 +-
zaqar/transport/wsgi/v2_0/queues.py | 50 +-
zaqar/transport/wsgi/v2_0/subscriptions.py | 103 +++-
zaqar/transport/wsgi/v2_0/urls.py | 4 +-
254 files changed, 8872 insertions(+), 2902 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 0f828f5..5b913db 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6 +6 @@ pbr>=1.6 # Apache-2.0
-Babel>=1.3 # BSD
+Babel>=2.3.4 # BSD
@@ -9,2 +9,2 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT
-iso8601>=0.1.9 # MIT
-keystonemiddleware!=4.1.0,>=4.0.0 # Apache-2.0
+iso8601>=0.1.11 # MIT
+keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0
@@ -14 +14 @@ WebOb>=1.2.3 # MIT
-stevedore>=1.5.0 # Apache-2.0
+stevedore>=1.16.0 # Apache-2.0
@@ -17,2 +17,2 @@ oslo.cache>=1.5.0 # Apache-2.0
-oslo.config>=3.7.0 # Apache-2.0
-oslo.context>=0.2.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.context>=2.9.0 # Apache-2.0
@@ -22,2 +22,2 @@ oslo.serialization>=1.10.0 # Apache-2.0
-oslo.utils>=3.5.0 # Apache-2.0
-oslo.policy>=0.5.0 # Apache-2.0
+oslo.utils>=3.16.0 # Apache-2.0
+oslo.policy>=1.9.0 # Apache-2.0
@@ -28,2 +28,2 @@ autobahn>=0.10.1 # MIT License
-requests!=2.9.0,>=2.8.1 # Apache-2.0
-futurist>=0.11.0 # Apache-2.0
+requests>=2.10.0 # Apache-2.0
+futurist!=0.15.0,>=0.11.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index a2fb1cc..93fe258 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8 +8 @@ hacking<0.11,>=0.10.0
-mock>=1.2 # BSD
+mock>=2.0 # BSD
@@ -18 +18 @@ ddt>=1.0.1 # MIT
-fixtures>=1.3.1 # Apache-2.0/BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
@@ -24 +24 @@ testtools>=1.4.0 # MIT
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
@@ -26 +26 @@ oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0
-openstack-doc-tools>=0.23 # Apache-2.0
+openstackdocstheme>=1.5.0 # Apache-2.0
@@ -28 +28,2 @@ oslotest>=1.10.0 # Apache-2.0
-reno>=0.1.1 # Apache2
+reno>=1.8.0 # Apache2
+os-api-ref>=1.0.0 # Apache-2.0
@@ -30,2 +31,2 @@ reno>=0.1.1 # Apache2
-# Tempest Plugin
-tempest-lib>=0.14.0 # Apache-2.0
+# Tempest
+tempest>=12.1.0 # Apache-2.0
1
0
06 Oct '16
We are tickled pink to announce the release of:
os-collect-config 5.0.0: Collect and cache metadata, run hooks on
changes.
This release is part of the newton release series.
For more details, please see below.
Changes in os-collect-config 0.1.37..5.0.0
------------------------------------------
9ee07c9 Revert "Treat ec2 collector data as immutable"
8717436 Treat ec2 collector data as immutable
08ba371 Merge deployments data for zaqar collectors
e5518c1 Set a timeout for all data fetches using requests
7be429c Updated from global requirements
a88e273 Updated from global requirements
9a5c168 Updated from global requirements
9b78c81 Updated from global requirements
d45a8c5 Deprecated tox -downloadcache option removed
f092ea1 Switch to oslo.log again
ab5c66d Revert "Switch to oslo.log"
6965331 Switch to oslo.log
8b28ced Replace assertEqual(None, *) with assertIsNone in tests
Diffstat (except docs and test files)
-------------------------------------
openstack-common.conf | 3 -
os_collect_config/cfn.py | 10 +-
os_collect_config/collect.py | 5 +-
os_collect_config/ec2.py | 16 +-
os_collect_config/heat.py | 2 +-
os_collect_config/heat_local.py | 2 +-
os_collect_config/local.py | 2 +-
os_collect_config/merger.py | 2 +-
os_collect_config/openstack/__init__.py | 0
os_collect_config/openstack/common/__init__.py | 17 -
os_collect_config/openstack/common/gettextutils.py | 479 --------------
os_collect_config/openstack/common/importutils.py | 73 ---
os_collect_config/openstack/common/jsonutils.py | 196 ------
os_collect_config/openstack/common/local.py | 45 --
os_collect_config/openstack/common/log.py | 713 ---------------------
os_collect_config/openstack/common/strutils.py | 311 ---------
os_collect_config/openstack/common/timeutils.py | 210 ------
os_collect_config/request.py | 10 +-
os_collect_config/zaqar.py | 8 +-
requirements.txt | 25 +-
test-requirements.txt | 18 +-
tox.ini | 1 -
27 files changed, 113 insertions(+), 2100 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 923f651..1cd1a37 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4 +4 @@
-pbr>=1.6
+pbr>=1.6 # Apache-2.0
@@ -6,11 +6,12 @@ pbr>=1.6
-anyjson>=0.3.3
-eventlet>=0.17.4
-python-keystoneclient!=1.8.0,>=1.6.0
-python-heatclient>=0.6.0
-python-zaqarclient>=0.3.0
-requests!=2.9.0,>=2.8.1
-iso8601>=0.1.9
-lxml>=2.3
-oslo.config>=2.7.0 # Apache-2.0
-six>=1.9.0
-dogpile.cache>=0.5.4
+anyjson>=0.3.3 # BSD
+eventlet!=0.18.3,>=0.18.2 # MIT
+python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # Apache-2.0
+python-heatclient>=1.1.0 # Apache-2.0
+python-zaqarclient>=1.0.0 # Apache-2.0
+requests>=2.10.0 # Apache-2.0
+iso8601>=0.1.11 # MIT
+lxml>=2.3 # BSD
+oslo.config>=3.12.0 # Apache-2.0
+oslo.log>=1.14.0 # Apache-2.0
+six>=1.9.0 # MIT
+dogpile.cache>=0.6.1 # BSD
diff --git a/test-requirements.txt b/test-requirements.txt
index 95d5cf5..5913a12 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -6,9 +6,9 @@ hacking<0.10,>=0.9.2
-coverage>=3.6
-discover
-fixtures>=1.3.1
-mock>=1.2
-python-subunit>=0.0.18
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
-testrepository>=0.0.18
-testscenarios>=0.4
-testtools>=1.4.0
+coverage>=3.6 # Apache-2.0
+discover # BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
+mock>=2.0 # BSD
+python-subunit>=0.0.18 # Apache-2.0/BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
+testrepository>=0.0.18 # Apache-2.0/BSD
+testscenarios>=0.4 # Apache-2.0/BSD
+testtools>=1.4.0 # MIT
1
0
We are content to announce the release of:
murano 3.0.0: Murano API
This release is part of the newton release series.
For more details, please see below.
3.0.0
^^^^^
New Features
************
* Added a new manifest format 1.4.0. Introduced the 'Scope' keyword
for class methods to declare a method's accessibility from outside
through the API call.
* Implemented the capability for API endpoint "/catalog/packages" to
filter 'id', 'category', 'tag' properties using the 'in' operator.
An example of using the 'in' operator for 'id' is
'id=in:id1,id2,id3'. This filter is added using syntax that conforms
to the latest guidelines from the OpenStack API-WG.
* Added the "timeout" parameter to "runCommand" and "putFile"
methods of the "io.murano.configuration.Linux" class.
* Added the "driver" configuration option to the "networking" group.
It allows to explicitly select the networking driver. It supports
'neutron' and 'nova' options. If set to "None" (default), murano
attempts to use 'neutron' if available, 'nova' otherwise. The change
is backward compatible.
* Added the "description_text" field to environment and environment
templates database tables and respective API objects.
* Introduced a new MuranoPL class "io.murano.system.GC" Now MuranoPL
garbage collector can be used to set up destruction dependencies
between murano objects. If object Foo is subscribed to object Bar's
destruction, it will be notified through a specific handler. If both
Foo and Bar are going to be destroyed during one execution session,
Foo will be destroyed after Bar. You can omit the handler, in this
case destruction order will also be preserved. Handler can be a
static or a usual function.
* New on-request garbage collector for MuranoPL objects were
implemented. Garbage collection is triggered by
io.murano.system.GC.collect() static method. Garbage collector
destroys all object that are not reachable anymore. GC can handle
objects with cross-references and isolated object graphs. When
portion of object model becomes not reachable it destroyed in
predictable order such that child objects get destroyed before their
parents and, when possible, before objects that are subscribed to
their destruction notifications.
* Internally, both pre-deployment garbage collection (that was done
by comparision of "Objects" and "ObjectsCopy") and post-deployment
orphan object collection are now done through the new GC.
* io.murano.system.GC.isDoomed() static method was added. It can be
used within the ".destroy" method to test if other object is also
going to be destroyed.
* io.murano.system.GC.isDestroyed() static method was added. It
checks if the object is destroyed and thus no methods can be invoked
on it.
* Implemented the capability for the helper methods of Linux class
to run concurrently if executed for different VM agents.
* Added the following meta-classes to the core library - "Title"
"Description" "HelpText" "Hidden" "Section" "Position"
"ModelBuilder". These classes will later be used to implement
dynamic object model generation.
* Added an overload of the new function - "new($model, $owner)". It
loads complete object graph in a single call. Objects in the model
can have cross references. In that case, this is the only way to
instantiate the graph. Objects might be specified either in object
model format (with '?' attribute or in MuranoPL format (used for
Meta definitions).
* The contract "class()" now uses the same approach to load classes
from dictionaries. Thus the same two syntaxes apply there as well.
* Added Support for application deployment across OpenStack regions.
Now, all OpenStack resource classes inherit from
"io.murano.CloudResource" that provides ".getRegion()" method and
"regionName" property. This allows to assign resources to different
regions. ".getRegion()" returns "io.murano.CloudRegion" instance
that resource or its parent belongs to. "CloudRegion" has interface
similar to "Environment" class and is the correct way to get
"HeatStack" instance associated with the region, default network
configuration, security group manager and agent listener instances.
"Environment" now acts as default region so backward compatibility
is not broken. However new applications should not use environment
to set security group rules but rather a region(s) of their
instance(s) in order to work correctly when their instances were
configured to use region other than the default.
* Added the "api_workers" option to "murano" config group. It
controls the number of API workers launched by murano. If not set,
it would default to the number of CPUs available.
* Added a new engine RPC call to generate json-schema from MuranoPL
class. The schema may be generated either from the entire class or
for specific model builders - static actions that can be used to
generate object model from their input. Class schema is built by
inspecting class properties and method schema using the same
algorithm but applied to its arguments.
* Implemented a new framework for MuranoPL contracts. Now, instead
of several independent implementations of the same yaql methods
(string(), class() etc.) all implementations of the same method are
combined into single class. Therefore, we now have a class per
contract method. This also simplifies development of new contracts.
Each such class can provide methods for data transformation (default
contract usage), validation that is used to decide if the method can
be considered an extension method for the value, and json schema
generation method that was moved from the schema generator script.
* Previously, when a class overrode a property from its parent class
the value was stored separately for both of them, transformed by
each of the contracts. Thus each class saw the value of its
contract. In absolute majority of the cases, the observed value was
the same. However, if the contracts were compatible on the provided
value (say int() and string() contracts on the value "123") they
were different. This is considered to be a bad pattern. Now, the
value is stored only once per object and transformed by the contract
defined in the actual object type. All base contracts are used to
validate the transformed object thus this pattern will not work
anymore.
* The value that is stored in the object's properties is obtained by
executing special "finalize" contract implementation which by
default returns the input value unmodified. Because validation
happens on the transformed value before it gets finalized it is
possible for transformation to return a value that will pass the
validation though the final value won't. This is used to relax the
template() contract limitation that prevented child class from
excluding additional properties from the template.
* The "string()" contract no longer converts everything to string
values. Now it only converts scalar values to strings. Previous
behavior allowed "string()" property to accept lists and convert
them to their Python string representation which is clearly not what
developers expected.
* Due to refactoring, contracts work a little bit faster because
there is no more need to generate yaql function definition for each
contract method on each call.
* Changed the type representation in object model. Previous format
was to have three attributes in "?" section of the object - type,
classVersion and package where only the "type" is mandatory. Now
they are merged into single attribute "type" that has a format
"typeName/version@package". Version and package parts are still
optional.
* Previously, when pre-deployment garbage collection occurred it
executed ".destroy" method for objects that were present in the
"ObjectsCopy" section of the object model (which is the the snapshot
of the model after last deployment) and not present in the current
model anymore (because they were deleted through the API between
deployments). If the destroyed objects were to access another object
that was not deleted it was accessing its copy from the
"ObjectsCopy". Thus any changes to the internal state made by that
object were lost after the garbage collection finished (that is,
before the ".deploy" method call) and could not affect the
deployment. Now, if the object is present in both "Objects" and
"ObjectsCopy", a single instance (the one from "Objects") is used
for both garbage collection and deployment. As a consequence,
instances (in their ".destroy" method) now may observe changes made
to other objects they refer if they were not deleted, but modified
through the API. In some rare cases, it may break existing
applications.
* Separated murano service broker from murano-api into a murano-
cfapi service. Created a separate database and "paste.ini" for
service broker.
* Added a new API endpoint "v1/actions" to call static public
methods. It accepts class name, method name, method arguments, and
optionally package name and class version in the request body. This
call does not create an environment, object instances or database
records.
* Implemented a new contract function "template()". "template()"
works similar to the "class()" in regards to the data validation but
does not instantiate objects. Instead, the data is left in the
object model in dictionary format so that it could be instantiated
later with the "new()" function. Additionally, the function allows
excluding specified properties from validation and from the
resulting template so that they could be provided later. Objects
that are assigned to the property or argument with "template()"
contract will be automatically converted to their object model
representation.
* Split "Instance"'s ".deploy()" method into two phases -
"beginDeploy()" and "endDeploy()". This allows the application
developer to provision multiple instances at once without the need
to push the stack for each instance.
* Added API endpoint
"/templates/{env_template_id}/services/{path:.*?}" for environment
template application update operation.
* Added the capability to declare MuranoPL YAML methods with
variable length positional and keyword arguments. This is done using
argument "Usage" attribute. Regular arguments have Standard usage
which is the default. Variable length args (args in Python) should
have "Usage: VarArgs" and keyword args (kwargs) are declared with
"Usage: KwArgs". Inside the method they are seen as a list and a
dictionary correspondingly. For such arguments contracts are written
for individual argument values thus no need to write them as
lists/dicts.
Upgrade Notes
*************
* New database migration 015 has to be applied.
Deprecation Notes
*****************
* Deprecated the 'Usage Action' keyword. For format versions >=
1.4.0, use 'Scope Public' instead.
* Renamed the "workers" option from the "engine" group to
"engine_workers" to reduce ambiguity with the "api_workers" option.
Security Issues
***************
* cve-2016-4972 has been addressed. In ceveral places Murano used
loaders inherited directly from yaml.Loader when parsing MuranoPL
and UI files from packages. This is unsafe, because this loader is
capable of creating custom python objects from specifically
constructed yaml files. With this change all yaml loading operations
are done using safe loaders instead.
Bug Fixes
*********
* Core Library's init scripts used to have various problems
detecting pre-installed (by DIB) murano-agent on non-ubuntu images.
Agent setup script now checks wider list of directories before
attempting to install murano-agnet and service script now does not
impose strict script location.
* API call for deleting a service from environment template did not
return result of its operation. The issue is fixed.
* Fixed a bug when the UI dialog was not displayed in Murano
Dashboard for applications which don't have UI definitions bundled
in the package but generate them based on the package contents
instead. This usually affected HOT-based packages and other non-
muranopl- based applications.
* Removed the need for Keystone v2 options (admin_user,
admin_password, admin_tenant_name) when Keystone v3 is in use.
* Previously murano assumed that the service user and service
project are in the 'Default' domain. These values can now be set in
"keystone_authtoken" config group.
* Equality check (assertEqual) in test-runner can now properly
compare two MuranoPl objects.
* Prevented the resource leak for objects created during deployment
with "new()" function call.
* Murano is now able to deploy applications in the environments with
disabled Neutron Security Groups. Detection is based on the presence
of 'security-group' Neutron extension.
* Whenever murano-engine accesses script files, text script files
are opened in 'rU' mode which recognizes all types of newlines, and
binary files are opened in 'rb' mode to prevent their corruption.
* It is now possible to use version specifications like '=0.0.0'
when "semantic_version" library version '2.3.1' is installed.
Previously such specifications caused an error and '==0.0.0' had to
be used.
* Murano engine no longer logs methods "string()", "json()", and
"yaml()" of the 'io.murano.system.Resources' class. This is done to
prevent UnicodeDecodeError's when transferring binary files to
murano agent.
* The test-runner now outputs the tests it runs and their results to
stdout directly, instead of the logging system.
* The test-runner now does not output logs to stderr by default
unless a 'use_stderr' parameter is specified in the configuration
file.
* Fixed the issue that prevented the test-runner from properly
invoking "setUp" and "tearDown" methods of fixtures in some cases.
Changes in murano 2.0.0.0rc1..3.0.0
-----------------------------------
76e7d9e Updated from global requirements
6bb6702 Convert =0 version specs to ==0 specs
34aa88c Move getRegion() in CinderVolume to deploy() method
359c8c6 Fixes ObjectDestroyedError on StackTrace object
c141e1b Update UPPER_CONSTRAINTS_FILE for stable/newton
c64576e Update .gitreview for stable/newton
0e55368 Forces applications to be deleted before Heat stack
ce8e2ec Prevent logging of result of resources.string() method call
9bbd33d Prevent executor finalization until exception are not handled
c5623fd Small fixes in Core Library
2b57eb3 Serialization of destruction dependencies
dc050d4 GC.isDoomed() and GC.isDestroyed() functions were added
0aea40e Serialization of destroyed objects
dcbf5e1 Raise exception on call of method of destroyed object
c472691 Pass receiver to :GC.subscribeDestruction explicitly
584571c On-demand garbage collection
3ab0be1 Break cyclic references in DSL
e9fe5b2 Cleanup, clarify newton release-notes
9a2a63c Fix multiple errors in network configuration function
0122dfe Add joinedNetworks property to Instance class
1e6d619 Use openstack-client variables in openstack CLI calls
ab4d479 Better detect and configure murano-agent pre-installed on image
3799303 Updated from global requirements
cdd248d Adjust region name for network selection in devstack
1833842 Make tests compose packages in temporarily directory
44bc27a [Docs] Add info about using abstract dependencies between applications
da52590 Use default region_name parameter in clients
275a19c Create glare client in functional tests
a21e514 Move misplaced reno notes in place
f4cf8b3 Fix NetworkExplorer creation
2cf88ab Fix adds notes about replacements of items between % signs
b7c2cb4 Fixes invalid expression in app dev framework
087db8c Set the desired hostname to a instance using murano instead of Murano Pattern
bcef79b Add __ne__ built-in function for murano
0ccef5f Object's properties deserialize was fixed
33d8e18 Fix for Wrong reference in glare docs
8506fe4 ObjectStore parent lookup with temporary id was fixed
b1b57a2 Updated from global requirements
8757c6d Type name resolution was fixed
6162510 Increase description column length in task table
882136d An initial commit for MuranoPL garbage collector
ac7940a MasterSlave* classes were deleted
01b3ac3 Alive objects from Objects and ObjectsCopy were merged
2b30594 Migrate JSON schema generator to new framework
acd269f Update clients with multi-region support
077a3c3 Update app dev framework with multi-region support
7e8fb5f New framework for contracts
2f21323 TestFixture mocks were updated
1f3b1be firstOrDefault() was replaced with first(null)
7f21be2 Replace functions 'Dict.get' and 'del' with 'Dict.pop'
47c8f09 Updated from global requirements
688dcd0 TrivialFix: Remove cfg import unused
dfdc374 modify the home-page info with the developer documentation
ca733fe Correct murano reraising of exception
d331a4f Update Readme with correct Doc URL
0d1c6bd Install all dashboard/panel files from murano-dashboard
1e16a4e OpenstackSecurityConfigurable was renamed
bc4c548 Allow network driver selection override
50744ca Fix some typos in wsgi.py & catalog.py
8c1355f Updated from global requirements
6314553 Remove pot files
6395787 Adds more replica provider primitives
0a379e5 Migration of replication to the template() contract
79a6914 Clean imports in code
f1d20ec Update the UI test running documentation
66f310b Change Murano API detection in tests and fix tests itself
079f8c5 [docs] Add instruction to add policy files to horizon
5ff901b Use upper constraints for all jobs in tox.ini
7f8ee1b Base Application classes for App Development Framework
5da4efd Extracted replication capabilities from ServerGroup
855e4ea Updated from global requirements
8eb7d93 Revert "Fix config group for SSL in tempest test"
ffe85fc Fix config group for SSL in tempest test
50a7aa4 Fixes congress integration
f965e10 SoftwareComponent hierarchy refactoring
58d92ab Improved Server naming and provisioning reporting
f309068 Renamed Instance* to Server*
1995f04 AppDevFramework: Server Replication reporting
603b1c0 [Docs] Moves network configuration to admin-guide
8501b18 Refactor 'wait_for_environment_deploy' function
53a30e5 Use glare urls explicitly in devstack plugin
4665730 Add tempest versioning test suite
bb2d0e5 Support for multi-regional apps was added
be69379 Do not serialize Config properties
91d5289 Don't put non-initialized objects into ObjectStore
23a6718 template() contract function was introduced
5475ac1 Updated from global requirements
36b9110 Refactor merge_dicts helper function tests
23fbee8 Add multiple api workers
a6ff115 Trivial: Add python identifiers in setup.cfg.
da7e9ea Linux.runCommand() and Linux.putFile() are now non-synchronized
30846bb Changes type format in object model
a75c16e Fixed reporting in app development framework
2957b61 [docs] Remove "meta-class" term
d1bfc7d Updated from global requirements
30a6805 Add missing parenthesis
9e1f490 Add deployment murano tempest test
e689487 Make service broker work with GLARE again.
3ddfc96 Fix a typo in the rst file
0172551 Move .init call into separate load phase
d6456f0 Get rid of type origins
1d1e388 Add py2.7 and py3.4 identifiers in setup.cfg.
cf22984 Add cinder volumes attachment tests
595c38a Updated from global requirements
27b8f75 [docs] Fix formatting in installation guide
36915a9 Add base and packages for tempest murano scenario tests
1d721d7 Nested new() were not using 2-phase load
dcd32b5 Duplicate objects were instantiated for redeclared properties
209af8f Updated from global requirements
204cbd9 [Docs]Update the Murano service broker install guide
0fe151d Fix string interpolation to delayed by logging
e113f12 Add Python 3.5 classifier and venv
d344e42 Use assertIn, assertNotIn and assertIsNotNone
e59e943 Do not override credential provider in tenant isolation tests
b604434 [Docs] Move user and appdev guides from draft to main docs
ef85e7e SoftwareComponent implementation
e0e3b10 MuranoPL forms implementation
0694559 Respect per region RabbitMQ configs in Agent[Listener]
f2c1973 Updated from global requirements
c4c8be6 Concurrent Execution Control
79dcbb3 InstanceGroup and InstanceProvider hierarchy
e853cf8 Event Notification pattern implemented
d9f0add Replication classes of Application Development Framework library
0b30d06 TestFixtureWithEnvironment class for TestRunner
79b2a2b dump() function added to DSL
77e3c6e Ability to instantiate object graph
abf5fa3 Refactor tempest utils readlines() to read()
61447fd Fix typo in package loader logging
3b53096 Use assertEqual() instead of assertDictEqual()
904cb24 Fix used package versions for stable inheritance
0a835b8 Refactoring of the ObjectStore passing in DSL
e36c534 [Docs] Change FQNs of murano apps and add examples of FQNs
9e502d8 [Docs] Add information about Config property usage
3d8d5a0 Use devstack for service broker use separate paste and db
0854616 Replace OpenStack LLC with OpenStack Foundation
c25f867 Enable static action tests with glare backend
7e1cf62 Add tempest GLARE sanity check tests
0ef402c Initial implementation of artifacts client in tempest plugin
5a7a477 [Docs] Clarify how to run API and engine in separate terminal
9d97f58 Fix cfapi test job
157ceea Devstack install murano-dashboard with murano RBAC policy
e1da1e9 Updated from global requirements
b976914 [Docs]Update the Murano Dashboard install guide
efd0360 Fixed owner usage for dict-based new() function
c9a5855 Updated from global requirements
6550ab4 [Docs] Add parameter resources-dir for hot-packages
5221ddc Updated from global requirements
51cd192 Add py27-mitaka tox target
8a2c60e Updated from global requirements
3d20f25 Use keystoneauth1 lib for authentication helpers
4c6c823 fix a spelling mistake: applcation should be application
7d186c1 Implement API call and RPC call for static actions
75bded1 Could not invoke static methods from Python
81f1d00 [Docs] Add info about 2.2.0 version of execution plan template format
bbe19e4 Add filter based on 'in' operator
317e9bf Use upper-constraints in tox test environments
0b51fb4 [docs] Added murano gerrit dashboard
b899d79 Do not use stevedore namespase as a prefix for class FQNs
89952c1 Use the absolute path for the session backend
8fb54da Use multiprocessing.cpu_count instead of psutil.NUM_CPUS
a7c7ade Improve cover job output
edad301 Updated from global requirements
01af152 Improved Test-Runner's output
63b7556 Test-runner now properly invokes setUp/tearDown methods
d7de282 Added equality handlers for MuranoObjectInterface
2c165ee Update namespaces for MagnumBay application
2548c66 Include objects created with new() into the ObjectStore
6aedcda Allows Spec::transform treat generators as list
9d0c88a Convert script line endings for the target OS
345117b Improve __init__ detection
65902f9 Fix error in Magnum-app
b18df82 Amended reno note about booting from cinder volumes
c30eaf4 [Docs] Add a reference to the article about multi regions
09098e6 Use Murano-specific Horizon settings when devstack install
b555511 Get package service backend when package import in devstack
d5dc0f4 [Plugins] Updated from global requirements
9bdf4c0 Add metadata to the result of action serialization
91fd92c Implement new syntax for action declaration
d8ccb21 Fix issue with deployment with cloudbase-init on Windows
2423a2b Updated from global requirements
28de8c3 Use SafeLoader to load yaml files
0b495c5 Added a muranoPL-specific override of 'call'
5b8ab82 Refactor API params tuples to sets
f5f7fde Format logs in devstack setup
806fd1c Including a description field for environment and environment templates
262a89c Updated from global requirements
a9ae12a [docs] Add information about TestSuiteMultipleEnvironments
c25ef03 Use a correct config option in example plugin
7cc34e8 Add io.murano namespace to demo app for example plugin
2463680 Use murano CLI for importing core library in devstack
88d92b5 Remove version from example and heat-translator plugins
d17c3f6 Fix missing parenthesis in _getSubnetworks call
6e35fb4 Fix application template update
a0670b6 Two-phase Instance deployment support
55a785a Fixed a bug when Heat Stack Update was called without template changes
9c45ec7 Finish porting unit tests to Python 3
bcd838a Port API v1 unit tests to Python 3
f25776a Fixed inability to deploy if security groups are disabled
b2eef6e Port test_agent and test_catalog to Python 3
6f04c2b Implement meta-classes for UI hints
904aa45 Add timeout to the methods of Linux class
c0bfff2 Fix defaultGroupName of SecurityGroupManager Class
233c141 Devstack add compile message for murano-dashboard
7368211 Change to tempest stable API
eb4b0be Reorder release notes to put master on top
6377e95 [Docs] Add an example about usage of static methods in contracts
67e58f3 Updated from global requirements
0ce227e Update version of source files in cloudify example app
c4e6eaf Update obsolete way of initialize tempest client manager
e427334 Update functional test _get_stack
78037fc Updated from global requirements
c711590 [Docs] Fix typos
cae7e61 Updated *_domain_id to "Default"
596b551 Magnum plugin: import exceptions robustly
c3938f7 fix typo oslo.messaginga
1adc59c Fixing application template deletion
c58e69f Fix mismatch description in env template api doc
41d9114 Updated from global requirements
eb1d5e4 Updated from global requirements
59ce0ce [devstack] Pass MURANO_USE_GLARE as bool, not as str
badc062 Update version of cloudify client in cloudify plugin requirements
176379f Move service broker api to the top level
deb487d Generate separate db for murano service broker
5c8285d Replace obsolete way of get creds in tempest tests
b09cc17 Documentation to use sub-templates in HOT packages
95b0782 Updated from global requirements
928ee3f [Docs] Replace names and images
a4503d7 [Admin Guide] System requirements
afa85b3 Added package references to generated UIs
623ebce Forced flush of tmp file to disk on ui retrieval
fc2cf95 Support GLARE configuration in Murano tempest plugin
b460c80 Fix up Assignment table in MuranoPL Reference
a78185b Remove unnecessary executable permissions
f89df60 [Docs] Add info about manifest format to versioning docs
53f43dd Updated from global requirements
1835767 [docs] Fix small typo in docs about automated tests
01fc294 Updated from global requirements
67f2cc2 [Trivial] Fix mismatch string format argument
80d92d8 Add models and migration for new service broker db
89a4c4c Increase status report messages time resolution
c659b2d Correct wrong import statement in Magnum App
edbcc5f Updated from global requirements
2929f02 Trivial: ignore openstack/common in flake8 exclude list
b5ef3b5 [docs] Add info about sanity_check suits
66727be Updated from global requirements
5bb2782 [Docs] Install diskimage-builder by pip
c8d9394 [Murano Docs] Versioning
091b4d1 Allow insecure SSL communications with RabbitMQ
bc8943c Modified docstrings to comply with pep8 H405 style check
15df963 Change incorrect murano-agent bin file location
8946b95 Updated from global requirements
ca9a6ed Remove explicit version from magnum plugin
5ea200e Support GLARE installation in devstack Murano plugin
5c94920 [Murano Docs] Cinder Volume supporting
21e877c Pass [rabbitmq]/ca_certs file to murano-spawned instance
df4c52d Skip test_migrations on Python 3
440b4b1 Revert the destroy execution order
2460acb [Trivial] Remove unnecessary executable privilege
10aaacb [Murano Docs] Murano Contributor rules
675db99 Fix tempest.conf generation
789dabb Support for *args/**kwargs was added to the MuranoPL
a787d63 Deployment reports datetime DB insertion was fixed
c4bf6f8 Incorrect Method param inheritance was fixed
dc52dea [Docs] Add limitations of deployment murano-agent by cloud-init
aaf9e54 Fix typo in word "settings" in docs about using glare
7102898 Fix inaccuracies in docs about murano apps tests
9536e51 Fixes race condition in HeatStack
34154bc Fixed a bug in _get_glare_url
3f8fc9e Updated from global requirements
f6e795e Fix base64 on Python 3 in yaql_functions.py
2461bad Port cloudfoundry/test_cfapi.py to Python 3
398b8cd Port test_plugin_loader to Python 3
205c0c6 Fix tox -e py34
9aa8200 Imported Translations from Zanata
35a93b0 [Murano Docs] Edits
d0c2310 Use unified decorator for retries
0467994 Updated from global requirements
bff8809 [Murano Docs] Remove articles
47b295c [Murano docs] Publish CLI section
776a4ce Switch to using get_notification_listner
3f45506 Change clusterip field description in docs about Dynamic UI
31d6877 Add ability to configure home region in murano devstack installation
cb0561a Do not force admin credentials in service broker test suite
622ef5e Remove tempest-lib from test-requirements.
12636e2 Increased the size of TEXT columns to store large object models
2858689 'GetPackageUI' API can now be called even if Glare is used
9490e43 Updated from global requirements
f273a92 Add help string for cfapi opts
318941f Updated from global requirements
612a253 Heat stack deletion for HOT/TOSCA packages was fixed
bae0639 [Murano Docs] Install murano-agent by cloud-init
1a94cd0 [Murano Docs] Manage environment templates
960297e Add documentation about bundles
6136b8d Define context.roles with base class
37eba4d [docs] Add stable branch backporting documentation
b43be07 Updated from global requirements
004998f [docs] Documentation about using Glare Artifact Repository
911a026 [docs] Fixed cfapi CLI command sections
51de28f Track status report timestamp
7a51f3a Fix Heat Resource Leak in LinuxMuranoInstance
73370b6 Fix links for samples in heat-transtator plugin
855e5f0 Destroy orphan objects
699aa62 Don't use bash eval for Linux.runCommand
8e29b34 Add note about big files download
6faf217 [Murano Docs] Set environment variables
d7666cf Fix case-sensitive filter value in cfapi
17443fa Docs about murano applications debugging
54e115c Do not upgrade packages with cloud-init
1b847ff Fix typos in Murano files
3c6440a Clean up the configure network doc
3ae3ba5 Revert "Use unified retrying decorator"
275ee88 Use unified retrying decorator
595798b Document changes in Dynamic UI
b92b7e6 Adds a test for logo validation
9dba63e Handling of ephemeral methods was fixed
41be239 Removed warnings from building docs
3337d28 [doc] Fix typo in directory name
5e6a6c3 [doc] Fix formatting for argument
ad5afaa Also package murano_tempest_tests
b5a1245 Remove unnecessary packages.rst
e702315 [doc] Fix malformed table error.
5c3f6b2 Use _LW for deprecation warning
32fc6d8 Update reno for stable/mitaka
49893c8 Fix error on deleting Magnum App
Diffstat (except docs and test files)
-------------------------------------
.gitignore | 2 +
.gitreview | 1 +
README.rst | 2 +-
.../cloudify_tosca_package.py | 10 +-
.../nodecellar_example_application/README.rst | 2 +-
contrib/plugins/cloudify_plugin/requirements.txt | 2 +-
.../Classes/MagnumBayApp.yaml | 90 +++
.../Classes/MagnumBaymodel.yaml | 131 +++++
.../com.intel.magnum.plugin.MagnumApp/UI/ui.yaml | 212 +++++++
.../com.intel.magnum.plugin.MagnumApp/logo.png | Bin 0 -> 8767 bytes
.../manifest.yaml | 26 +
.../Classes/MagnumBayApp.yaml | 83 ---
.../Classes/MagnumBaymodel.yaml | 131 -----
.../magnum-app/io.murano.apps.MagnumApp/UI/ui.yaml | 212 -------
.../magnum-app/io.murano.apps.MagnumApp/logo.png | Bin 8767 -> 0 bytes
.../io.murano.apps.MagnumApp/manifest.yaml | 26 -
.../magnum_plugin/magnum_plugin/__init__.py | 28 +-
contrib/plugins/magnum_plugin/requirements.txt | 2 +-
contrib/plugins/magnum_plugin/setup.cfg | 1 -
.../Classes/ImageValidatorMixin.yaml | 1 +
.../murano_exampleplugin/__init__.py | 2 +-
.../plugins/murano_exampleplugin/requirements.txt | 2 +-
contrib/plugins/murano_exampleplugin/setup.cfg | 1 -
.../plugin/csar_package.py | 14 +-
.../murano_heat-translator_plugin/requirements.txt | 2 +-
.../sample/hello_world/README.rst | 2 +-
.../sample/wordpress/README.rst | 2 +-
.../murano_heat-translator_plugin/setup.cfg | 1 -
devstack/README.rst | 10 +
devstack/plugin.sh | 253 ++++++--
devstack/settings | 8 +-
.../app_migrating/app_migrate_to_juno.rst | 103 ++++
.../app_migrating/app_migrate_to_kilo.rst | 133 +++++
.../app_migrating/app_migrate_to_liberty.rst | 274 +++++++++
.../appdev-guide/cinder_volume_supporting.rst | 153 +++++
.../appdev-guide/hotpackages/chef_server.png | Bin 0 -> 33595 bytes
.../appdev-guide/hotpackages/chef_server_form.png | Bin 0 -> 52798 bytes
.../appdev-guide/muranopackages/dynamic_ui.rst | 418 +++++++++++++
.../muranopackages/package_structure.rst | 41 ++
.../appdev-guide/muranopackages/repository.rst | 31 +
.../appdev-guide/muranopackages/structure.png | Bin 0 -> 114003 bytes
.../appdev-guide/muranopackages/structure.vdx | 1 +
.../draft/admin-guide/admin_troubleshooting.rst | 4 +-
.../configure_cloud_foundry_service_broker.rst | 93 ++-
.../admin-guide/deploy_murano/configure_ssl.rst | 1 +
.../draft/admin-guide/deploy_murano/devstack.rst | 12 +-
.../admin-guide/deploy_murano/install_manually.rst | 99 ++--
.../admin-guide/deploy_murano/prerequisites.rst | 178 ++++++
.../draft/admin-guide/figures/add-interface.png | Bin 0 -> 75928 bytes
.../admin-guide/figures/network-topology-1.png | Bin 0 -> 23559 bytes
.../admin-guide/figures/network-topology-2.png | Bin 0 -> 29217 bytes
.../policy_enforcement/policy_enf_dev.rst | 16 +-
.../app_migrating/app_migrate_to_juno.rst | 103 ----
.../app_migrating/app_migrate_to_kilo.rst | 133 -----
.../app_migrating/app_migrate_to_liberty.rst | 274 ---------
.../draft/appdev-guide/hotpackages/chef_server.png | Bin 34766 -> 0 bytes
.../appdev-guide/hotpackages/chef_server_form.png | Bin 64980 -> 0 bytes
.../draft/appdev-guide/hotpackages/compose.rst | 86 ---
.../draft/appdev-guide/murano_pl/actions.rst | 77 ---
.../draft/appdev-guide/murano_pl/class_templ.rst | 576 ------------------
.../draft/appdev-guide/murano_pl/core_lib.rst | 323 ----------
.../draft/appdev-guide/murano_pl/metadata.rst | 316 ----------
.../draft/appdev-guide/murano_pl/reflection.rst | 267 ---------
.../draft/appdev-guide/murano_pl/statics.rst | 180 ------
.../appdev-guide/muranopackages/dynamic_ui.rst | 369 ------------
.../muranopackages/package_structure.rst | 41 --
.../appdev-guide/muranopackages/repository.rst | 31 -
.../appdev-guide/muranopackages/structure.png | Bin 114003 -> 0 bytes
.../appdev-guide/muranopackages/structure.vdx | 1 -
.../draft/contributor-guide/dev_guidelines.rst | 26 +
.../draft/contributor-guide/stable_branches.rst | 66 +++
.../draft/enduser-guide/deploying_using_cli.rst | 183 ------
.../draft/enduser-guide/figures/add_key_pair.png | Bin 68313 -> 0 bytes
.../draft/enduser-guide/figures/add_pkg_info.png | Bin 73575 -> 0 bytes
.../figures/add_to_env/add_component.png | Bin 75227 -> 0 bytes
.../figures/add_to_env/add_from_cat.png | Bin 158972 -> 0 bytes
.../figures/add_to_env/add_more_apps.png | Bin 19543 -> 0 bytes
.../figures/add_to_env/add_to_env.png | Bin 155363 -> 0 bytes
.../figures/add_to_env/configure_app.png | Bin 31702 -> 0 bytes
.../figures/add_to_env/drag_and_drop.png | Bin 74718 -> 0 bytes
.../figures/add_to_env/quick_deploy.png | Bin 154893 -> 0 bytes
.../enduser-guide/figures/add_to_env/quick_env.png | Bin 88719 -> 0 bytes
.../draft/enduser-guide/figures/app_category.png | Bin 145339 -> 0 bytes
.../draft/enduser-guide/figures/app_details.png | Bin 49342 -> 0 bytes
.../draft/enduser-guide/figures/app_filter.png | Bin 79080 -> 0 bytes
.../enduser-guide/figures/app_filter_example.png | Bin 89056 -> 0 bytes
.../draft/enduser-guide/figures/app_logs.png | Bin 45705 -> 0 bytes
.../enduser-guide/figures/browse_zip_file.png | Bin 36614 -> 0 bytes
.../draft/enduser-guide/figures/bundle_name.png | Bin 37371 -> 0 bytes
.../enduser-guide/figures/component-details.png | Bin 50710 -> 0 bytes
.../enduser-guide/figures/delete_application.png | Bin 79716 -> 0 bytes
.../draft/enduser-guide/figures/deploy_env.png | Bin 79753 -> 0 bytes
.../draft/enduser-guide/figures/deploy_env_2.png | Bin 46702 -> 0 bytes
.../enduser-guide/figures/env-component-logs.png | Bin 48071 -> 0 bytes
.../enduser-guide/figures/env_default_network.png | Bin 33111 -> 0 bytes
.../draft/enduser-guide/figures/environments.png | Bin 42964 -> 0 bytes
.../draft/enduser-guide/figures/import_bundle.png | Bin 34674 -> 0 bytes
.../draft/enduser-guide/figures/import_package.png | Bin 34741 -> 0 bytes
.../draft/enduser-guide/figures/murano_actions.png | Bin 115870 -> 0 bytes
.../enduser-guide/figures/qs_app_category.png | Bin 29792 -> 0 bytes
.../enduser-guide/figures/qs_package_details.png | Bin 72403 -> 0 bytes
.../enduser-guide/figures/qs_package_import.png | Bin 53518 -> 0 bytes
.../draft/enduser-guide/figures/qs_package_url.png | Bin 31351 -> 0 bytes
.../enduser-guide/figures/qs_quick_deploy.png | Bin 39992 -> 0 bytes
.../enduser-guide/figures/qs_quick_deploy_2.png | Bin 75803 -> 0 bytes
.../draft/enduser-guide/figures/qs_quick_env.png | Bin 81441 -> 0 bytes
.../draft/enduser-guide/figures/repository.png | Bin 41589 -> 0 bytes
.../enduser-guide/figures/select_packages.png | Bin 110216 -> 0 bytes
.../enduser-guide/figures/topology_element_1.png | Bin 3523 -> 0 bytes
.../enduser-guide/figures/topology_element_2.png | Bin 3739 -> 0 bytes
.../enduser-guide/figures/topology_kubernetes.png | Bin 123355 -> 0 bytes
.../enduser-guide/figures/topology_wordpress.png | Bin 88608 -> 0 bytes
.../enduser-guide/log_in_to_murano_instance.rst | 59 --
.../draft/enduser-guide/manage_applications.rst | 586 -------------------
.../draft/enduser-guide/manage_environments.rst | 111 ----
.../figures/add_to_env/add_component.png | Bin 0 -> 74659 bytes
.../figures/add_to_env/add_from_cat.png | Bin 0 -> 116546 bytes
.../figures/add_to_env/add_more_apps.png | Bin 0 -> 38825 bytes
.../figures/add_to_env/add_to_env.png | Bin 0 -> 114816 bytes
.../figures/add_to_env/configure_app.png | Bin 0 -> 31702 bytes
.../figures/add_to_env/drag_and_drop.png | Bin 0 -> 74752 bytes
.../figures/add_to_env/quick_deploy.png | Bin 0 -> 115092 bytes
.../enduser-guide/figures/add_to_env/quick_env.png | Bin 0 -> 88183 bytes
.../enduser-guide/figures/app_filter_example.png | Bin 0 -> 92015 bytes
.../enduser-guide/figures/browse_zip_file.png | Bin 0 -> 36614 bytes
.../enduser-guide/figures/component-details.png | Bin 0 -> 58897 bytes
.../enduser-guide/figures/delete_application.png | Bin 0 -> 77180 bytes
.../enduser-guide/figures/env-component-logs.png | Bin 0 -> 54181 bytes
.../enduser-guide/figures/env_default_network.png | Bin 0 -> 38814 bytes
.../enduser-guide/figures/import_package.png | Bin 0 -> 40224 bytes
.../enduser-guide/figures/murano_actions.png | Bin 0 -> 96061 bytes
.../enduser-guide/figures/qs_app_category.png | Bin 0 -> 29792 bytes
.../enduser-guide/figures/qs_package_details.png | Bin 0 -> 72403 bytes
.../enduser-guide/figures/qs_package_import.png | Bin 0 -> 82187 bytes
.../enduser-guide/figures/qs_package_url.png | Bin 0 -> 31351 bytes
.../enduser-guide/figures/qs_quick_deploy.png | Bin 0 -> 39992 bytes
.../enduser-guide/figures/qs_quick_deploy_2.png | Bin 0 -> 75803 bytes
.../enduser-guide/figures/select_packages.png | Bin 0 -> 138633 bytes
.../enduser-guide/figures/topology_element_1.png | Bin 0 -> 3523 bytes
.../enduser-guide/figures/topology_element_2.png | Bin 0 -> 3739 bytes
.../enduser-guide/figures/topology_kubernetes.png | Bin 0 -> 135930 bytes
.../enduser-guide/figures/topology_wordpress.png | Bin 0 -> 87725 bytes
.../enduser-guide/log_in_to_murano_instance.rst | 59 ++
etc/murano/logging.conf.sample | 2 +-
etc/murano/murano-cfapi-paste.ini | 30 +
etc/murano/murano-paste.ini | 6 -
etc/oslo-config-generator/murano-cfapi.conf | 6 +
meta/io.murano.applications/Classes/baseapps.yaml | 85 +++
meta/io.murano.applications/Classes/component.yaml | 440 ++++++++++++++
meta/io.murano.applications/Classes/events.yaml | 104 ++++
.../Classes/replication.yaml | 218 +++++++
meta/io.murano.applications/Classes/servers.yaml | 239 ++++++++
meta/io.murano.applications/manifest.yaml | 59 ++
meta/io.murano/Classes/CloudRegion.yaml | 57 ++
meta/io.murano/Classes/CloudResource.yaml | 33 ++
meta/io.murano/Classes/Environment.yaml | 87 +--
meta/io.murano/Classes/SharedIp.yaml | 33 +-
meta/io.murano/Classes/configuration/Linux.yaml | 17 +-
meta/io.murano/Classes/metadata/Description.yaml | 22 +
meta/io.murano/Classes/metadata/HelpText.yaml | 22 +
meta/io.murano/Classes/metadata/ModelBuilder.yaml | 24 +
meta/io.murano/Classes/metadata/Title.yaml | 22 +
.../Classes/metadata/engine/Serialize.yaml | 25 +
.../Classes/metadata/engine/Synchronize.yaml | 27 +
meta/io.murano/Classes/metadata/forms/Hidden.yaml | 24 +
.../io.murano/Classes/metadata/forms/Position.yaml | 28 +
meta/io.murano/Classes/metadata/forms/Section.yaml | 32 +
meta/io.murano/Classes/resources/CinderVolume.yaml | 19 +-
.../Classes/resources/ConfLangInstance.yaml | 13 +-
.../Classes/resources/ExistingNeutronNetwork.yaml | 14 +-
.../Classes/resources/HeatSWConfigInstance.yaml | 11 +-
meta/io.murano/Classes/resources/Instance.yaml | 103 ++--
.../io.murano/Classes/resources/LinuxInstance.yaml | 4 +-
.../Classes/resources/LinuxMuranoInstance.yaml | 33 +-
meta/io.murano/Classes/resources/Network.yaml | 12 +-
.../Classes/resources/NeutronNetwork.yaml | 53 +-
.../Classes/resources/NeutronNetworkBase.yaml | 21 +-
meta/io.murano/Classes/resources/NovaNetwork.yaml | 5 +-
meta/io.murano/Classes/resources/Volume.yaml | 2 +
.../Classes/system/AwsSecurityGroupManager.yaml | 13 +-
.../Classes/system/DummySecurityGroupManager.yaml | 23 +
.../system/NeutronSecurityGroupManager.yaml | 14 +-
.../Classes/system/SecurityGroupManager.yaml | 7 +-
meta/io.murano/Classes/test/TestFixture.yaml | 60 ++
meta/io.murano/Resources/Agent-v2.template | 5 +-
meta/io.murano/Resources/RunCommand.template | 15 +-
meta/io.murano/Resources/conflang.conf | 2 -
meta/io.murano/Resources/linux-init.sh | 7 +
meta/io.murano/Resources/murano-agent | 8 +-
meta/io.murano/Resources/murano-agent.conf | 2 +-
meta/io.murano/Resources/murano-agent.service | 2 +-
meta/io.murano/Resources/murano-init.conf | 4 +-
meta/io.murano/Resources/murano-init.sh | 35 +-
meta/io.murano/Resources/scripts/runCommand.sh | 14 -
meta/io.murano/Resources/windows-init.ps1 | 17 +-
meta/io.murano/manifest.yaml | 18 +-
murano/api/middleware/context.py | 4 +
murano/api/middleware/ext_context.py | 11 +-
murano/api/middleware/fault.py | 4 +-
murano/api/middleware/ssl.py | 4 +-
murano/api/middleware/version_negotiation.py | 10 +-
murano/api/v1/__init__.py | 9 +-
murano/api/v1/catalog.py | 104 +++-
murano/api/v1/cloudfoundry/__init__.py | 0
murano/api/v1/cloudfoundry/cfapi.py | 328 -----------
murano/api/v1/cloudfoundry/router.py | 57 --
murano/api/v1/environments.py | 2 +-
murano/api/v1/router.py | 21 +
murano/api/v1/schemas.py | 97 ++-
murano/api/v1/static_actions.py | 75 +++
murano/api/v1/template_applications.py | 26 +-
murano/api/v1/templates.py | 34 +-
murano/api/v1/validation_schemas.py | 74 +++
murano/api/versions.py | 4 -
murano/cfapi/__init__.py | 0
murano/cfapi/cfapi.py | 339 +++++++++++
murano/cfapi/router.py | 57 ++
murano/cmd/api.py | 9 +-
murano/cmd/cfapi.py | 6 +-
murano/cmd/cfapi_db_manage.py | 80 +++
murano/cmd/engine.py | 2 +-
murano/cmd/manage.py | 1 -
murano/cmd/test_runner.py | 87 ++-
murano/common/app_loader.py | 8 +-
murano/common/auth_utils.py | 127 ++--
murano/common/cf_config.py | 76 +++
murano/common/config.py | 24 +-
murano/common/engine.py | 169 ++++--
murano/common/helpers/token_sanitizer.py | 7 +-
murano/common/messaging/mqclient.py | 12 +-
murano/common/plugins/extensions_loader.py | 5 +-
murano/common/rpc.py | 13 +
murano/common/server.py | 16 +-
murano/common/statservice.py | 3 +-
murano/common/utils.py | 119 ++--
murano/common/wsgi.py | 34 +-
murano/context.py | 9 +-
murano/db/catalog/api.py | 10 +-
murano/db/cfapi_migration/__init__.py | 0
murano/db/cfapi_migration/alembic.ini | 54 ++
.../db/cfapi_migration/alembic_migrations/README | 15 +
.../db/cfapi_migration/alembic_migrations/env.py | 48 ++
.../alembic_migrations/script.py.mako | 37 ++
.../versions/001_initial_version.py | 67 +++
murano/db/cfapi_migration/migration.py | 86 +++
murano/db/cfapi_models.py | 61 ++
.../versions/005_environment-template.py | 7 +-
.../versions/013_increase_description_text_size.py | 61 ++
.../014_increase_status_time_resolution.py | 61 ++
.../versions/015_adding_text_description.py | 47 ++
.../016_increase_task_description_text_size.py | 49 ++
murano/db/migration/migration.py | 1 +
murano/db/models.py | 2 +
murano/db/services/cf_connections.py | 2 +-
murano/db/services/core_services.py | 48 +-
murano/db/services/environments.py | 6 +
murano/db/services/sessions.py | 8 +-
murano/db/session.py | 6 +-
murano/dsl/attribute_store.py | 33 +-
murano/dsl/constants.py | 11 +-
murano/dsl/contracts/__init__.py | 42 ++
murano/dsl/contracts/basic.py | 129 ++++
murano/dsl/contracts/check.py | 139 +++++
murano/dsl/contracts/contracts.py | 303 ++++++++++
murano/dsl/contracts/instances.py | 240 ++++++++
murano/dsl/dsl.py | 162 +++---
murano/dsl/dsl_types.py | 30 +-
murano/dsl/exceptions.py | 10 +
murano/dsl/executor.py | 314 +++++++---
murano/dsl/helpers.py | 312 +++++++---
murano/dsl/lhs_expression.py | 281 ++++-----
murano/dsl/meta.py | 45 +-
murano/dsl/murano_method.py | 97 ++-
murano/dsl/murano_object.py | 375 ++++++++----
murano/dsl/murano_package.py | 22 +-
murano/dsl/murano_property.py | 15 +-
murano/dsl/murano_type.py | 126 ++--
murano/dsl/object_store.py | 326 ++++++++---
murano/dsl/package_loader.py | 12 +
murano/dsl/principal_objects/__init__.py | 2 +
murano/dsl/principal_objects/garbage_collector.py | 85 +++
murano/dsl/principal_objects/stack_trace.py | 4 +-
murano/dsl/principal_objects/sys_object.py | 4 +-
murano/dsl/reflection.py | 17 +-
murano/dsl/schema_generator.py | 283 +++++++++
murano/dsl/serializer.py | 151 ++++-
murano/dsl/type_scheme.py | 389 -------------
murano/dsl/typespec.py | 55 +-
murano/dsl/yaql_functions.py | 90 ++-
murano/dsl/yaql_integration.py | 81 +--
murano/engine/mock_context_manager.py | 7 +-
murano/engine/package_loader.py | 83 ++-
murano/engine/system/agent.py | 41 +-
murano/engine/system/agent_listener.py | 12 +-
murano/engine/system/common.py | 17 +-
murano/engine/system/heat_stack.py | 116 ++--
murano/engine/system/logger.py | 7 +-
murano/engine/system/mistralclient.py | 116 ----
murano/engine/system/net_explorer.py | 18 +-
murano/engine/system/resource_manager.py | 9 +-
murano/engine/system/status_reporter.py | 4 +-
murano/engine/system/system_objects.py | 4 +-
murano/engine/system/test_fixture.py | 5 -
murano/engine/system/workflowclient.py | 122 ++++
murano/engine/system/yaql_functions.py | 26 +-
murano/engine/yaql_yaml_loader.py | 6 +-
murano/locale/murano-log-error.pot | 117 ----
murano/locale/murano-log-info.pot | 63 --
murano/locale/murano-log-warning.pot | 105 ----
murano/locale/murano.pot | 440 --------------
murano/locale/ru/LC_MESSAGES/murano-log-error.po | 14 +-
murano/locale/ru/LC_MESSAGES/murano-log-info.po | 8 +-
murano/locale/ru/LC_MESSAGES/murano-log-warning.po | 8 +-
murano/locale/ru/LC_MESSAGES/murano.po | 38 +-
murano/opts.py | 15 +
murano/packages/exceptions.py | 5 +-
murano/packages/hot_package.py | 15 +-
murano/packages/load_utils.py | 2 +-
murano/packages/package_base.py | 1 +
murano/policy/congress_rules.py | 21 +-
murano/policy/model_policy_enforcer.py | 17 +-
murano/services/actions.py | 4 +-
murano/services/static_actions.py | 36 ++
.../engine/test_conf_language_support.py | 8 +-
.../unit/db/migration/test_migrations_base.py | 23 +-
.../unit/db/services/test_templates_service.py | 4 +-
.../unit/dsl/foundation/test_package_loader.py | 11 +-
.../unit/dsl/meta/TestMethodParamInheritance.yaml | 30 +
.../unit/dsl/test_method_param_inheritance.py | 35 ++
.../unit/engine/system/test_garbage_collector.py | 79 +++
.../test.mpl.v1.app/manifest_with_broken_logo.yaml | 19 +
.../test.mpl.v1.app/test_logo.png.not_valid | Bin 0 -> 210 bytes
.../policy/modify/actions/test_action_manager.py | 10 +-
.../policy/modify/actions/test_default_actions.py | 109 ++--
.../unit/policy/test_model_policy_enforcer.py | 18 +-
murano/utils.py | 1 +
.../extras/MockApp/Classes/mock_muranopl.yaml | 30 +-
.../Classes/ApacheHttpServer.yaml | 81 +++
.../Resources/index.html | 8 +
.../manifest.yaml | 28 +
.../Classes/Lighttpd.yaml | 55 ++
.../Resources/DeployLighttpd.template | 27 +
.../Resources/scripts/deployLighttpd.sh | 14 +
.../io.murano.apps.test.Lighttpd/manifest.yaml | 24 +
.../Classes/UpdateExecutor.yaml | 47 ++
.../Resources/scripts/update.sh | 14 +
.../manifest.yaml | 22 +
.../application_catalog_client.py | 30 +-
.../services/artifacts/__init__.py | 0
.../services/artifacts/artifacts_client.py | 158 +++++
.../api/application_catalog/artifacts/__init__.py | 0
.../api/application_catalog/artifacts/base.py | 119 ++++
.../artifacts/test_repository.py | 54 ++
.../artifacts/test_versioning.py | 164 ++++++
.../artifacts/test_versioning_negative.py | 97 +++
.../api/application_catalog/test_categories.py | 11 +-
.../test_categories_negative.py | 10 +-
.../api/application_catalog/test_env_templates.py | 37 +-
.../test_env_templates_negative.py | 21 +-
.../api/application_catalog/test_environments.py | 12 +-
.../test_environments_negative.py | 15 +-
.../api/application_catalog/test_repository.py | 38 +-
.../test_repository_negative.py | 43 +-
.../application_catalog/test_services_negative.py | 29 +-
.../application_catalog/test_sessions_negative.py | 21 +-
.../api/application_catalog/test_static_actions.py | 75 +++
.../test_static_actions_negative.py | 108 ++++
.../service_broker/test_service_broker_actions.py | 17 +-
.../service_broker/test_service_broker_negative.py | 6 +-
.../application_catalog/test_cinder_volumes.py | 485 +++++++++++++++
.../application_catalog/test_deployment.py | 251 ++++++++
.../notes/action-syntax-3f2cbe843801f80d.yaml | 7 +
.../add_api_in_operator-371e3a1d2aec6421.yaml | 8 +
...dd_timeout_to_linux_class-05d1f573a883f3ce.yaml | 4 +
.../better-detect-agent-9ef8892a4bfb72cd.yaml | 7 +
.../notes/cinder-volumes-0412875c1011f8eb.yaml | 4 +-
.../config-network-driver-77c82d151dead620.yaml | 6 +
...elete-app-in-env-template-d8e07d3b860f0441.yaml | 3 +
.../enable-hot-for-glare-8026f2dccad1732e.yaml | 6 +
...d-adding_text_description-25bd77f36ee370ba.yaml | 5 +
.../notes/garbage-collection-50e78c4c9d47eba6.yaml | 11 +
.../notes/gc-collect-165e73bbaf345d74.yaml | 13 +
.../gc-isdoomed-isdestroyed-9598a6e15dbf36a0.yaml | 8 +
.../notes/keystone-v3-0e287679f7f40a2a.yaml | 8 +
.../linux-helpers-async-243fc1adbbe5c512.yaml | 4 +
.../notes/meta-for-ui-72f5b58c6d17599f.yaml | 12 +
.../notes/model-load-c1eb24843d30e414.yaml | 9 +
.../multi-regional-apps-b64afbaeafd5b9c5.yaml | 16 +
.../multiple-api-workers-60492ddc2e3ff0aa.yaml | 9 +
...object-interface-equality-9fc8048be61bd539.yaml | 4 +
.../notes/muranopl-forms-4a3fb8153f26bbcf.yaml | 8 +
.../new-contract-framework-1dede2d16b2e9c71.yaml | 38 ++
...objects-resource-leak-fix-33a2eca3a4ccb8af.yaml | 4 +
...pe-format-in-object-model-da6976291057ab31.yaml | 10 +
...neutron-sec-group-support-2d69082b7226d6c0.yaml | 5 +
...bjects-copy-objects-merge-8f2752b1a1a18af0.yaml | 18 +
.../safeloader-cve-2016-4972-19035a2a091ec30a.yaml | 9 +
.../script-line-endings-db632db9e24237a3.yaml | 6 +
...ervice-broker-from-murano-f6ee48576f51d893.yaml | 4 +
.../spec-semver-library-436b0db35fbd4c37.yaml | 5 +
.../notes/static-actions-61759be796299039.yaml | 7 +
.../notes/string-logging-20b8e60a957ba6b7.yaml | 5 +
.../notes/template-contract-b71840cbc35eb478.yaml | 11 +
.../test-runner-output-fix-e942e221be189424.yaml | 6 +
...t-runner-set-up-tear-down-a269a31734544a3a.yaml | 4 +
...two-phase-instance-deploy-81d37e7987abc792.yaml | 5 +
...pdate-app-in-env-template-08d92b22bd1355f5.yaml | 4 +
.../notes/var-kw-args-c42c31678d8bc747.yaml | 11 +
releasenotes/source/index.rst | 3 +-
releasenotes/source/mitaka.rst | 6 +
requirements.txt | 42 +-
setup.cfg | 15 +-
test-requirements.txt | 16 +-
tools/cover.sh | 8 +
tools/lintstack.py | 4 +-
tox.ini | 19 +-
618 files changed, 23500 insertions(+), 12739 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index c616a3f..25a6576 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -5 +5 @@ pbr>=1.6 # Apache-2.0
-Babel>=1.3 # BSD
+Babel>=2.3.4 # BSD
@@ -7,2 +7,2 @@ SQLAlchemy<1.1.0,>=1.0.10 # MIT
-stevedore>=1.5.0 # Apache-2.0
-alembic>=0.8.0 # MIT
+stevedore>=1.16.0 # Apache-2.0
+alembic>=0.8.4 # MIT
@@ -11,2 +11,2 @@ PasteDeploy>=1.5.0 # MIT
-Routes!=2.0,!=2.1,>=1.12.3;python_version=='2.7' # MIT
-Routes!=2.0,>=1.12.3;python_version!='2.7' # MIT
+Routes!=2.0,!=2.1,!=2.3.0,>=1.12.3;python_version=='2.7' # MIT
+Routes!=2.0,!=2.3.0,>=1.12.3;python_version!='2.7' # MIT
@@ -18 +18 @@ six>=1.9.0 # MIT
-netaddr!=0.7.16,>=0.7.12 # BSD
+netaddr!=0.7.16,>=0.7.13 # BSD
@@ -21 +21,2 @@ jsonpatch>=1.1 # BSD
-keystonemiddleware!=4.1.0,>=4.0.0 # Apache-2.0
+keystoneauth1>=2.10.0 # Apache-2.0
+keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0
@@ -23,0 +25 @@ yaql>=1.1.0 # Apache 2.0 License
+debtcollector>=1.2.0 # Apache-2.0
@@ -29,3 +31,3 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
-python-heatclient>=0.6.0 # Apache-2.0
-python-neutronclient!=4.1.0,>=2.6.0 # Apache-2.0
+python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
+python-heatclient>=1.4.0 # Apache-2.0
+python-neutronclient>=5.1.0 # Apache-2.0
@@ -33,8 +35,8 @@ python-muranoclient>=0.8.2 # Apache-2.0
-python-congressclient<2000,>=1.0.0 # Apache-2.0
-python-mistralclient>=1.0.0 # Apache-2.0
-oslo.db>=4.1.0 # Apache-2.0
-oslo.config>=3.7.0 # Apache-2.0
-oslo.concurrency>=3.5.0 # Apache-2.0
-oslo.context>=0.2.0 # Apache-2.0
-oslo.policy>=0.5.0 # Apache-2.0
-oslo.messaging>=4.0.0 # Apache-2.0
+python-congressclient<2000,>=1.3.0 # Apache-2.0
+python-mistralclient>=2.0.0 # Apache-2.0
+oslo.db!=4.13.1,!=4.13.2,>=4.10.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.concurrency>=3.8.0 # Apache-2.0
+oslo.context>=2.9.0 # Apache-2.0
+oslo.policy>=1.9.0 # Apache-2.0
+oslo.messaging>=5.2.0 # Apache-2.0
@@ -43,2 +45,2 @@ oslo.serialization>=1.10.0 # Apache-2.0
-oslo.service>=1.0.0 # Apache-2.0
-oslo.utils>=3.5.0 # Apache-2.0
+oslo.service>=1.10.0 # Apache-2.0
+oslo.utils>=3.16.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index d59b5b0..8944542 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -7,2 +7,2 @@ coverage>=3.6 # Apache-2.0
-fixtures>=1.3.1 # Apache-2.0/BSD
-mock>=1.2 # BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
+mock>=2.0 # BSD
@@ -12 +11,0 @@ sqlalchemy-migrate>=0.9.6 # Apache-2.0
-tempest-lib>=0.14.0 # Apache-2.0
@@ -17,2 +16,3 @@ unittest2 # BSD
-pylint==1.4.5 # GNU GPL v2
-requests!=2.9.0,>=2.8.1 # Apache-2.0
+pylint==1.4.5 # GPLv2
+requests>=2.10.0 # Apache-2.0
+os-testr>=0.7.0 # Apache-2.0
@@ -21 +21 @@ requests!=2.9.0,>=2.8.1 # Apache-2.0
-PyMySQL>=0.6.2 # MIT License
+PyMySQL!=0.7.7,>=0.6.2 # MIT License
@@ -26 +26 @@ oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
@@ -28 +28 @@ sphinxcontrib-httpdomain # BSD
-reno>=0.1.1 # Apache2
+reno>=1.8.0 # Apache2
1
0
06 Oct '16
We are satisfied to announce the release of:
sahara-image-elements 5.0.0: Image elements for Sahara
This release is part of the newton release series.
For more details, please see below.
Changes in sahara-image-elements 4.0.0.0rc1..5.0.0
--------------------------------------------------
e990d78 Update .gitreview for stable/newton
42c4a37 Re-adding mapr-eco-repository repo
5dce4ca [DOC] fix doc about hdp local mirror
02600e0 update zookeeper download link
a155e16 [HDP] Fix HDP image for deploy on ubuntu
0d1739f Bump version of ambari
b7d83da Merge Vanilla and Spark plugins
7998195 Fix incorrect ambari repo link
e568680 Add support for building MapR 5.2 prebuilt images
253376c add element for kerberos artifacts
a0f8b24 use trusty ubuntu for ambari images
feff7a5 Adding ability to create CentOS7 images with MapR
ec50d50 drop vanilla 2.6.0 support from elements
cb60bb9 Add Kafka 2.0.2 to CDH 5.5 and 5.7
347b0a0 improve building images in gate for vanilla
6010b9a Setup JAVA_HOME for cloudera java distro
3b3559f Install Pyleus to Storm image
038ea3a [CDH] Support CDH 5.7
70040a0 [HDP2.4] Add ambari 2.2.1.0 element
08e4221 Fixing diskimage-create.sh README.rst
d55baf9 Fix Mapr security repo
bf7b64f replace extjs location to sahara-files
ac40a25 Allow creation of Storm 1.0.1 images
497e4bb remove hdp from elements
9223cb9 build all images for ambari in gate
e032435 Add support for building Centos 7 images for CDH 5.5
ff42411 fix gate jobs for building centos images
8ba8ee6 ignore safe sudo check
b363dc2 Change url link to Zookeeper package
bc56576 Add check Spark version
d560394 Add hadoop openstack swift jar to plugins images
Diffstat (except docs and test files)
-------------------------------------
.gitreview | 1 +
diskimage-create/README.rst | 102 ++++++---
diskimage-create/diskimage-create.sh | 246 ++++++++++++---------
elements/ambari/README.rst | 4 +-
elements/ambari/package-installs.yaml | 1 +
elements/ambari/pre-install.d/90-ambari-repo-conf | 4 +-
elements/extjs/install.d/80-install-extjs | 2 +-
elements/extjs/root.d/50-download-extjs | 2 +-
elements/hadoop-cloudera/README.rst | 9 +-
elements/hadoop-cloudera/element-deps | 1 +
.../environment.d/01-set-java-distro | 1 +
elements/hadoop-cloudera/install.d/34-java | 30 +++
.../hadoop-cloudera/install.d/50-install-cloudera | 32 ++-
.../hadoop-cloudera/pre-install.d/10-add-mirror | 106 ++++++++-
elements/hadoop-hdp/README.rst | 37 ----
elements/hadoop-hdp/element-deps | 6 -
elements/hadoop-hdp/install.d/40-install-hdp | 163 --------------
elements/hadoop-hdp/package-installs.yaml | 5 -
.../hadoop-hdp/post-install.d/10-disable-repos | 35 ---
.../hadoop-hdp/post-install.d/40-turn-off-gmetad | 32 ---
elements/hadoop-hdp/post-install.d/50-turn-on-ntpd | 31 ---
.../post-install.d/55-turn-off-ambari-services | 34 ---
.../post-install.d/58-turn-off-hadoop-2-services | 41 ----
.../post-install.d/60-enable-local-hdp-repos | 52 -----
.../hadoop-hdp/source-repository-ambari-updates | 1 -
elements/hadoop-hdp/source-repository-hadoopswift | 2 -
elements/hadoop-hdp/source-repository-hdputils | 1 -
elements/hadoop-mapr/README.rst | 10 +-
.../hadoop-mapr/install.d/40-mapr-dependencies | 4 +-
elements/hadoop-mapr/install.d/41-scala | 2 +-
.../hadoop-mapr/install.d/43-mapr-core-repository | 2 +-
.../hadoop-mapr/install.d/44-mapr-eco-repository | 15 +-
elements/hadoop-mapr/post-install.d/99-requiretty | 2 +-
.../post-install.d/99-update-repository | 2 +-
elements/hadoop-mapr/resources/package_utils.sh | 10 +-
elements/hadoop-mapr/resources/spec_5.2.0.json | 41 ++++
.../hadoop-mapr/resources/spec_5.2.0_ubuntu.json | 38 ++++
elements/hadoop/README.rst | 10 +-
elements/hadoop/install.d/40-setup-hadoop | 3 -
elements/hdp-local-mirror/README.rst | 2 +-
elements/kdc/README.rst | 13 ++
elements/kdc/element-deps | 1 +
elements/kdc/package-installs.yaml | 7 +
elements/kdc/pkg-map | 18 ++
.../pre-install.d/76-unlimited-security-artifacts | 12 +
elements/oozie/install.d/50-setup-oozie | 8 +-
elements/oozie/root.d/0-check-oozie | 7 +-
elements/oozie/root.d/50-download-oozie | 5 +-
elements/spark/README.rst | 18 +-
elements/spark/post-install.d/20-spark | 15 +-
elements/spark/root.d/0-check-spark | 8 +-
elements/spark/root.d/50-download-spark | 40 ++--
elements/storm/install.d/60-storm | 18 +-
elements/storm/root.d/50-download-storm | 5 +
elements/swift_hadoop/README.rst | 2 +-
elements/swift_hadoop/post-install.d/81-add-jar | 41 +++-
elements/zookeeper/install.d/40-zookeeper | 8 +-
tools/gate/build-images | 26 ++-
tox.ini | 3 +
59 files changed, 684 insertions(+), 693 deletions(-)
1
0
06 Oct '16
We are delighted to announce the release of:
searchlight 1.0.0: OpenStack Search Service
This release is part of the newton release series.
For more details, please see below.
1.0.0
^^^^^
The initial mitaka release (0.2) did not support ElasticSearch 2.x.
ElasticSearch made several changes to internal index data mappings
that were not backwards compatible with ElasticSearch 1.x. This
release has all the changes required for Searchlight to support both
ElasticSearch 1.x and 2.x.
Searchlight now supports oslo pools instead of requiring a different
topic for each listener. This simplifies deployment configuration.
Previously, services would have to be specifically configured to send
notifications to the Searchlight topic. With this change, Searchlight
can share topics with other services such as Ceilometer.
Adds ability to include region_name in mapping for multi-region
deployments.
Allow aggregations as part of a search request.
Add new functionality to searchlight-manage to force a re-indexing
from Elasticsearch only. This option allows new mappings to be applied
to the Elasticsearch indices without the overhead of retrieving data
from the plugin APIs.
Allow dynamic index settings to allow Searchlight to be tuned
differently from other indices on a shared Elasticsearch cluster.
Searchlight will provide resource type info for any reference IDs in
the facet results.
Support version as a search param to include _version in search
results.
Adds the ability to attain a document count per resource type from the
/facets endpoint. Doc count is always returned for a type; field
listing is optional.
Added multi-thread support for indexing.
Allow wildcard character as an argument to searchlight-manage index
sync --type
Added support for neutron floating IPs.
Added suppport for neutron security groups and security group rules.
Add support for the new Neutron tenant RBAC policy functionality. This
allows networks to be shared across tenants.
Add support to searchlight-manage for displaying with Elasticsearch
indices/aliases are currently being used by Searchlight.
Reduce the number of calls to the nova API on instance boots.
The ability to distribute resource type indexing across resource
groups has now been fully enabled. This allows for improved indexing
speed and clustering customization.
Adds support for service policy files. Simplifies existing API policy
support.
Added project_id to mappings.
Disable returning options for the facet fields.
Add checks for resource group name configuration.
Correct facet options on analyzed string fields.
Fix the information returned by the API plugins endpoint.
No longer index the data of child plugins as part of indexing a parent
plugin, which fixes ConflictErrors occuring specifying parent plugins
as the argument to searchlight-manage index sync --type.
Fix routing on delete operations for neutron resources.
Remove support for Designate v1 api because it has already been
deprecated. Simplify list zones and recordsets code by using v2 api's
built-in pagination.
Add "updated_at" field to all mappings.
Some Neutron ports that do not send notificatons to Searchlight. Do
not initially index them.
Fix mappings for fields erroneously marked as 'nested', and allow
facet options for 'object' fields.
Improved index sync performance.
Change the way of doing facets search, replace use of
search_type=count with size=0
New Features
************
* Added microversion support for Nova plugins, the default API
version for Nova plugins is 2.1. User can change the API
microversion for Nova using compute_api_version config option added
in section service_credentials:nova.
* Adds nova plugin for server groups.
* Adds configuration option to include region_name in documents to
assist with multi-region tribe node Elasticsearch setups.
* Allows aggregations to be requested (as 'aggs' or 'aggregations'
in a search request body). See
https://www.elastic.co/guide/en/elasticsearch/reference/current
/search-aggregations.html
* Add a new option to the searchlight-manage index command to force
a re-indexing from Elasticsearch only. The plugin APIs will not be
used with option.
* Allows a deployer to override some index-level Elasticsearch
settings for all Searchlight-created indices. This is useful when
running Searchlight on a shared Elasticsearch cluster. Settings are
overriden using the elasticsearch.index_settings config option.
* Searchlight will provide resource type info for any reference IDs
in the facet results. Parent reference types will also be part of
the info This will allow UI's and CLI's to explore the data
seamlessly
* version is supported as a search parameter to return an always-
incrementing _version value with each search result.
* Includes 'doc_count' in /facets response for each resource type.
* Adds nova plugin for flavors.
* Adds nova plugin for hypervisors.
* When searchlight-manage is indexing, all resource types will be
indexed in a separate thread. Multiple threads will be run
concurrently.
* The --type argument to searchlight-manage index sync accepts a
wildcard as the final character.
* Adds neutron floating IP support (OS::Neutron::FloatingIP).
* Added suppport for neutron security groups and security group
rules (OS::Neutron::SecurityGroup).
* Add support for the new Neutron tenant RBAC policy functionality.
This allows networks to be shared across tenants.
* Add support to searchlight-manage for displaying with
Elasticsearch indices/aliases are currently being used by
Searchlight. The indices/aliases are based on the specified
configuration file.
* Replace some of the nova get server API calls with partial
document updates that just update the OS-EXT-STS state fields.
* Reenabled multiple indices by allowing plugins to be managed under
different resource groups.
* API configuration can include service policy files such that
access to various resources can match whether their service API
allows access for a user.
* Openstack is moving away from 'tenant' as the nomenclature for
keystone projects, preferring 'project' instead. Some services
already return tenant_id and project_id (or just project_id); this
patch adds project_id to those that do not.
* Currently the "facet" endpoint will return all information
relating to a facet field, including the options. The new parameter
"exclude_options" to the facet endpoint will prevent the options for
all facet fields from being returned. This option will avoid an
aggregation query being performed on Elasticsearch, improving
performance. The default behavior (not specifying this option) is
return the options for all facet fields.
Known Issues
************
* There are no notifications for server groups from nova yet, so we
recommend putting it in its own resource group and scheduling a cron
job to periodically re-sync. This will create a very low overhead
way to keep the index up to date. The index latency will be
dependent on how often you re-sync the data.
* This is a change that is not compatible with the API response in
Mitaka; changes have been made in Newton to the python client and
UI.
* There are no notifications for flavors from nova yet, so we
recommend putting it to its own resource group and scheduling a cron
job to re-sync with little overhead.
* There are no notifications for hypervisor from nova yet, so we
recommend putting it to its own resource group and scheduling a cron
job to re-sync with little overhead.
* Currently no notifications are sent on status changes for floating
IPs (see https://bugs.launchpad.net/neutron/+bug/1593793)
* Using multiple resource groups will impact sort behavior when
sorting on fields across resource types when all the resource types
don't have the requested 'sort-by field'. Follow the guidelines
below to avoid errors.
https://www.elastic.co/guide/en/elasticsearch/reference/current
/search-request-sort.html#_ignoring_unmapped_fields
https://www.elastic.co/guide/en/elasticsearch/reference/current
/search-request-sort.html#_missing_values
* Deployment should use Designate v2 api. if v1 api is in use,
designate plugin won't function correctly.
Upgrade Notes
*************
* To support ElasticSearch 1.x and 2.x, several internal data
mappings have been updated. If you previously deployed Searchlight,
you will need to reindex your resource data using "searchlight-
manage index sync". You may re-index everything or limit it to
"OS::Nova::Server", "OS::Glance::Image", "OS::Glance::Metadef",
"OS::Cinder:Volume", and "OS::Cinder::Snapshot".
* To start using a pool with a shared topic instead of separate
topics:
* Update the respective service configuration files for
searchlight enabled plugins to only publish to a single topic
(e.g. set "notification_topics = notifications"). If you have
Ceilometer enabled, this must be the same topic which Ceilometer
uses. You also typically must restart each service for the change
to take effect. For example, in "nova.conf", "glance-api.conf",
"cinder.conf", "neutron.conf", and "designate.conf":
notification_topics = notifications
* Update "searchlight.conf". In the "[resource_plugin]" section,
set "notifications_topic" to match the shared topic that you set
in the service configurations. In the "[listener]" sections, set
"notifications_pool" to your desired pool name. The default name
starting in Newton is "searchlight". The "notifications_pool" does
not have to match anything from other services configuration
files. For example:
[resource_plugin]
notifications_topic = notifications
[listener]
notifications_pool = searchlight
* A version of python-searchlightclient greater than 2.1 must be
used with this change, as well as the Newton UI version.
* The existing searchlight policy.json has been simplified. The only
supported options for resource policy are to enable or disable it
for all request types (i.e. the 'allow', 'query' 'facet' suffixes
are no longer in effect).
* Mappings have changed for several types; it is strongly
recommended all data be reindexed and the API service restarted.
Deprecation Notes
*****************
* Two fields returned in the API plugins endpoint 'name' and 'index'
have been removed.
Bug Fixes
*********
* Bug 1570213 Apply query to highlight query
* Bug 1532010 Ensure consistency in mapping field types
* Bug 1570674 Fix unicode error when booting instance from volume
* Bug 1570199 Fix inconsistent mapping in image plugin
* Bug 1568709 Remove port.create.end handler from nova
* Bug 1565015 Add volume.retype event
* Bug 1583215 Correct Cinder exchange value
* Bug 1583215 Enable notification messaging pools
* Overwrite child plugin's resource_group_name during parent
registration. Child plugins must live on the same resource group as
their parents. This fixes a issue that a misconfiguration of child
plugin's resource_group_name results in inconsistentency with the
parent plugin.
* Facet options don't work well with analyzed string fields, because
the aggregation runs on tokens. These fields have non-analyzed raw
subfields for sorting. Include raw subfields in facet results so
that users can use them to do exact matches.
* The API plugins API endpoint is returning outdated, duplicated and
incomplete information. The hat-trick of uselessness. This fix will
correct that. The duplicated field ('name') has been removed. The
outdated field has been renamed (from 'index' to 'alias-search').
The missing field has been added ('alias-listener').
* Bug 1584148 Version conflict during re-indexing
* Fixes a bug where routing wasn't specified for delete operations
on neutron child resources.
* Fixes some compatibility issues around metafield access in es 1.x
versus es 2.x.
* All Elasticsearch mappings need to include the field "udated_at".
This field is used by Searchlight UI to sort the results.
* Searchlight is indexing unused Neutron ports. Some Neutron ports
do not result in notifications to Searchlight. We do not want to
initially index them.
* Bug 1619047 Unnest nova server flavor and image mapping properties
Other Notes
***********
* The maximum number of threads that will run concurrently can be
controlled by setting "max_workers" in the Searchlight configuration
file. [manage] max_workers = 6
* Searchlight uses configured resource group name to create
ElasticSearch indices. Some characters are invalid to index names
and will cause faiure of index creation. Add verification to check
if resource group name is valid.
* In mitaka, indexing a parent plugin also reindexed the data for
any children it had. That is no longer the case.
* Improved searchlight-manage index sync performance by disabling
automatic refresh of newly created indices until they are fully
populated and activated for search queries.
* Facets search is aggregation-only, no actual documents need to be
returned. We do it by setting search_type in search request before,
but it has been deprecated in Elasticsearch 2.x, use recommended
size=0 instead.
Changes in searchlight 0.2.0.0rc1..1.0.0
----------------------------------------
58b7ed1 api-ref: Add Searchlight api-ref
def91d6 Pin tox upper-constraints for stable/newton
79d21a9 Return __searchlight-user-role with highlighting
7fb2e41 RBAC for network sharing
42b7d7b Support X_IS_ADMIN_PROJECT header on auth response
31f19de 'deleted_at' Date field needed for get_version
12950c7 Restrict image member list
54e05ec Do not index some Neutron ports.
6550b19 Cinder mapping incorrect.
99c2e77 Log indexing exceptions at 'exception' level
6eda437 Security group rule race condition
00f7edc Update .gitreview for stable/newton
4ca188e Indicate 'nested' on facet types
87fc192 Disable returning options for the facet fields.
0f9e6f1 Add "updated_at" field to all mappings.
177818b Handle nova error states correctly
241ec28 standardize release note page ordering
3625a7c Allow aggregations in search requests
3f406fa Change Elasticsearch installation doc
3d47b25 Change Designate recordset mapping
ed1bfac Replace some 'nested' mapping types with 'object'
dbdbb67 [api-ref] Remove temporary block code in conf.py
8ff2c39 to_dict instead of __dict__ for cinder resource
0b99d8e Verify resource group name
18b120f Fix exceptions error for image plugin
304362e Fix translation of exceptions
f8f7d24 Use more specific asserts in tests
d8ea63b Revert "Allow aggregations in search requests"
004bfc1 Raise NotImplementedError instead of NotImplemented
58a8870 Add region name to mappings and documents
f8e4e3e Allow aggregations in search requests
fb0676d Config logABug feature for searchlight api-ref
71510e9 Updated from global requirements
afbac8e API plugin endpoint needs aliases
d783c87 TrivialFix: Remove logging import unused
77ac53b Updated from global requirements
0d64da5 Get ready for os-api-ref sphinx theme change
0233200 Updated from global requirements
328f825 Service policy file support
3d7b4a1 Document the Searchlight architecture
535a3da Add nova server groups plugin
26e73de :Move elasticsearch install to plugin, use v2.3.4
63f0f56 Ignore nova deleting state changes
001e471 Force Elasticsearch indexing
1a9e06d Updated from global requirements
f25b28b Fix nova exchange exception
3fd15c8 Add warning for plugins without notifications
f408598 api-ref: add base structure
87fc290 Updated from global requirements
2652ed5 Add totals to facets
194e778 Support 'shelve' and 'reboot' instance notifications
5cda277 Allow wildcard in index --type argument
f7d1a48 Flavor Plugin
ae6a7b7 Log notification events
18c2061 Fixing typo in manage.py
a7e1a8a Remove discover from test-requirements
558a6c5 Display ES indices
72c2b86 Updated from global requirements
c575f3e Speed up Searchlight indexing
4fc6786 Small improvement for dynamic index settings doc
fa70962 Fix --type handling for indexing parent plugins
dcec06c Cleanup i18n marker functions to match Oslo docs
5d1030a Updated from global requirements
056be20 Remove designate v1 artifacts
79b0efd A typo commit in the documentation
7d0146a Add Neutron security group plugin
993811f Add Python 3.5 classifier and venv
771c5e6 Replace assertEqual(None, *) with assertIsNone in tests
89acc42 Fixes: Typo Error in helper.py
01eee1d Correct reraising of exception
6ad19a5 Spelling fixes in README
4fc01db Updated from global requirements
6a87126 Config: no need to set default=None
bee851b Allow dynamic index settings
0e4c767 Reduce number of API calls on nova boot
85680b4 Neutron floating IP support.
fbfef7b Expand designate functional tests
8736baa Fix typo errors in the description
0b6a5c0 make string.letters python3 compatible
32fd04b Handle JSON object in hypervisor serialization
d36e9ed Updated from global requirements
f6fe6fd Updated from global requirements
217480f Enabled Neutron notifications in sample local.conf
a15e3e6 Determine routing before delete on neutron events
21042fd Add microversion support for nova plugins
0074de2 Image type should be a facet
028943d Expose 'version' as a search parameter
33066df Correct doc_values info in authoring plugins docs
2e9bac3 Updated from global requirements
3beda06 Change Nova API version to V2.1
04ec2a1 Add project_id to all serialized documents
3e64068 Fix reindexing conflict handling in es2
e58d3f3 Correct es 2.x functional test config
bf693c5 Imported Translations from Zanata
fdd0c01 Updated from global requirements
94a2660 Return raw subfields on facets of analyzed string fields
1d440ff Updated from global requirements
09b81c4 Don't accept domain-scoped tokens for auth
87ad59b Fix python3 compatible issue in listener
4074746 python 3.4 support for urlparse
a924228 Remove mention of oslo.log 'verbose' option
16446d9 Correct wrong link in doc
787752a Use Notification Pools by Default
ea75c13 Added os_endpoint_type to various OpenStack clients
6798c5d Add Release notes for fixes backported to 0.2.1
942de0a Enable notification messaging pools
83f389a Updated from global requirements
e4878e0 Adds Nova Functional Tests
f2ed51b Correct cinder 'exchange' value
7f56603 Resource types for reference and parent id
75f46cb Replace use of search_type=count with size=0
54a0687 Updated from global requirements
204d7b4 Apply query to highlight query
21b2960 Include zone_id in Designate Recordset mapping
a2dbacc remove verbose option in devstack plugin
31f58bf Simple Script for Generating Resources
ebcb52b Overwrite child plugin resource_group_name durning parent registration
d2b536f Updated from global requirements
4592cdf Improve plugin authoring information
4f56a1e Reenable multiple elasticsearch indices
e55c7ff Updated from global requirements
67354da Ensure consistency in mapping field types
0d927ef Add freeform search syntax to docs
f7f5573 Imported Translations from Zanata
1142b72 Error deleting index in initial syncing
2d1e87a Updated from global requirements
5d9bcba use thread safe fnmatch
83d6e6b Document that initial sync is mandatory
730f9f1 Hypervisor plugin
3bb82e0 Allow limit=0 In Searches
789f39a [Trivial] Remove unnecessary executable privilege
aeb4fc2 Make func test indexing more consistent
101c734 Add cleanup in functional tests
e6f649c Making searchlight-manage python-3 compatible
c7e4fcd Fix unicode error when booting instance from volume
4d06c6a Imported Translations from Zanata
2a16363 Updated from global requirements
ccbb9b2 Add release note for index refresh disabling
81f7956 Zero Downtime Re-indexing Functional Tests.
1e4f0f6 Updated from global requirements
195a1f7 Add volume.retype event
93c6cfa Remove port.create.end handler from nova
725862a Breaking up utils.py
477d508 Disable index refresh during data re-sync
5db5612 Fix inconsistent mapping in image plugin
8eb705d Use delete_index instead of alias_listener_update
8bcca43 Updated from global requirements
52146ec Add region_name to service_credentials for devstack
7ad2648 Updated from global requirements
d9b1230 Listen to instance volume attach/detach events
acd51ea Add searchlight-ui to example local.conf
99b0e51 Add Link to client and ui
ca1aeed Zero Downtime Re-indexing Unit Tests.
800b46c Proposed clarification on zero-downtime reindexing doc
2a44b63 Backward compatibility with designate v1 api
4abf961 Disable oslo_config file discovery in tests
3b7dc51 Backward compatibility with designate v1 api
bcc8243 Increase heap for elasticsearch in func tests
1e80553 Fix _is_multiple_alias_exception signature
574a095 Add missing zero-downtime indexing documentation
8a287fc Increase heap for elasticsearch in func tests
3b132f8 Eliminate stray asterisk
600b691 Fix _is_multiple_alias_exception signature
a73cd1d Add missing zero-downtime indexing documentation
9fecc29 Disable oslo_config file discovery in tests
bb5ff07 Re-indexing optimization for doc_type
cb2137b Re-indexing optimization for doc_type
25ca131 Add missing mitaka release notes
531447d Zero Downtime Re-indexing Error Handling.
bc9eeaa Zero Downtime Re-indexing Error Handling.
b1d339b Don't index DHCP ports
ba67c2c Add missing mitaka release notes
3826498 Update reno for stable/mitaka
a5ccab8 Update .gitreview for stable/mitaka
22e05a8 Don't index DHCP ports
d5c3b46 Replace deprecated LOG.warn with LOG.warning
Diffstat (except docs and test files)
-------------------------------------
.gitreview | 1 +
README.rst | 2 +-
api-ref/source/conf.py | 240 ++++++
api-ref/source/http-codes.yaml | 55 ++
api-ref/source/index.rst | 25 +
.../samples/create-search-admin-request.json | 6 +
.../samples/create-search-aggregation-request.json | 11 +
.../create-search-aggregation-response.json | 64 ++
.../source/samples/create-search-all-request.json | 5 +
.../source/samples/create-search-all-response.json | 61 ++
.../create-search-complex-boolean-request.json | 25 +
.../create-search-complex-boolean-response.json | 93 +++
.../samples/create-search-free-text-request.json | 9 +
.../samples/create-search-free-text-response.json | 45 ++
.../create-search-multiple-type-request.json | 6 +
.../create-search-phrase-field-request.json | 8 +
.../create-search-phrase-field-response.json | 45 ++
.../samples/create-search-single-type-request.json | 6 +
.../create-search-single-type-response.json | 44 ++
.../list-facets-all-response-exclude-options.json | 78 ++
.../source/samples/list-facets-all-response.json | 90 +++
.../list-facets-include-fields-response.json | 11 +
.../source/samples/list-facets-type-response.json | 27 +
api-ref/source/samples/list-plugins-response.json | 14 +
api-ref/source/search-parameters.yaml | 170 ++++
api-ref/source/searchlight-v1.inc | 695 ++++++++++++++++
devstack/README.rst | 6 +-
devstack/local.conf | 65 +-
devstack/plugin.sh | 127 ++-
etc/policy.json | 36 +-
images/Figure1.png | Bin 0 -> 38618 bytes
images/Figure2.png | Bin 0 -> 183985 bytes
images/Figure3.png | Bin 0 -> 158874 bytes
images/Figure4.png | Bin 0 -> 239153 bytes
.../notes/add-0_2_1-notes-e1f8910ad27d88bb.yaml | 62 ++
...nova-microversion-support-8b40da4458678f20.yaml | 6 +
.../add-server-groups-plugin-3dff1b85a3f716f5.yaml | 10 +
.../notes/bp-add-region-name-69fbe5b84b2883c6.yaml | 8 +
...allow-aggregation-queries-fd7d39cb33068be1.yaml | 8 +
...indexing-into-new-mapping-bd78e94842fd2705.yaml | 11 +
...nfigurable-index-settings-59e00230f669c1a8.yaml | 11 +
...source-type-relationships-d697d0aa6648a560.yaml | 9 +
.../notes/bp-expose-version-be6403da43341c98.yaml | 8 +
.../notes/bp-facet-totals-369e7cd81b15f658.yaml | 16 +
releasenotes/notes/bp-flavor-plugin.yaml | 7 +
.../bp-hypervisor-plugin-5d17ed1005004b43.yaml | 7 +
...x-performance-enhancement-c8dc21da98f45628.yaml | 12 +
.../bp-index-role-separation-535e01ba3bb2252e.yaml | 11 +
...bp-indexing-type-wildcard-94c38a719a9210af.yaml | 7 +
.../bp-neutron-floatingip-1291f3a76f0e4aed.yaml | 8 +
.../bp-neutron-securitygroup-6c1531e9f80030bd.yaml | 7 +
.../bp-neutron-tenant-rbac-0922f23111965f7c.yaml | 9 +
.../bp-orphaned-es-aliases-24c5b5dcd2eb3a41.yaml | 10 +
.../bp-reduce-nova-api-calls-64eef45bd19fa89c.yaml | 8 +
...reenable-multiple-indices-3c05b9dde9d45d62.yaml | 13 +
...port-service-policy-files-c7e5eeb83c6f4126.yaml | 13 +
...-zero-downtime-reindexing-a607d52b8711d845.yaml | 24 +
.../bug-1545069-project-id-f8fe895ecce095ff.yaml | 10 +
.../notes/bug-1547118-3cfde976f32c3b5c.yaml | 9 +
...bug-1557778-facet-options-798bba4cb4e3067a.yaml | 13 +
.../notes/bug-1558630-fb26ae51f17fa9e2.yaml | 8 +
.../notes/bug-1558884-231498c4a6f56a31.yaml | 9 +
.../bug-1583681-plugin-info-d7307783f36cad25.yaml | 13 +
...bug-1584148-indexing-type-07ea84db2e6c1da4.yaml | 11 +
...g-1588540-neutron-routing-e7187a02535d8b01.yaml | 8 +
.../notes/bug-1592984-34d9417422c37e37.yaml | 9 +
.../bug-16175150-fix-mapping-a51e535a13deed96.yaml | 7 +
...bug-1621914-neutron-ports-58cf966264c09ce4.yaml | 8 +
...s-1619047-nested-mappings-4ddb2d0b067b3093.yaml | 10 +
...refresh-during-reindexing-e0152b6524385745.yaml | 7 +
...use-size-in-facets-search-438d4c4317879dfc.yaml | 9 +
releasenotes/source/index.rst | 3 +-
releasenotes/source/mitaka.rst | 6 +
requirements.txt | 34 +-
searchlight/api/__init__.py | 0
searchlight/api/common.py | 5 -
searchlight/api/middleware/context.py | 22 +-
searchlight/api/middleware/gzip.py | 3 +-
searchlight/api/middleware/version_negotiation.py | 6 +-
searchlight/api/policy.py | 80 +-
searchlight/api/v1/__init__.py | 0
searchlight/api/v1/router.py | 0
searchlight/api/v1/search.py | 115 ++-
searchlight/api/versions.py | 6 +-
searchlight/cmd/__init__.py | 13 +-
searchlight/cmd/api.py | 4 +
searchlight/cmd/control.py | 4 +-
searchlight/cmd/manage.py | 410 ++++++++--
searchlight/common/config.py | 4 +-
searchlight/common/exception.py | 10 +-
searchlight/common/property_utils.py | 12 +-
searchlight/common/resource_types.py | 45 ++
searchlight/common/utils.py | 46 +-
searchlight/common/wsgi.py | 12 +-
searchlight/context.py | 15 +
searchlight/elasticsearch/__init__.py | 36 +-
searchlight/elasticsearch/plugins/base.py | 425 ++++++----
.../elasticsearch/plugins/cinder/__init__.py | 7 +-
.../elasticsearch/plugins/cinder/snapshots.py | 27 +-
.../cinder/snapshots_notification_handler.py | 18 +-
.../elasticsearch/plugins/cinder/volumes.py | 36 +-
.../plugins/cinder/volumes_notification_handler.py | 12 +-
.../elasticsearch/plugins/designate/__init__.py | 29 +-
.../plugins/designate/notification_handlers.py | 54 +-
.../elasticsearch/plugins/designate/recordsets.py | 39 +-
.../elasticsearch/plugins/designate/zones.py | 34 +-
.../elasticsearch/plugins/glance/__init__.py | 21 +-
searchlight/elasticsearch/plugins/glance/images.py | 70 +-
.../plugins/glance/images_notification_handler.py | 13 +-
.../elasticsearch/plugins/glance/metadefs.py | 48 +-
.../glance/metadefs_notification_handler.py | 6 +-
searchlight/elasticsearch/plugins/helper.py | 619 +++++++++++++++
.../elasticsearch/plugins/neutron/__init__.py | 35 +
.../elasticsearch/plugins/neutron/floatingips.py | 108 +++
.../elasticsearch/plugins/neutron/networks.py | 65 +-
.../plugins/neutron/notification_handlers.py | 357 ++++++++-
searchlight/elasticsearch/plugins/neutron/ports.py | 41 +-
.../elasticsearch/plugins/neutron/routers.py | 29 +-
.../plugins/neutron/security_groups.py | 100 +++
.../elasticsearch/plugins/neutron/subnets.py | 25 +-
searchlight/elasticsearch/plugins/nova/__init__.py | 70 +-
searchlight/elasticsearch/plugins/nova/flavors.py | 110 +++
.../elasticsearch/plugins/nova/hypervisors.py | 106 +++
.../plugins/nova/notification_handler.py | 351 +++++++++
.../elasticsearch/plugins/nova/servergroups.py | 84 ++
searchlight/elasticsearch/plugins/nova/servers.py | 98 ++-
.../plugins/nova/servers_notification_handler.py | 101 ---
.../elasticsearch/plugins/openstack_clients.py | 72 +-
.../elasticsearch/plugins/swift/__init__.py | 4 +-
.../elasticsearch/plugins/swift/accounts.py | 11 +-
.../elasticsearch/plugins/swift/containers.py | 13 +-
searchlight/elasticsearch/plugins/swift/objects.py | 18 +-
.../plugins/swift/swift_notification_handler.py | 29 +-
searchlight/elasticsearch/plugins/utils.py | 755 ++++++------------
searchlight/listener.py | 46 +-
.../locale/de/LC_MESSAGES/searchlight-log-info.po | 47 ++
searchlight/locale/de/LC_MESSAGES/searchlight.po | 445 +++++++++++
.../locale/es/LC_MESSAGES/searchlight-log-info.po | 47 ++
searchlight/locale/es/LC_MESSAGES/searchlight.po | 413 ++++++++++
searchlight/locale/fr/LC_MESSAGES/searchlight.po | 424 ++++++++++
searchlight/locale/it/LC_MESSAGES/searchlight.po | 415 ++++++++++
searchlight/locale/ja/LC_MESSAGES/searchlight.po | 390 +++++++++
.../ko_KR/LC_MESSAGES/searchlight-log-info.po | 47 ++
.../locale/ko_KR/LC_MESSAGES/searchlight.po | 385 +++++++++
.../pt_BR/LC_MESSAGES/searchlight-log-info.po | 47 ++
.../locale/pt_BR/LC_MESSAGES/searchlight.po | 413 ++++++++++
searchlight/locale/ru/LC_MESSAGES/searchlight.po | 405 ++++++++++
.../locale/tr_TR/LC_MESSAGES/searchlight.po | 390 +++++++++
.../locale/zh_CN/LC_MESSAGES/searchlight.po | 375 +++++++++
.../locale/zh_TW/LC_MESSAGES/searchlight.po | 374 +++++++++
searchlight/notifier.py | 7 -
searchlight/plugin.py | 2 +-
searchlight/service_policies.py | 103 +++
.../unit/test_neutron_securitygroup_plugin.py | 213 +++++
setup.cfg | 6 +
test-requirements.txt | 14 +-
test-scripts/README.rst | 22 +
test-scripts/generate-swift-data.py | 9 +-
test-scripts/listener.py | 72 ++
test-scripts/resources.sh | 214 +++++
tools/colorizer.py | 3 +-
tox.ini | 19 +-
231 files changed, 20319 insertions(+), 2387 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index f8bcd25..b90f6f9 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -14,2 +14,2 @@ PasteDeploy>=1.5.0 # MIT
-Routes!=2.0,!=2.1,>=1.12.3;python_version=='2.7' # MIT
-Routes!=2.0,>=1.12.3;python_version!='2.7' # MIT
+Routes!=2.0,!=2.1,!=2.3.0,>=1.12.3;python_version=='2.7' # MIT
+Routes!=2.0,!=2.3.0,>=1.12.3;python_version!='2.7' # MIT
@@ -17,3 +17,3 @@ WebOb>=1.2.3 # MIT
-oslo.config>=3.7.0 # Apache-2.0
-oslo.concurrency>=3.5.0 # Apache-2.0
-oslo.context>=0.2.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.concurrency>=3.8.0 # Apache-2.0
+oslo.context>=2.9.0 # Apache-2.0
@@ -21,4 +21,4 @@ oslo.middleware>=3.0.0 # Apache-2.0
-oslo.service>=1.0.0 # Apache-2.0
-oslo.utils>=3.5.0 # Apache-2.0
-stevedore>=1.5.0 # Apache-2.0
-keystonemiddleware!=4.1.0,>=4.0.0 # Apache-2.0
+oslo.service>=1.10.0 # Apache-2.0
+oslo.utils>=3.16.0 # Apache-2.0
+stevedore>=1.16.0 # Apache-2.0
+keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0
@@ -30 +30 @@ Paste # MIT
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
+python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
@@ -37,2 +37,2 @@ oslo.log>=1.14.0 # Apache-2.0
-oslo.messaging>=4.0.0 # Apache-2.0
-oslo.policy>=0.5.0 # Apache-2.0
+oslo.messaging>=5.2.0 # Apache-2.0
+oslo.policy>=1.9.0 # Apache-2.0
@@ -41 +41 @@ oslo.serialization>=1.10.0 # Apache-2.0
-osprofiler>=1.1.0 # Apache-2.0
+osprofiler>=1.4.0 # Apache-2.0
@@ -43 +43 @@ osprofiler>=1.1.0 # Apache-2.0
-elasticsearch<2.0,>=1.3.0 # Apache-2.0
+elasticsearch<=3.0.0,>=2.0.0 # Apache-2.0
@@ -47 +47 @@ python-designateclient>=1.5.0 # Apache-2.0
-python-glanceclient>=2.0.0 # Apache-2.0
+python-glanceclient!=2.4.0,>=2.3.0 # Apache-2.0
@@ -49,2 +49,2 @@ python-novaclient!=2.33.0,>=2.29.0 # Apache-2.0
-python-neutronclient!=4.1.0,>=2.6.0 # Apache-2.0
-python-cinderclient>=1.3.1 # Apache-2.0
+python-neutronclient>=5.1.0 # Apache-2.0
+python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 3cc4eee..2459aff 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -9 +9 @@ hacking<0.11,>=0.10.0
-Babel>=1.3 # BSD
+Babel>=2.3.4 # BSD
@@ -13,2 +13 @@ coverage>=3.6 # Apache-2.0
-discover # BSD
-fixtures>=1.3.1 # Apache-2.0/BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
@@ -16,3 +15,3 @@ mox3>=0.7.0 # Apache-2.0
-mock>=1.2 # BSD
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
-requests!=2.9.0,>=2.8.1 # Apache-2.0
+mock>=2.0 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
+requests>=2.10.0 # Apache-2.0
@@ -23 +22 @@ oslotest>=1.10.0 # Apache-2.0
-reno>=0.1.1 # Apache2
+reno>=1.8.0 # Apache2
@@ -27,0 +27 @@ httplib2>=0.7.5 # MIT
+os-api-ref>=1.0.0 # Apache-2.0
1
0
We are excited to announce the release of:
sahara 5.0.0: Sahara project
This release is part of the newton release series.
With source available at:
http://git.openstack.org/cgit/openstack/sahara
For more details, please see below.
5.0.0
^^^^^
Documentation about distributed periodics are extended with steps
about installation additional libs required for correct work of
coordination backend. Please refer Advanced Configuration Guide for
details.
New Features
************
* Add impala 2.2 to MapR plugin
* MaR 5.2.0 is supported in MapR plugin.
* Add Sentry to MapR plugin
* Kafka was added in CDH 5.5 and CDH 5.7
* Add ability to automaticaly generate better configurations for
Ambari cluster by using 'ALWAYS_APPLY' strategy
* Versions 5.5.0 and 5.7.0 of Cloudera plugin are declared as
stable.
* CDH 5.7 is supported in CDH plugin.
* Added integration of Designate for hostname resolution through dns
servers
* Added event log for HDP plugin
* Kerberos support implemented for Cloudera and Ambari plugins. New
oozie client implemented to support authentication for oozie in
kerberized cluster.
* Custom health check is added to MapR plugin
* MapR 5.1.0.mrv2 is now Enabled.
* Remove support for Spark standalone in MapR plugin
* The following service versions were added to MapR 5.2.0 plugin -
HBase 1.1 - Drill 1.6 - Mahout 0.11 0.12 - Spark 1.6.1 - Impala 2.5
* Added rack awareness feature for CDH 5.5 and CDH 5.7
* Added rack awareness feature for HDP plugin
* Refactoring the logic on how floating ips are used by Sahara. This
change will allow the coexistence of cluster using floating ips with
cluster that do not.
* Separate transport url can be used for notifications purposes now,
to enable this feature 'transport_url' should be provided in
'oslo_messaging_notifications' section.
Upgrade Notes
*************
* migration from keystoneclient to keystoneauth is done for using
auth features of keystone.
* Migration from novaclient.v2.images to glanceclient
Deprecation Notes
*****************
* Versions 5, 5.3.0, 5.4.0 of Cloudera plugin are deprecated. It is
no longer maintainted and supposed to be removed in P release.
* Convert to cluster template feature is no longer supported by all
plugins.
* Option 'infrastructure engine' is removed from sahara configs.
* MapR 5.0.0.mrv2 is now Deprecated. It is not recommended for
usage. It is better to use MapR 5.1.0.mrv2 instead. This version of
plugin will be removed in Ocata release.
* Support of HDP 2.0.6 plugin was removed. Use Ambari plugin
instead.
Bug Fixes
*********
* Fixed incorrect down scaling of ambari cluster
* Fixed launching Hive jobs in Ambari plugin.
* Starting Ambari clusters on Centos 7 is fixed.
* CA certificate handling in keystone, nova, neutron and cinder
clients are fixed (#330635)
* After decommissioning hosts all assoicated configs groups will be
removed in ambari plugin.
* Fix uncompleted event logs for Oozie and Drill in MapR plugin.
* This patch adds ability to work with swift by using Keystone API
v3
* Fixed issue with handling endpoint_type during creation neutron
client, closed bug 1564805
* Fixed issues with hardcoded password during creation MySQL
database for Oozie, bug 1541122.
* Fixed issues with hardcoded password during starting hive process,
bug 1498035.
Changes in sahara 4.0.0.0rc1..5.0.0
-----------------------------------
bb96d84 Fix API compatibility issue
8622ca6 Fix incorrect event log for ambari
6e12a9f Documentation updates for stable/newton
5c9c023 Add workaround for Hue on CentOS 7
b178798 Fix ZooKeeper check for CentOS 7
5c775d2 Fill tempest.conf with Sahara-specific values
47579ee Update .gitreview for stable/newton
55f6eae Update link reference
c5baed4 [DOC] update doc about vanilla image builder
356417f [DOC] update doc about vanilla plugin
777b425 do not use artifacts at sahara files
602a771 fix docs env
c4d3474 [doc] change location of swiftfs jars
f18096a [DOC] update doc about cluster statuses
a01c3a1 [DOC] update doc about registering image
d693946 write docs about enabling kerberos
d0e47db [DOC] update doc about CDH image builder
1a738fd [DOC] update user doc about CDH plugin
d8e69c5 [Doc] Small fixes according to Spark on Vanilla supporting
fcc7bfe [Ambari] fix Ubuntu deploy
83a6a28 Remove entry point of sahara tempest plugin
a2a07ae Updated from global requirements
b3447fb Remove Tempest-like tests for clients (see sahara-tests)
f825ab3 standardize release note page ordering
5120dd7 reimplement oozie client as abstract
0d8af80 allow configuration of strategy for UI
f41bb7d [DOC] Add docs about pagination abilities
5b0960b Add MapR core 5.2
3a40553 [api-ref] Stop supporting os-api-ref 1.0.0
eb408c5 Add new version pack for services
2de626a Add event log for HDP plugin
e4207c8 Update api-ref docs for Designate feature
38784a4 Add Sentry service v1.6 to MapR plugin
cf7ed7f Add custom health check for MapR plugin
a2cf981 Rename all ClusterContext variables to 'cluster_context'
53f7854 Replace mfs.exchange with g.copy_file where it is possible
3b70e65 [DOC] Update user doc about Designate
89708bb [DOC] Fix misprints in api-ref
ff3e431 Spark on Vanilla Clusters
d44716f Added rack awareness in CDH plugin
3a4a3a0 [Doc] add description of "plugin update" to api ref
836e6fe Updated from global requirements
36640a8 Remove support for Spark standalone
af1a6bd Remove ssl config for Hue
94bb5a2 Refactor service home dir owner setting
8635ef6 [Ambari] More flexible auto configuration
1400bbe Fix invalid security repo
9f8c219 Added rack awareness in HDP plugin
5ccdc8e Updated from global requirements
9a322af use _LE() to wrap the error message
78beeb1 Added option to disable sahara db for storing job binaries.
fe46ce8 Config logABug feature for Sahara api-ref
53c5089 Remove unused config.CONF
4278b95 improve logging for job execution failure
abaf513 Updating DOC on floating IPs change
05b0dd0 Updated from global requirements
64cd756 Fix wait conditions with SSL deployments
2c7066a Enabling MapR on CentOS7
4eda721 Updated from global requirements
49d0e30 Fix wrong instance count in provision events
6427bc6 [doc] Fix some problems in docs
75053af delete unused LOG in some files
36c7b3d TrivialFix: Remove logging import usused
8efd656 Fix mapr cluster deployment
cdf8d23 Remove MAPR_USER variable
2cb0a6d Delete useless 'pass'
9694735 Updated from global requirements
2d54607 replace assertListEqual() to assertEqual()
7af18b9 Updated from global requirements
153ea9a Error handling during hosts file generation
047ab5a Replace 'lsb_release -is' with the method 'get_os_distrib'
2e78134 Add auto configs to HDP plugin
1a24d87 Correct reraising of exception
da70bca Fix wrong epel version for CentOS 7
c8dad2e Clean imports in code
3f6817a Adding release note to floating ips change
585c3af Updated from global requirements
d7f1793 Remove hardcoded password from db schema
2e74ca0 Get ready for os-api-ref sphinx theme change
e3c25ef Replace old CLI calls in grenade
4013eaa Updated from global requirements
7350d7b Add Kafka to CDH 5.5 and CDH 5.7
68e70f9 Updated from global requirements
ce5b034 plugins:patch is now admin only operation
91f99b7 Fix small bugs in pagination
deda629 Fix wrong hue-livy process name and move installation
0b89714 Fix wrong owner setting for config files
3340aff copying oozie.warden to prevent failure
ca5986d Updated from global requirements
13fcb05 Image argument validation and declaration
c28f98c [ambari] support kerberos deployment
1851067 [cdh] kerberos support implementation
758f38a kerberos infra deployment impl
dae16db Fixed the error with updating the job via command line
9cf7c30 Add sorting ability to Sahara-API
0383199 Health check for Designate
189e6a8 Fix configs for repos and swift urls in CDH 5.7
124abc1 Added documentation for Designate feature
8d9b2fd Documentation for image gen CLI and framework
6039058 Updated from global requirements
3441978 Updated from global requirements
d769212 Updated from global requirements
d9a2f00 Designate integration
de89c9d Updated from global requirements
e750d49 Correct reraising of exception
5c04f10 Updated from global requirements
60763fc Updated from global requirements
57f8cd1 Updated from global requirements
ffd9b8e labels for CDH plugin
4bb6437 Changing zookeeper path while updating conf
7b4091a labels for MapR plugin
d29b4cf Remove hardcoded password for Oozie service
49b63d5 Refactor the logic around use of floating ips
6eb1937 Adding argument-related validators for image configurability
cb9b45e Configuration engine for image generation CLI
4ba99fa Use assertEqual() instead of assertDictEqual()
3d5f745 improve error message for execution with retries
837a1dc remove infrastructure engine option
2d9c8ab Add pagination ability to Sahara-API
e552a61 [DOC] Added docs for sahara+ironic
e1b9ff8 [DOC] Inform operators about limited quotas
6265773 delete two unused LOG
ce7c3da Updated from global requirements
74991fe Remove unused LOG
f5fcc36 Updated from global requirements
e839480 Fixing unit tests for image create
ea44774 improved scaling for cdh plugin
0246945 Adding Pyleus configs to Storm plugin
d9bd498 Add Python 3.5 classifier and venv
61dd4c0 Docs should use "--plugin-version" instead of "--version"
4b70223 CLI for Plugin-Declared Image Declaration
5e91182 make ability to return real plugins in list ops
76f1a81 Failed to download ext-2.2.zip from dev.sencha.com
9789e6c Adding Python Jobs using Pyleus
6d492a2 Simplify tox hacking rule to match other projects
c90a1da [DOC] Cleanup time for incomplete clusters
4cebb9c improvements on api for plugins
d067680 Resolves issue where to allow custom repo URLS
e5d50d2 Updated from global requirements
feb46bd Updated from global requirements
c8869f7 don't serialize auto security group if not needed
28ca79d Fix typo in ambari_plugin.rst
f4cb71d replace import of future to the top
694549d fix building api ref docs
5dfc7a8 The addition of the parentheses for py3
30bba52 [DOC] Update installation guide
9b50406 use sessions for creating heatclient
fdb37d5 Fixed spelling error
1bb6c99 forbid cluster creation without secondarynamenode
6d0d359 Fix subdirectory typo in sahara db template Readme file
357ec36 Updated from global requirements
3f43a76 Upgrade Storm plugin to version 1.0.1
60dcbd5 Updated from global requirements
9d6b648 Add Impala 2.2 to MapR plugin
8db9a47 Support of CDH 5.7
27be399 fixing sahara-engine setup in devstack
745e384 Fix typo in configs_recommendations.rst
145a41a Remove outdated tools
feac29b [DOC] improve docs
fe6392f Fix typo in cdh_plugin.rst
3766836 Fix glanceclient.v2.images
f48eb7d Remove unecessary decorators from private interface
cf57c1f Ignore Nova config drive in devices list
879d2a6 plugins api impl
ef00d94 sleep before waiting requests
6bf92e7 allow to specify notifications transport url
890f341 ability to configure endpoint type for services
a36dfbc Updated from global requirements
fba2d67 novaclient.v2.images to glanceclient migration
f51e04e Updated from global requirements
3ad8bab Update documentation for hadoop-swift
535b8a2 Updated from global requirements
ddc2eeb Updated from global requirements
8f0eba2 [DOC] updated docs about keystone cli
b8d99ec Trivial: Fix wrong button name in dashboard user guide
b5a04b9 Updated from global requirements
cadfbdc implement db ops for plugin's api
7077cd2 replace seriailization of plugin to PluginManager
c27c5ca Moving WADL docs to Sahara repository
f41c51a Remove convert to cluster template feature
2758511 Trivial: Remove useless words in CDN image builder doc
38b629e Updated from global requirements
1a2f118 remove ability to create barbicanclient
9d42820 Fix the ca certificate handling in the client sessions
b122ffb fix grenade from mitaka upgrade
9158062 remove config groups associated with removed hosts
905e104 Updated from global requirements
6329a0a workaround to fix ambari start on centos7
165172f Updated from global requirements
3bdeeb1 Fix provision events for installing services
12281b4 New version of HDP plugin 2.4
8c7cfb7 Display credentials info in cluster general info
9a1300b Updated from global requirements
ef2103e Improve timeout message when cluster create fails
e72e9fb Updated from global requirements
7f32aed Modify HDP plugin doc for Ambari plugin
2460408 Fix retrieve auth_url and python 3 jobs
e3e3b7e Readable logging for Heat templates
e50c211 Use split_path from oslo.utils
0e499c7 Added "\" In quickstart guide
5396abe Corrects MapR distro selection for RHEL
b9b2dca Fix cluster creation with another tenant
f096c91 Updated from global requirements
947d20a Added unit tests for CDH 5.5.0 deploy file
9274c8d Updated from global requirements
252e6ef [Trivial] Remove unnecessary executable privilege
b9e3c17 Updated from global requirements
99127d5 Code refactoring of ambari deploy processes
e718b28 Fix down scaling of ambari cluster
8f365af HDP hive HDFS support
1aa4364 improve description of ambari plugin
037ba02 Remove hdp 2.0.6 plugin
78a981b Updated from global requirements
3bc9f05 Fix grenade
966bb6f Updated from global requirements
64355f5 Minimise number of auto security groups
54d4647 remove verbose option in devstack plugin
beba6e7 use the only method to initialize client
8ee0fae Updated from global requirements
ea68895 Resolve bug with long value to RANDSTR function.
2a338f5 Change 'Hbase' to 'HBase' string in spark service
1a7c0b0 Updated from global requirements
de5977d Remove openstack/common related stuff
14bf94b Added unit tests for ha_helper file
9ede053 Updated from global requirements
3de43e2 Updated from global requirements
7f3a594 Fix typo in Spark service
44c5332 Renamed job execution and templates endpoints
d9acf6e Fix doc about scenario and Tempest tests
f24cfa9 keystoneclient to keystoneauth migration
0bf0dda Helper method to use dnf instead of yum on fedora >=22
aa79252 PrettyTable and rfc3986 are no longer used in tests
2425aff Update the links to the RDO project
ef0bf04 Focus the documentation on distributed mode
75c5e8d Updated from global requirements
ac82ffc cdh plugin yum install option "-y" missing
954e227 update options mentioned in tempest readme
501f88c Update hadoop swift docs
b5827af Updated from global requirements
e36d2c6 Fix doc build if git is absent
b30b09a Added new unittest to oozie module
ebb044d Updated from global requirements
bde808f SPI Method to Validate Images
a48d29e Added tests for sahara cli
a16baae Fix unavailable MCS link
590ccac Define context.roles with base class
e417b17 Update the Administrator Guide link
b942a08 Updated from global requirements
45baf17 Updated from global requirements
818db68 Change property for auto creating schema
1a450ed Remove unsupported services from 5.1.0
df341a5 Updated from global requirements
dcdcf24 Updated from global requirements
021d441 Bandit password tests
b57c5f3 Workaround for temporary Oozie bug
72b5a58 Fixing the bandit config
fe7adb5 Pkg installation to ssh_remote
6764aef fix syntax error in ui dev docs
65330b8 Set libext path for Oozie 4.0.1, 4.1.0
ef38159 rename service api modules
ef05a56 Fixing grenade job
b5bb9f9 Add hadoop openstack swift jar to ambari cluster
09a7c3a Fix Hue integration with Spark and Hive
eba054c Move bandit to pep8
1e973cb Revert "Remove PyMySQL and psycopg2 from test-requirements.txt"
866514e Do not build config example for readthedocs.org
581ba2e Remove PyMySQL and psycopg2 from test-requirements.txt
fb238a0 Correctly configure Spark with Hive, HBase
0374f93 Set libext path for Oozie 4.0.1, 4.1.0
379ccf8 Add hive property for Hue < 0.9.0
38b793d Updated Sahara arch diagram
974db26 Fix incorrect visualization of MapR versions
1af4701 Updated volumes section in docs
eb1b308 Update reno for stable/mitaka
6daf4c0 Update .gitreview for stable/mitaka
Diffstat (except docs and test files)
-------------------------------------
.coveragerc | 3 +-
.gitignore | 1 +
.gitreview | 1 +
api-ref/source/cluster-templates.inc | 255 ++
api-ref/source/clusters.inc | 338 ++
api-ref/source/conf.py | 237 ++
api-ref/source/data-sources.inc | 214 ++
api-ref/source/event-log.inc | 42 +
api-ref/source/image-registry.inc | 251 ++
api-ref/source/index.rst | 20 +
api-ref/source/job-binaries.inc | 266 ++
api-ref/source/job-binary-internals.inc | 261 ++
api-ref/source/job-executions.inc | 329 ++
api-ref/source/job-types.inc | 61 +
api-ref/source/jobs.inc | 267 ++
api-ref/source/node-group-templates.inc | 271 ++
api-ref/source/parameters.yaml | 1124 +++++++
api-ref/source/plugins.inc | 187 ++
.../cluster-template-create-request.json | 17 +
.../cluster-template-create-response.json | 82 +
.../cluster-template-show-response.json | 82 +
.../cluster-template-update-request.json | 11 +
.../cluster-template-update-response.json | 67 +
.../cluster-templates-list-response.json | 140 +
.../samples/clusters/cluster-create-request.json | 9 +
.../samples/clusters/cluster-create-response.json | 128 +
.../samples/clusters/cluster-scale-request.json | 15 +
.../samples/clusters/cluster-scale-response.json | 370 ++
.../samples/clusters/cluster-show-response.json | 128 +
.../samples/clusters/cluster-update-request.json | 4 +
.../samples/clusters/cluster-update-response.json | 128 +
.../samples/clusters/clusters-list-response.json | 327 ++
.../clusters/multiple-clusters-create-request.json | 11 +
.../multiple-clusters-create-response.json | 6 +
.../data-source-register-hdfs-request.json | 6 +
.../data-source-register-hdfs-response.json | 14 +
.../data-source-register-swift-request.json | 10 +
.../data-source-register-swift-response.json | 14 +
.../data-sources/data-source-show-response.json | 14 +
.../data-sources/data-source-update-request.json | 4 +
.../data-sources/data-source-update-response.json | 14 +
.../data-sources/data-sources-list-response.json | 28 +
.../event-log/cluster-progress-response.json | 72 +
.../image-registry/image-register-request.json | 4 +
.../image-registry/image-register-response.json | 25 +
.../image-registry/image-show-response.json | 24 +
.../image-registry/image-tags-add-request.json | 7 +
.../image-registry/image-tags-add-response.json | 27 +
.../image-registry/image-tags-delete-request.json | 5 +
.../image-registry/image-tags-delete-response.json | 25 +
.../image-registry/images-list-response.json | 48 +
.../samples/job-binaries/create-request.json | 9 +
.../samples/job-binaries/create-response.json | 13 +
.../source/samples/job-binaries/list-response.json | 37 +
.../source/samples/job-binaries/show-data-response | 3 +
.../source/samples/job-binaries/show-response.json | 13 +
.../samples/job-binaries/update-request.json | 5 +
.../samples/job-binaries/update-response.json | 13 +
.../job-binary-internals/create-response.json | 12 +
.../job-binary-internals/list-response.json | 24 +
.../job-binary-internals/show-data-response | 3 +
.../job-binary-internals/show-response.json | 12 +
.../job-binary-internals/update-request.json | 4 +
.../job-binary-internals/update-response.json | 12 +
.../samples/job-executions/cancel-response.json | 120 +
.../samples/job-executions/job-ex-response.json | 120 +
.../job-executions/job-ex-update-request.json | 3 +
.../job-executions/job-ex-update-response.json | 120 +
.../samples/job-executions/list-response.json | 122 +
.../samples/job-types/job-types-list-response.json | 209 ++
.../source/samples/jobs/job-create-request.json | 11 +
.../source/samples/jobs/job-create-response.json | 35 +
.../source/samples/jobs/job-execute-request.json | 19 +
.../source/samples/jobs/job-execute-response.json | 30 +
api-ref/source/samples/jobs/job-show-response.json | 26 +
.../source/samples/jobs/job-update-request.json | 4 +
.../source/samples/jobs/job-update-response.json | 26 +
.../source/samples/jobs/jobs-list-response.json | 66 +
.../node-group-template-create-request.json | 13 +
.../node-group-template-create-response.json | 33 +
.../node-group-template-show-response.json | 37 +
.../node-group-template-update-request.json | 10 +
.../node-group-template-update-response.json | 30 +
.../node-group-templates-list-response.json | 76 +
.../samples/plugins/plugin-show-response.json | 12 +
.../samples/plugins/plugin-update-request.json | 7 +
.../samples/plugins/plugin-update-response.json | 32 +
.../plugins/plugin-version-show-response.json | 92 +
.../samples/plugins/plugins-list-response.json | 41 +
bandit.yaml | 4 +-
devstack/plugin.sh | 25 +-
devstack/settings | 3 +-
devstack/upgrade/cluster-create.json | 8 -
devstack/upgrade/cluster-template.json | 12 -
devstack/upgrade/from-mitaka/upgrade-sahara | 15 +
devstack/upgrade/ng-template.json | 10 -
devstack/upgrade/resources.sh | 256 +-
devstack/upgrade/settings | 2 +-
devstack/upgrade/shutdown.sh | 3 +-
.../userdoc/advanced.configuration.guide.rst | 131 +-
etc/edp-examples/README.rst | 2 +-
etc/sahara/policy.json | 1 +
etc/sahara/sahara.conf.sample-basic | 6 +
.../notes/add-impala-2.2-c1649599649aff5c.yaml | 3 +
.../notes/add-mapr-520-3ed6cd0ae9688e17.yaml | 3 +
.../notes/add-mapr-sentry-6012c08b55d679de.yaml | 3 +
.../notes/add_kafka_in_cdh-774c7c051480c892.yaml | 3 +
.../notes/ambari-downscaling-b9ba759ce9c7325e.yaml | 3 +
.../notes/ambari-hive-92b911e0a759ee88.yaml | 3 +
.../ambari-server-start-856403bc280dfba3.yaml | 3 +
.../auto_configs_for_hdp-011d460d37dcdf02.yaml | 4 +
.../notes/ca-cert-fix-5c434a82f9347039.yaml | 4 +
.../notes/cdh-labels-5695d95bce226051.yaml | 7 +
.../notes/cdh_5_7_support-9522cb9b4dce2378.yaml | 3 +
.../config-groups-ambari-837de6d33eb0fa87.yaml | 4 +
...nvert-to-cluster-template-43d502496d18625e.yaml | 4 +
.../designate-integration-784c5f7f29546015.yaml | 4 +
.../notes/engine-opt-258ff1ae9b04d628.yaml | 3 +
.../notes/event_log_for_hdp-a114511c477ef16d.yaml | 3 +
...-install-provision-events-c1bd2e05bf2be6bd.yaml | 3 +
...doop-swift-jar-for-ambari-4439913b01d42468.yaml | 4 +
...nor-endpoint-type-neutron-4583128c383d9745.yaml | 4 +
releasenotes/notes/kerberos-76dd297462b7337c.yaml | 5 +
...-to-keystonauth-migration-c75988975ad1a506.yaml | 4 +
.../notes/mapr-health-check-2eba3d742a2b853f.yaml | 3 +
.../notes/mapr-labels-5cc318616db59403.yaml | 7 +
...r-remove-spark-standalone-293ca864de9a7848.yaml | 3 +
...apr-services-new-versions-b32c2e8fe07d1600.yaml | 8 +
...nt_images_to_glanceclient-0266a2bd92b4be05.yaml | 3 +
.../rack_awareness_for_cdh-e0cd5d4ab46aa1b5.yaml | 3 +
.../rack_awareness_for_hdp-6e3d44468cc141a5.yaml | 3 +
...factor-floating-ips-logic-9d37d9297f3621b3.yaml | 5 +
...hard-coded-oozie-password-b97475c8772aa1bd.yaml | 4 +
...dcoded-password-from-hive-eb923b518974e853.yaml | 5 +
.../notes/remove-hdp-137d0ad3d2389b7a.yaml | 4 +
.../notes/transport_url-5bbbf0bb54d81727.yaml | 5 +
...eeper-configuration-steps-48c3d9706c86f227.yaml | 7 +
releasenotes/source/index.rst | 3 +-
releasenotes/source/mitaka.rst | 6 +
requirements.txt | 48 +-
sahara/api/middleware/auth_valid.py | 23 +-
sahara/api/v10.py | 49 +-
sahara/api/v11.py | 51 +-
sahara/api/v2/cluster_templates.py | 10 +-
sahara/api/v2/clusters.py | 8 +-
sahara/api/v2/data_sources.py | 10 +-
sahara/api/v2/images.py | 2 +-
sahara/api/v2/job_binaries.py | 16 +-
sahara/api/v2/job_executions.py | 22 +-
sahara/api/v2/job_types.py | 2 +-
sahara/api/v2/jobs.py | 26 +-
sahara/api/v2/node_group_templates.py | 11 +-
sahara/api/v2/plugins.py | 15 +-
sahara/cli/image_pack/__init__.py | 0
sahara/cli/image_pack/api.py | 125 +
sahara/cli/image_pack/cli.py | 125 +
sahara/cli/sahara_all.py | 4 -
sahara/cli/sahara_api.py | 4 -
sahara/conductor/api.py | 23 +-
sahara/conductor/manager.py | 21 +
sahara/conductor/objects.py | 30 +-
sahara/conductor/resource.py | 25 +-
sahara/config.py | 24 +-
sahara/context.py | 16 +-
sahara/db/api.py | 38 +-
.../versions/031_added_plugins_table.py | 46 +
.../versions/032_add_domain_name.py | 38 +
sahara/db/sqlalchemy/api.py | 254 +-
sahara/db/sqlalchemy/models.py | 19 +
sahara/db/templates/README.rst | 2 +-
sahara/exceptions.py | 18 +-
sahara/main.py | 21 +-
sahara/openstack/__init__.py | 0
sahara/openstack/commons.py | 67 -
sahara/plugins/ambari/client.py | 87 +-
sahara/plugins/ambari/common.py | 5 +
sahara/plugins/ambari/configs.py | 20 +-
sahara/plugins/ambari/decomission_helper.py | 118 -
sahara/plugins/ambari/deploy.py | 438 ++-
sahara/plugins/ambari/edp_engine.py | 6 +
sahara/plugins/ambari/health.py | 2 +-
sahara/plugins/ambari/plugin.py | 50 +-
sahara/plugins/ambari/requests_helper.py | 145 +
sahara/plugins/ambari/resources/configs-2.4.json | 1331 ++++++++
sahara/plugins/ambari/resources/images/image.yaml | 6 +
sahara/plugins/ambari/validation.py | 6 +
sahara/plugins/base.py | 46 +-
sahara/plugins/cdh/abstractversionhandler.py | 2 +-
sahara/plugins/cdh/client/clusters.py | 60 +
sahara/plugins/cdh/client/cms.py | 22 +
sahara/plugins/cdh/client/hosts.py | 5 +-
sahara/plugins/cdh/client/resource.py | 6 +-
sahara/plugins/cdh/client/types.py | 1 +
sahara/plugins/cdh/cloudera_utils.py | 103 +-
sahara/plugins/cdh/commands.py | 4 +-
sahara/plugins/cdh/config_helper.py | 5 +-
sahara/plugins/cdh/deploy.py | 42 +
sahara/plugins/cdh/edp_engine.py | 6 +
sahara/plugins/cdh/plugin.py | 16 +
sahara/plugins/cdh/plugin_utils.py | 6 +
sahara/plugins/cdh/v5_3_0/deploy.py | 18 +-
sahara/plugins/cdh/v5_4_0/deploy.py | 18 +-
sahara/plugins/cdh/v5_5_0/cloudera_utils.py | 25 +-
sahara/plugins/cdh/v5_5_0/config_helper.py | 7 +
sahara/plugins/cdh/v5_5_0/deploy.py | 29 +-
sahara/plugins/cdh/v5_5_0/plugin_utils.py | 11 +-
.../cdh/v5_5_0/resources/kafka-kafka_broker.json | 476 +++
.../v5_5_0/resources/kafka-kafka_mirror_maker.json | 482 +++
.../cdh/v5_5_0/resources/kafka-service.json | 374 +++
sahara/plugins/cdh/v5_5_0/versionhandler.py | 8 +
sahara/plugins/cdh/v5_7_0/__init__.py | 0
sahara/plugins/cdh/v5_7_0/cloudera_utils.py | 470 +++
sahara/plugins/cdh/v5_7_0/config_helper.py | 250 ++
sahara/plugins/cdh/v5_7_0/db_helper.py | 41 +
sahara/plugins/cdh/v5_7_0/deploy.py | 242 ++
sahara/plugins/cdh/v5_7_0/edp_engine.py | 54 +
sahara/plugins/cdh/v5_7_0/plugin_utils.py | 167 +
sahara/plugins/cdh/v5_7_0/resources/cdh_config.py | 139 +
sahara/plugins/cdh/v5_7_0/resources/cdh_config.sh | 3 +
.../cdh/v5_7_0/resources/create_hive_db.sql | 4 +
.../cdh/v5_7_0/resources/create_sentry_db.sql | 4 +
.../plugins/cdh/v5_7_0/resources/flume-agent.json | 428 +++
.../cdh/v5_7_0/resources/flume-service.json | 164 +
.../cdh/v5_7_0/resources/hbase-gateway.json | 110 +
.../v5_7_0/resources/hbase-hbaserestserver.json | 440 +++
.../v5_7_0/resources/hbase-hbasethriftserver.json | 446 +++
.../plugins/cdh/v5_7_0/resources/hbase-master.json | 506 +++
.../cdh/v5_7_0/resources/hbase-regionserver.json | 836 +++++
.../cdh/v5_7_0/resources/hbase-service.json | 740 ++++
.../cdh/v5_7_0/resources/hdfs-balancer.json | 68 +
.../cdh/v5_7_0/resources/hdfs-datanode.json | 644 ++++
.../v5_7_0/resources/hdfs-failovercontroller.json | 332 ++
.../plugins/cdh/v5_7_0/resources/hdfs-gateway.json | 116 +
.../plugins/cdh/v5_7_0/resources/hdfs-httpfs.json | 440 +++
.../cdh/v5_7_0/resources/hdfs-journalnode.json | 446 +++
.../cdh/v5_7_0/resources/hdfs-namenode.json | 812 +++++
.../cdh/v5_7_0/resources/hdfs-nfsgateway.json | 386 +++
.../v5_7_0/resources/hdfs-secondarynamenode.json | 446 +++
.../plugins/cdh/v5_7_0/resources/hdfs-service.json | 1400 ++++++++
.../plugins/cdh/v5_7_0/resources/hive-gateway.json | 98 +
.../cdh/v5_7_0/resources/hive-hivemetastore.json | 392 +++
.../cdh/v5_7_0/resources/hive-hiveserver2.json | 704 ++++
.../resources/hive-metastore-sentry-safety.xml | 15 +
.../resources/hive-server2-sentry-safety.xml | 12 +
.../plugins/cdh/v5_7_0/resources/hive-service.json | 722 ++++
sahara/plugins/cdh/v5_7_0/resources/hive-site.xml | 61 +
.../plugins/cdh/v5_7_0/resources/hive-webhcat.json | 332 ++
.../v5_7_0/resources/hue-hue_load_balancer.json | 236 ++
.../cdh/v5_7_0/resources/hue-hue_server.json | 356 ++
.../cdh/v5_7_0/resources/hue-kt_renewer.json | 200 ++
.../plugins/cdh/v5_7_0/resources/hue-service.json | 674 ++++
.../cdh/v5_7_0/resources/impala-catalogserver.json | 506 +++
.../cdh/v5_7_0/resources/impala-impalad.json | 710 ++++
.../plugins/cdh/v5_7_0/resources/impala-llama.json | 506 +++
.../cdh/v5_7_0/resources/impala-service.json | 632 ++++
.../cdh/v5_7_0/resources/impala-statestore.json | 374 +++
.../cdh/v5_7_0/resources/kafka-kafka_broker.json | 476 +++
.../v5_7_0/resources/kafka-kafka_mirror_maker.json | 482 +++
.../cdh/v5_7_0/resources/kafka-service.json | 374 +++
sahara/plugins/cdh/v5_7_0/resources/kms-kms.json | 530 +++
.../plugins/cdh/v5_7_0/resources/kms-service.json | 86 +
.../v5_7_0/resources/ks_indexer-hbase_indexer.json | 356 ++
.../cdh/v5_7_0/resources/ks_indexer-service.json | 188 ++
.../plugins/cdh/v5_7_0/resources/mapred-site.xml | 157 +
.../cdh/v5_7_0/resources/oozie-oozie_server.json | 662 ++++
.../cdh/v5_7_0/resources/oozie-service.json | 236 ++
.../cdh/v5_7_0/resources/priority-one-confs.json | 45 +
.../resources/sentry-impala-client-safety.xml | 16 +
.../cdh/v5_7_0/resources/sentry-sentry_server.json | 308 ++
.../cdh/v5_7_0/resources/sentry-service.json | 278 ++
.../plugins/cdh/v5_7_0/resources/solr-gateway.json | 32 +
.../plugins/cdh/v5_7_0/resources/solr-service.json | 404 +++
.../cdh/v5_7_0/resources/solr-solr_server.json | 440 +++
.../cdh/v5_7_0/resources/spark-gateway.json | 170 +
.../cdh/v5_7_0/resources/spark-service.json | 152 +
.../resources/spark-spark_yarn_history_server.json | 314 ++
.../cdh/v5_7_0/resources/sqoop-service.json | 110 +
.../cdh/v5_7_0/resources/sqoop-sqoop_server.json | 398 +++
.../plugins/cdh/v5_7_0/resources/yarn-gateway.json | 476 +++
.../cdh/v5_7_0/resources/yarn-jobhistory.json | 476 +++
.../cdh/v5_7_0/resources/yarn-nodemanager.json | 764 +++++
.../cdh/v5_7_0/resources/yarn-resourcemanager.json | 674 ++++
.../plugins/cdh/v5_7_0/resources/yarn-service.json | 542 +++
.../cdh/v5_7_0/resources/zookeeper-server.json | 596 ++++
.../cdh/v5_7_0/resources/zookeeper-service.json | 224 ++
sahara/plugins/cdh/v5_7_0/validation.py | 232 ++
sahara/plugins/cdh/v5_7_0/versionhandler.py | 100 +
sahara/plugins/exceptions.py | 36 +
sahara/plugins/fake/plugin.py | 29 +-
sahara/plugins/hdp/__init__.py | 0
sahara/plugins/hdp/ambariplugin.py | 450 ---
sahara/plugins/hdp/clusterspec.py | 385 ---
sahara/plugins/hdp/confighints_helper.py | 81 -
sahara/plugins/hdp/configprovider.py | 94 -
sahara/plugins/hdp/edp_engine.py | 42 -
sahara/plugins/hdp/hadoopserver.py | 279 --
sahara/plugins/hdp/saharautils.py | 33 -
sahara/plugins/hdp/versions/__init__.py | 0
.../plugins/hdp/versions/abstractversionhandler.py | 75 -
.../plugins/hdp/versions/version_2_0_6/__init__.py | 0
.../hdp/versions/version_2_0_6/edp_engine.py | 46 -
.../resources/ambari-config-resource.json | 3536 --------------------
.../resources/default-cluster.template | 1845 ----------
.../versions/version_2_0_6/resources/topology.sh | 21 -
.../plugins/hdp/versions/version_2_0_6/services.py | 1271 -------
.../hdp/versions/version_2_0_6/versionhandler.py | 1165 -------
.../plugins/hdp/versions/versionhandlerfactory.py | 59 -
sahara/plugins/images.py | 992 ++++++
sahara/plugins/kerberos.py | 400 +++
sahara/plugins/labels.py | 226 ++
sahara/plugins/mapr/abstract/health_checker.py | 24 +
.../plugins/mapr/base/base_cluster_configurer.py | 90 +-
sahara/plugins/mapr/base/base_cluster_context.py | 21 +
sahara/plugins/mapr/base/base_edp_engine.py | 10 +-
sahara/plugins/mapr/base/base_health_checker.py | 121 +
sahara/plugins/mapr/base/base_node_manager.py | 26 +-
sahara/plugins/mapr/base/base_version_handler.py | 6 +
sahara/plugins/mapr/domain/distro.py | 10 +-
sahara/plugins/mapr/domain/node_process.py | 2 +
sahara/plugins/mapr/domain/service.py | 40 +-
sahara/plugins/mapr/plugin.py | 15 +
sahara/plugins/mapr/resources/add_mapr_repo.sh | 4 +-
sahara/plugins/mapr/resources/add_mapr_user.sh | 2 -
.../plugins/mapr/resources/add_security_repos.sh | 1 -
sahara/plugins/mapr/services/drill/drill.py | 12 +-
sahara/plugins/mapr/services/flume/flume.py | 0
sahara/plugins/mapr/services/hbase/hbase.py | 17 +-
.../services/hbase/resources/hbase-default.json | 5 +
sahara/plugins/mapr/services/hive/hive.py | 62 +-
.../mapr/services/hive/resources/hive-default.xml | 4 +-
.../mapr/services/hive/resources/sentry-db.xml | 27 +
.../services/hive/resources/sentry-default.xml | 45 +
sahara/plugins/mapr/services/httpfs/httpfs.py | 4 +-
sahara/plugins/mapr/services/hue/hue.py | 157 +-
.../mapr/services/hue/resources/hue_3.9.0.template | 15 +-
sahara/plugins/mapr/services/impala/impala.py | 116 +-
.../services/impala/resources/impala-env.sh.j2 | 20 +-
sahara/plugins/mapr/services/mahout/mahout.py | 12 +
.../plugins/mapr/services/management/management.py | 14 +-
.../plugins/mapr/services/mapreduce/mapreduce.py | 14 +-
sahara/plugins/mapr/services/maprfs/maprfs.py | 7 +-
sahara/plugins/mapr/services/mysql/mysql.py | 32 +-
sahara/plugins/mapr/services/oozie/oozie.py | 75 +-
sahara/plugins/mapr/services/sentry/__init__.py | 0
.../services/sentry/resources/global-policy.ini | 6 +
.../services/sentry/resources/sentry-default.xml | 4 +
sahara/plugins/mapr/services/sentry/sentry.py | 165 +
.../services/spark/resources/spark-default.json | 16 -
.../services/spark/resources/spark-env.template | 2 +-
sahara/plugins/mapr/services/spark/spark.py | 370 +-
sahara/plugins/mapr/services/sqoop/sqoop2.py | 7 +-
sahara/plugins/mapr/services/swift/swift.py | 28 +-
sahara/plugins/mapr/services/yarn/yarn.py | 14 +-
sahara/plugins/mapr/util/general.py | 4 +-
sahara/plugins/mapr/util/maprfs_helper.py | 33 +-
sahara/plugins/mapr/util/password_utils.py | 70 +
sahara/plugins/mapr/util/validation_utils.py | 33 +
.../mapr/versions/v5_0_0_mrv2/version_handler.py | 1 +
.../mapr/versions/v5_1_0_mrv2/version_handler.py | 5 +-
.../plugins/mapr/versions/v5_2_0_mrv2/__init__.py | 0
.../plugins/mapr/versions/v5_2_0_mrv2/context.py | 60 +
.../mapr/versions/v5_2_0_mrv2/version_handler.py | 70 +
.../mapr/versions/version_handler_factory.py | 2 +-
sahara/plugins/opts.py | 26 +
sahara/plugins/provisioning.py | 83 +-
sahara/plugins/resources/create-principal-keytab | 12 +
sahara/plugins/resources/cron-file | 2 +
sahara/plugins/resources/cron-script | 3 +
sahara/plugins/resources/kdc_conf | 16 +
sahara/plugins/resources/kdc_conf_redhat | 13 +
.../plugins/resources/krb-client-init.sh.template | 14 +
sahara/plugins/resources/krb5_config | 11 +
.../resources/mit-kdc-server-init.sh.template | 34 +
sahara/plugins/spark/config_helper.py | 2 +-
sahara/plugins/spark/plugin.py | 14 +-
sahara/plugins/storm/config_helper.py | 29 +-
sahara/plugins/storm/edp_engine.py | 25 +-
sahara/plugins/storm/plugin.py | 39 +-
sahara/plugins/utils.py | 18 +-
sahara/plugins/vanilla/hadoop2/config.py | 74 +-
sahara/plugins/vanilla/hadoop2/config_helper.py | 103 +
sahara/plugins/vanilla/hadoop2/oozie_helper.py | 6 +-
.../vanilla/hadoop2/resources/create_hive_db.sql | 8 +-
.../vanilla/hadoop2/resources/create_oozie_db.sql | 4 +-
.../vanilla/hadoop2/resources/spark-cleanup.cron | 2 +
.../hadoop2/resources/tmp-cleanup.sh.template | 48 +
sahara/plugins/vanilla/hadoop2/run_scripts.py | 20 +
sahara/plugins/vanilla/hadoop2/starting_scripts.py | 6 +
sahara/plugins/vanilla/hadoop2/utils.py | 44 +
sahara/plugins/vanilla/hadoop2/validation.py | 6 +
sahara/plugins/vanilla/plugin.py | 10 +
sahara/plugins/vanilla/utils.py | 14 +
sahara/plugins/vanilla/v2_7_1/config_helper.py | 18 +-
sahara/plugins/vanilla/v2_7_1/edp_engine.py | 46 +
sahara/plugins/vanilla/v2_7_1/versionhandler.py | 32 +-
sahara/service/api.py | 310 --
sahara/service/api/__init__.py | 23 +
sahara/service/api/v10.py | 286 ++
sahara/service/api/v11.py | 269 ++
sahara/service/api/v2/__init__.py | 0
sahara/service/api/v2/cluster_templates.py | 43 +
sahara/service/api/v2/clusters.py | 175 +
sahara/service/api/v2/data_sources.py | 41 +
sahara/service/api/v2/images.py | 68 +
sahara/service/api/v2/job_binaries.py | 72 +
sahara/service/api/v2/job_executions.py | 109 +
sahara/service/api/v2/job_types.py | 78 +
sahara/service/api/v2/jobs.py | 45 +
sahara/service/api/v2/node_group_templates.py | 43 +
sahara/service/api/v2/plugins.py | 46 +
sahara/service/coordinator.py | 5 +-
sahara/service/edp/api.py | 277 --
sahara/service/edp/job_manager.py | 18 +-
sahara/service/edp/oozie/engine.py | 18 +-
sahara/service/edp/oozie/oozie.py | 117 +-
sahara/service/edp/storm/engine.py | 139 +-
sahara/service/engine.py | 39 +-
sahara/service/health/health_check_base.py | 125 +-
sahara/service/heat/commons.py | 2 +-
sahara/service/heat/heat_engine.py | 15 +-
sahara/service/heat/templates.py | 138 +-
sahara/service/networks.py | 43 +-
sahara/service/ntp_service.py | 8 +-
sahara/service/ops.py | 5 +-
sahara/service/periodic.py | 2 +-
sahara/service/quotas.py | 18 +-
sahara/service/sessions.py | 42 +-
sahara/service/shares.py | 3 +-
sahara/service/validation.py | 106 +-
sahara/service/validations/base.py | 28 +-
.../service/validations/cluster_template_schema.py | 3 +
sahara/service/validations/cluster_templates.py | 2 +-
sahara/service/validations/clusters.py | 8 +-
sahara/service/validations/clusters_scaling.py | 10 +-
sahara/service/validations/edp/data_source.py | 25 +
sahara/service/validations/edp/job.py | 7 +-
sahara/service/validations/edp/job_binary.py | 7 +
.../service/validations/edp/job_binary_internal.py | 13 +
sahara/service/validations/edp/job_execution.py | 37 +-
sahara/service/validations/node_group_templates.py | 2 +-
sahara/service/validations/plugins.py | 16 +-
sahara/service/volumes.py | 14 +-
sahara/swift/utils.py | 22 +-
.../tempest/scenario/data_processing/README.rst | 117 -
.../tempest/scenario/data_processing/__init__.py | 0
.../tempest/scenario/data_processing/config.py | 34 -
.../tempest/scenario/data_processing/plugin.py | 37 -
.../unit/cli/image_pack/test_image_pack_api.py | 72 +
.../plugins/ambari/test_decommission_helper.py | 68 -
.../unit/plugins/ambari/test_requests_helper.py | 96 +
.../unit/plugins/cdh/base_plugin_utils_test.py | 3 +-
.../unit/plugins/cdh/v5/test_versionhandler.py | 11 +-
.../unit/plugins/cdh/v5_3_0/test_versionhandler.py | 15 +-
.../unit/plugins/cdh/v5_4_0/test_versionhandler.py | 15 +-
.../plugins/cdh/v5_5_0/test_config_helper_550.py | 5 +-
.../unit/plugins/cdh/v5_5_0/test_versionhandler.py | 15 +-
.../plugins/cdh/v5_7_0/test_config_helper_570.py | 168 +
.../unit/plugins/cdh/v5_7_0/test_edp_engine.py | 200 ++
.../plugins/cdh/v5_7_0/test_plugin_utils_570.py | 25 +
.../unit/plugins/cdh/v5_7_0/test_spark_engine.py | 69 +
.../unit/plugins/cdh/v5_7_0/test_validation.py | 123 +
.../unit/plugins/cdh/v5_7_0/test_versionhandler.py | 140 +
.../plugins/hdp/resources/config-resource.json | 42 -
.../hdp/resources/sample-ambari-blueprint.json | 195 --
.../unit/plugins/hdp/test_clusterspec_hdp2.py | 2035 -----------
.../unit/plugins/hdp/test_confighints_helper.py | 147 -
.../unit/plugins/hdp/test_versionmanagerfactory.py | 35 -
.../plugins/hdp/versions/version_2_0_6/__init__.py | 0
.../hdp/versions/version_2_0_6/test_edp_engine.py | 98 -
.../unit/plugins/mapr/test_cluster_context.py | 5 +-
.../unit/plugins/vanilla/hadoop2/test_utils.py | 75 +
.../service/validation/edp/test_data_source.py | 21 +-
.../unit/service/validation/edp/test_job_binary.py | 2 +-
.../validation/edp/test_job_binary_internal.py | 2 +-
.../service/validation/edp/test_job_executor.py | 2 +-
.../service/validation/edp/test_job_interface.py | 3 +-
.../validation/test_cluster_create_validation.py | 8 +-
.../validation/test_cluster_delete_validation.py | 4 +-
.../validation/test_cluster_scaling_validation.py | 4 +-
.../test_cluster_template_create_validation.py | 11 +-
.../test_cluster_template_update_validation.py | 2 +-
.../validation/test_cluster_update_validation.py | 6 +-
.../test_ng_template_validation_create.py | 2 +-
.../test_ng_template_validation_update.py | 2 +-
sahara/topology/topology_helper.py | 4 +
sahara/utils/api.py | 13 +-
sahara/utils/cluster.py | 62 +-
sahara/utils/edp.py | 7 +-
sahara/utils/files.py | 10 +
sahara/utils/hacking/checks.py | 4 -
sahara/utils/hacking/logging_checks.py | 16 +-
sahara/utils/network.py | 43 +
sahara/utils/openstack/barbican.py | 29 -
sahara/utils/openstack/base.py | 37 +-
sahara/utils/openstack/glance.py | 47 +
sahara/utils/openstack/heat.py | 14 +-
sahara/utils/openstack/images.py | 178 +-
sahara/utils/openstack/keystone.py | 60 +-
sahara/utils/openstack/neutron.py | 12 +-
sahara/utils/openstack/nova.py | 2 -
sahara/utils/procutils.py | 5 +-
sahara/utils/remote.py | 27 +-
sahara/utils/rpc.py | 57 +-
sahara/utils/ssh_remote.py | 189 +-
sahara/utils/types.py | 7 +
sahara/utils/wsgi.py | 3 -
setup.cfg | 8 +-
test-requirements.txt | 23 +-
tools/build_docs | 3 -
tools/install_venv | 3 -
tools/install_venv.py | 76 -
tools/install_venv_common.py | 212 --
tools/lintstack.py | 39 +-
tools/run_fast_checks | 3 -
tools/run_pep8 | 3 -
tools/run_pylint | 3 -
tools/with_venv.sh | 4 -
tox.ini | 29 +-
634 files changed, 51648 insertions(+), 18978 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 506f672..488089d 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -7,3 +7,3 @@ pbr>=1.6 # Apache-2.0
-alembic>=0.8.0 # MIT
-Babel>=1.3 # BSD
-castellan>=0.3.1 # Apache-2.0
+alembic>=0.8.4 # MIT
+Babel>=2.3.4 # BSD
+castellan>=0.4.0 # Apache-2.0
@@ -11,2 +11,2 @@ eventlet!=0.18.3,>=0.18.2 # MIT
-Flask<1.0,>=0.10 # BSD
-iso8601>=0.1.9 # MIT
+Flask!=0.11,<1.0,>=0.10 # BSD
+iso8601>=0.1.11 # MIT
@@ -15,5 +15,6 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT
-keystonemiddleware!=4.1.0,>=4.0.0 # Apache-2.0
-oslo.config>=3.7.0 # Apache-2.0
-oslo.concurrency>=3.5.0 # Apache-2.0
-oslo.context>=0.2.0 # Apache-2.0
-oslo.db>=4.1.0 # Apache-2.0
+keystoneauth1>=2.10.0 # Apache-2.0
+keystonemiddleware!=4.1.0,!=4.5.0,>=4.0.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.concurrency>=3.8.0 # Apache-2.0
+oslo.context>=2.9.0 # Apache-2.0
+oslo.db!=4.13.1,!=4.13.2,>=4.10.0 # Apache-2.0
@@ -22 +23 @@ oslo.log>=1.14.0 # Apache-2.0
-oslo.messaging>=4.0.0 # Apache-2.0
+oslo.messaging>=5.2.0 # Apache-2.0
@@ -24,2 +25,2 @@ oslo.middleware>=3.0.0 # Apache-2.0
-oslo.policy>=0.5.0 # Apache-2.0
-oslo.rootwrap>=2.0.0 # Apache-2.0
+oslo.policy>=1.9.0 # Apache-2.0
+oslo.rootwrap>=5.0.0 # Apache-2.0
@@ -27,7 +28,7 @@ oslo.serialization>=1.10.0 # Apache-2.0
-oslo.service>=1.0.0 # Apache-2.0
-oslo.utils>=3.5.0 # Apache-2.0
-paramiko>=1.16.0 # LGPL
-requests!=2.9.0,>=2.8.1 # Apache-2.0
-python-cinderclient>=1.3.1 # Apache-2.0
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
-python-manilaclient>=1.3.0 # Apache-2.0
+oslo.service>=1.10.0 # Apache-2.0
+oslo.utils>=3.16.0 # Apache-2.0
+paramiko>=2.0 # LGPLv2.1+
+requests>=2.10.0 # Apache-2.0
+python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
+python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
+python-manilaclient>=1.10.0 # Apache-2.0
@@ -36,2 +37,3 @@ python-swiftclient>=2.2.0 # Apache-2.0
-python-neutronclient!=4.1.0,>=2.6.0 # Apache-2.0
-python-heatclient>=0.6.0 # Apache-2.0
+python-neutronclient>=5.1.0 # Apache-2.0
+python-heatclient>=1.4.0 # Apache-2.0
+python-glanceclient!=2.4.0,>=2.3.0 # Apache-2.0
@@ -39 +41 @@ six>=1.9.0 # MIT
-stevedore>=1.5.0 # Apache-2.0
+stevedore>=1.16.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index b7d345e..a456df3 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -7,3 +7,2 @@ hacking<0.11,>=0.10.0
-Mako>=0.4.0 # MIT
-PrettyTable<0.8,>=0.7 # BSD
-PyMySQL>=0.6.2 # MIT License
+PyMySQL!=0.7.7,>=0.6.2 # MIT License
+bandit>=1.1.0 # Apache-2.0
@@ -13,2 +12,2 @@ doc8 # Apache-2.0
-fixtures>=1.3.1 # Apache-2.0/BSD
-mock>=1.2 # BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
+mock>=2.0 # BSD
@@ -17 +16,2 @@ oslotest>=1.10.0 # Apache-2.0
-os-testr>=0.4.1 # Apache-2.0
+os-api-ref>=1.0.0 # Apache-2.0
+os-testr>=0.7.0 # Apache-2.0
@@ -19,4 +19,3 @@ psycopg2>=2.5 # LGPL/ZPL
-pylint==1.4.5 # GNU GPL v2
-reno>=0.1.1 # Apache2
-rfc3986>=0.2.0 # Apache-2.0
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+pylint==1.4.5 # GPLv2
+reno>=1.8.0 # Apache2
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
@@ -24,2 +22,0 @@ sphinxcontrib-httpdomain # BSD
-tempest-lib>=0.14.0 # Apache-2.0
-testrepository>=0.0.18 # Apache-2.0/BSD
@@ -29,2 +25,0 @@ testtools>=1.4.0 # MIT
-
-python-saharaclient>=0.13.0 # Apache-2.0
1
0
06 Oct '16
We are grateful to announce the release of:
os-cloud-config 5.0.0: Configuration for OpenStack clouds.
This release is part of the newton release series.
For more details, please see below.
Changes in os-cloud-config 0.4.1..5.0.0
---------------------------------------
7e616a3 Updated from global requirements
b8603c8 Fix wrong usage of novaclient
b8a9dcb Adding the congress service
7bf38a3 Updated from global requirements
becaec5 Updated from global requirements
4ec35b7 Updated from global requirements
8af499e Add Heat CloudFormation service for endpoint creation
b7c1c82 Updated from global requirements
da41a10 Set compute API version to 2.1, don't use v3
9445cb1 Allow customizing keystone endpoint ports
4860bbd Formally deprecate os_cloud_config.nodes in favor of tripleo-common
e6a7cd7 Remove support for py33/py26
f49d3e3 Replace deprecated LOG.warn with LOG.warning
fd26d9d add aodh and gnocchi to keystone service list
10bf35c Put py34 first in the env order of tox
68d6796 Fix a typo in usage.rst
Diffstat (except docs and test files)
-------------------------------------
os_cloud_config/keystone.py | 85 ++++++++++++++++++-------
os_cloud_config/neutron.py | 10 +--
os_cloud_config/nodes.py | 16 +++--
os_cloud_config/utils/clients.py | 10 +--
requirements.txt | 10 +--
setup.cfg | 2 +-
test-requirements.txt | 6 +-
tox.ini | 2 +-
12 files changed, 106 insertions(+), 61 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index e6fea7e..8be32cf 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,2 +6,2 @@ pbr>=1.6 # Apache-2.0
-Babel>=1.3 # BSD
-python-glanceclient>=1.2.0 # Apache-2.0
+Babel>=2.3.4 # BSD
+python-glanceclient>=2.0.0 # Apache-2.0
@@ -9,2 +9,2 @@ python-ironicclient>=1.1.0 # Apache-2.0
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.6.0 # Apache-2.0
-python-neutronclient>=2.6.0 # Apache-2.0
+python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # Apache-2.0
+python-neutronclient>=4.2.0 # Apache-2.0
@@ -12 +12 @@ python-novaclient!=2.33.0,>=2.29.0 # Apache-2.0
-oslo.config>=3.4.0 # Apache-2.0
+oslo.config>=3.12.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 23f6eea..1529b5f 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8,2 +8,2 @@ discover # BSD
-fixtures>=1.3.1 # Apache-2.0/BSD
-mock>=1.2 # BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
+mock>=2.0 # BSD
@@ -11 +11 @@ python-subunit>=0.0.18 # Apache-2.0/BSD
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
1
0
06 Oct '16
We are excited to announce the release of:
os-apply-config 5.0.0: Config files from cloud metadata
This release is part of the newton release series.
For more details, please see below.
Changes in os-apply-config 0.1.32..5.0.0
----------------------------------------
69879da Updated from global requirements
3af31b4 Updated from global requirements
36f24df Updated from global requirements
a23d5c7 Updated from global requirements
85be7e2 Updated from global requirements
a023519 Updated from global requirements
6171221 Remove argparse from requirements
45ca3ce Deprecated tox -downloadcache option removed
Diffstat (except docs and test files)
-------------------------------------
requirements.txt | 11 +++++------
test-requirements.txt | 18 +++++++++---------
tox.ini | 1 -
3 files changed, 14 insertions(+), 16 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 3211172..d2ee6fd 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -4 +4 @@
-pbr>=1.6
+pbr>=1.6 # Apache-2.0
@@ -6,5 +6,4 @@ pbr>=1.6
-anyjson>=0.3.3
-argparse
-pystache
-PyYAML>=3.1.0
-six>=1.9.0
+anyjson>=0.3.3 # BSD
+pystache # MIT
+PyYAML>=3.1.0 # MIT
+six>=1.9.0 # MIT
diff --git a/test-requirements.txt b/test-requirements.txt
index 95d5cf5..5913a12 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -6,9 +6,9 @@ hacking<0.10,>=0.9.2
-coverage>=3.6
-discover
-fixtures>=1.3.1
-mock>=1.2
-python-subunit>=0.0.18
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
-testrepository>=0.0.18
-testscenarios>=0.4
-testtools>=1.4.0
+coverage>=3.6 # Apache-2.0
+discover # BSD
+fixtures>=3.0.0 # Apache-2.0/BSD
+mock>=2.0 # BSD
+python-subunit>=0.0.18 # Apache-2.0/BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD
+testrepository>=0.0.18 # Apache-2.0/BSD
+testscenarios>=0.4 # Apache-2.0/BSD
+testtools>=1.4.0 # MIT
1
0
We are happy to announce the release of:
zaqar-ui 1.0.0: Zaqar User Interface
This release is part of the newton release series.
With source available at:
http://git.openstack.org/cgit/openstack/zaqar-ui
Please report issues through launchpad:
http://bugs.launchpad.net/zaqar-ui
For more details, please see below.
Changes in zaqar-ui 1.0.0.0b3..1.0.0
------------------------------------
6ed5bfc Imported Translations from Zanata
Diffstat (except docs and test files)
-------------------------------------
zaqar_ui/locale/ko_KR/LC_MESSAGES/djangojs.po | 169 ++++++++++++++++++++++++++
1 file changed, 169 insertions(+)
1
0