We are amped to announce the release of: sahara 3.0.1: Sahara project This release is part of the liberty stable release series. With source available at: http://git.openstack.org/cgit/openstack/sahara With package available at: https://pypi.python.org/pypi/sahara For more details, please see below. 3.0.1 ^^^^^ Bug Fixes ********* * Fixed issue with launching clusters without auto-security groups. * Cluster with 'is_protected' field can be created now, closed bug 1510929 * Wait condition feature can be disabled by the option. * Event log hanling fixed in case of problems with volumes formatting, closed bug 1506839 * Fixed search of devices that need to be mounted, closed bug 1508872 * Fixed issues with using Swift as an output datasource. * Fixed arguments list in latest cdh plugin * Fixed problems with validation of job binaries update, closed bug 1508507 * Job execution can be created with 'is_protected' field, closed bug 1510994 * Issues with job types call are fixed. Jobs can be launched from UI if ambari plugin enabled. * Open ports for Kafka in Ambari plugin, closed bug 1512661 * Heat engine shows reasons in case of failure * Escape unhandled unicode characters in environment configuration files * Spark workers are found by fqdn * Timeouts for ssh operations are configurable now * Removed unneeded volume serialization * In case of XFS volume will be formatted with '-f' option. * Added additional filter to volume_type check, closed bug 1508017 Other Notes *********** * Fixed client tempest tests * Fixed tempest config group * Start using reno to manage release notes. Changes in sahara 3.0.0..3.0.1 ------------------------------ 1f3a1a1 Update image address CDH plugin document c716f5b Don't configure hadoop.tmp.dir in Spark plugin abdb0ea Updated from global requirements cc218dd Add release notes for missed bugs 4cd7844 Force releasenotes warnings to be treated as errors d717068 Option for disabling wait condition feature d438011 Remove sharps from release notes b419f5e Updated from global requirements d2a792d Fix Spark installation fails when parsing spark-env.sh ea18624 Remove unneeded 'self' in plugins.cdh.v5_4_0.plugin_utils 2c8b1eb Report stack_status_reason on heat failure a6b3a88 Remove unneeded volume serialization 0920c6d Add "unreleased" release notes page ddd875a Support reno for release notes management 9cb41e4 Fix spark workers do not start in MapR plugin f04da88 Make ssh timeout configurable a944047 Fix of client tests in tempest 5d4871c Fix tempest tests 0e5198b Fixing search of devices that need to be mount ece97e9 Fixing problem with validation of job binaries update 5c7b09d Add additional filter to volume_type check 653d76c Fixing event log handling during volumes mount 0777402 Fixing cluster creation with is_protected field ab75f2b Missing stuff for Kafka in Ambari plugin 95096e6 Fixing job execution creation with is_protected field 68301c4 Add first reno-based release note d736017 Changing fedora image url for grenade job 570a0ba Fixing grenade_job faaa266 Updated from global requirements cd9df52 Add -f option to formatting volumes 2fb37cd Updated from global requirements ea70bb8 Fix issue with job types in Ambari plugin 8abaa88 Updated from global requirements bd944eb Fix .gitreview for stable/liberty branch 7ab41f1 Switch to post-versioning 307c115 Fixing cluster creation without auto_security_group Diffstat (except docs and test files) ------------------------------------- .gitignore | 3 + .gitreview | 1 + devstack/upgrade/resources.sh | 2 +- devstack/upgrade/settings | 4 +- devstack/upgrade/shutdown.sh | 3 +- releasenotes/notes/.placeholder | 0 releasenotes/notes/asg-fix-6243442b85931c40.yaml | 3 + .../cluster-is-protected-eee9e2bd2595fdc8.yaml | 4 + .../notes/disable-wc-option-ddf011af6ebb32b7.yaml | 3 + .../notes/event-log-handle-97dd75f428aa28f7.yaml | 4 + .../notes/fix-search-devices-72026264ce828187.yaml | 4 + .../fix-spark-swift-outputs-b4cd588b5f4e8571.yaml | 3 + .../notes/fix-tempest-d68a1c712b219a64.yaml | 3 + .../notes/fix_cdh_args-ee888dfeaf8ef0de.yaml | 3 + .../job-binary-validation-3b97219ae8c8e7bb.yaml | 4 + .../job-exec-is-protected-31831f9f3fb51aeb.yaml | 4 + .../notes/job-types-ambari-0873ad7deeecf8ba.yaml | 4 + .../notes/ports-ambari-43c5be3f3e7b3107.yaml | 4 + ...how_stack_failure_reasons-02ca814308c61718.yaml | 3 + .../notes/spark-installation-5ed2c7ca75f79de1.yaml | 3 + .../notes/spark-workers-67ae5e0da9dec137.yaml | 3 + .../notes/ssh-timeout-310a05c55cb0a086.yaml | 3 + .../notes/start-using-reno-689372a0a8dd3093.yaml | 3 + .../notes/unneeded_volume-37032bc35630cb46.yaml | 3 + .../volume-formatting-with-f-1e4459ac32a8957c.yaml | 3 + .../notes/volume-type-check-58203c2df50999d2.yaml | 4 + releasenotes/source/_static/.placeholder | 0 releasenotes/source/_templates/.placeholder | 0 releasenotes/source/conf.py | 216 +++++++++++++++++++++ releasenotes/source/index.rst | 9 + releasenotes/source/liberty.rst | 6 + releasenotes/source/unreleased.rst | 5 + requirements.txt | 14 +- sahara/api/v10.py | 2 +- sahara/api/v11.py | 3 +- sahara/config.py | 6 +- sahara/db/sqlalchemy/api.py | 9 - sahara/plugins/ambari/plugin.py | 13 +- sahara/plugins/cdh/v5_4_0/plugin_utils.py | 2 +- sahara/plugins/mapr/domain/configuration_file.py | 12 +- sahara/plugins/mapr/services/spark/spark.py | 2 +- sahara/plugins/spark/config_helper.py | 2 - sahara/service/heat/templates.py | 62 +++--- sahara/service/validations/base.py | 4 +- sahara/service/validations/clusters.py | 7 + sahara/service/validations/edp/job_binary.py | 22 ++- sahara/service/validations/edp/job_execution.py | 8 + sahara/service/volumes.py | 69 ++++--- .../tempest/scenario/data_processing/README.rst | 22 ++- .../tempest/scenario/data_processing/plugin.py | 4 +- .../service/validation/edp/test_job_executor.py | 33 ++++ .../validation/test_cluster_update_validation.py | 36 ++++ sahara/utils/openstack/heat.py | 2 +- sahara/utils/ssh_remote.py | 46 ++++- setup.cfg | 1 - test-requirements.txt | 2 + tools/gate/scenario/pre_test_hook.sh | 5 + tox.ini | 3 + 67 files changed, 651 insertions(+), 198 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 9746470..d9a2c36 100644 --- a/requirements.txt +++ b/requirements.txt @@ -14 +14 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 -keystonemiddleware>=2.0.0 +keystonemiddleware!=2.4.0,>=2.0.0 @@ -21,2 +21,2 @@ oslo.log>=1.8.0 # Apache-2.0 -oslo.messaging!=1.17.0,!=1.17.1,>=1.16.0 # Apache-2.0 -oslo.middleware>=2.8.0 # Apache-2.0 +oslo.messaging!=1.17.0,!=1.17.1,!=2.6.0,!=2.6.1,!=2.7.0,!=2.8.0,!=2.8.1,!=2.9.0,!=3.1.0,>=1.16.0 # Apache-2.0 +oslo.middleware!=3.0.0,!=3.1.0,!=3.2.0,>=2.8.0 # Apache-2.0 @@ -27 +27 @@ oslo.service>=0.7.0 # Apache-2.0 -oslo.utils>=2.0.0 # Apache-2.0 +oslo.utils!=2.6.0,>=2.0.0 # Apache-2.0 @@ -29 +29 @@ paramiko>=1.13.0 -requests>=2.5.2 +requests!=2.8.0,!=2.9.0,>=2.5.2 @@ -32 +32 @@ python-cinderclient>=1.3.1 -python-keystoneclient>=1.6.0 +python-keystoneclient!=1.8.0,>=1.6.0 @@ -34 +34 @@ python-manilaclient>=1.3.0 -python-novaclient>=2.28.1 +python-novaclient!=2.33.0,>=2.28.1 diff --git a/test-requirements.txt b/test-requirements.txt index d55c732..09dba17 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -17,0 +18 @@ pylint==1.4.4 # GNU GPL v2 +reno>=0.1.1 # Apache2 @@ -22,0 +24 @@ testrepository>=0.0.18 +testresources>=0.2.4