We are mirthful to announce the release of: swift 2.11.0: OpenStack Object Storage This release is part of the ocata release series. Download the package from: https://tarballs.openstack.org/swift/ For more details, please see below. 2.11.0 ^^^^^^ New Features ************ * The improvements to EC reads made in Swift 2.10.0 have also been applied to the reconstructor. This allows fragments to be rebuilt in more circumstances, resulting in faster recovery from failures. * Instead of using a separate .durable file to indicate the durable status of an EC fragment archive, we rename the .data to include a durable marker in the filename. This saves one inode for every EC .data file. Existing .durable files will not be removed, and they will continue to work just fine. * Closed a bug where ssync may have written bad fragment data in some circumstances. A check was added to ensure the correct number of bytes is written for a fragment before finalizing the write. Also, erasure coded fragment metadata will now be validated on read requests and, if bad data is found, the fragment will be quarantined. * Added a configurable URL base to staticweb. * Support multi-range GETs for static large objects. * TempURLs using the "inline" parameter can now also set the "filename" parameter. Both are used in the Content-Disposition response header. * Mirror X-Trans-Id to X-Openstack-Request-Id. * SLO will now concurrently HEAD segments, resulting in much faster manifest validation and object creation. By default, two HEAD requests will be done at a time, but this can be changed by the operator via the new *concurrency* setting in the "[filter:slo]" section of the proxy server config. * Suppressed the KeyError message when auditor finds an expired object. * Daemons using InternalClient can now be properly killed with SIGTERM. * Added a "user" option to the drive-audit config file. Its value is used to set the owner of the drive-audit recon cache. * Throttle update_auditor_status calls so it updates no more than once per minute. * Suppress unexpected-file warnings for rsync temp files. Upgrade Notes ************* * Updated the PyECLib dependency to 1.3.1. * Note that after writing EC data with Swift 2.11.0 or later, that data will not be accessible to earlier versions of Swift. Critical Issues *************** * WARNING: If you are using the ISA-L library for erasure codes, please upgrade to liberasurecode 1.3.1 (or later) as soon as possible. If you are using isa_l_rs_vand with more than 4 parity, please read https://bugs.launchpad.net/swift/+bug/1639691 and take necessary action. Other Notes *********** * Various other minor bug fixes and improvements. Changes in swift 2.10.0..2.11.0 ------------------------------- bf473dd 2.11.0 release notes ba1f285 Fix typos in test_container.py e8a5448 Add X-Openstack-Request-Id to Access-Control-Expose-Headers f850ff0 SLO: Concurrently HEAD segments 40152f4 Fix up X-Trans-Id-Extra docs 8e122c6 Document X-Openstack-Request-Id in all responses d25216b remove double checks on account/container info 721cd16 Show file parameters as mandatory in swift-*-info man pages 243dd97 Add OPTIONS in manpages b69caea Include debug message for rsync tempfiles e6dea4d adding reno sphinx tree 5de9747 Stop shadowing built-in iter 2f31c04 Py3: Fixes header key dict f4adb2f Fix ZeroDivisionError in reconstructor.stats_line c2ce92a Fix signal handling for daemons with InternalClient 116462d Add probetest for response with duplicate frags 6574ce3 EC: reconstruct using non-durable fragments a4d77d9 Fix KeyError when auditor finds an expired object 3b88933 Make NullLogger more logger like 300d388 Use more specific asserts in test/probe tests 2a75091 Make ECDiskFileReader check fragment metadata e166e19 Remove duplicated 'User-Agent' header from object-updater's requests 33c18c5 Remove executable flag from some test modules 2bd8d05 Suppress unexpected file warnings for rsync temp files d0fe922 Mirror X-Trans-Id to X-Openstack-Request-Id c91ca53 Ignore Range values like "bytes=--0" 30f672e Refactor SLO If-Match / HEAD tests c3bc9f3 Add links for more detailed overview in overview_architecture 99a13d9 Fixed rysnc -> rsync typo 9847796 Set owner of drive-audit recon cache to swift user a8b80f0 update urls to newton 5064ebb Make Content-Disposition support `inline; filename=...` format. c3bc830 SAIO install bash scripts for a novice install 3218f8b Prevent ssync writing bad fragment data to diskfile 77f5b20 Throttle update_auditor_status calls f3be45c Simplify container_updater test code 19102c6 Apply bash error handling consistently in all bash scripts b13b49a EC - eliminate .durable files f0a8af2 Update pyeclib dependency to 1.3.1 1942618 [api-ref] Remove temporary block in conf.py f62df7b Add a configurable URL base to staticweb 53aebba Fix a typo in documentation 2ec4189 Fix when we set state in Spliterator a3dbc13 add assert error information ddd40a0 Add unit test content 45e4fd5 Fix typo: remove redundant 'that' 396baf4 Update test-requirements.txt 7dcfe5a Remove unneeded line from ring.utils.dispersion_report 9b98c89 Revises 'url' to 'URL' and 'json' to 'JSON' 1004509 Fix failing cross-policy-versioning functional test 05642d2 fix word spelling mistake 03b5e76 Improve functional tests and test client 3ae0b80 Remove redundant 'the' 52473ea Use six.moves.urllib.parse instead of urllib ccff366 Update diskfile docstring for get_ondisk_files 2f880f0 Fix typo in docstring a09e427 Fix a typo in proxy-server.conf-sample 7d2f8e5 Fix typo in docstring 31c92ac Update link reference in Swift document 0c8c764 Add functional tests for new versioned_write mode f07dfa2 Add test_PUT/DELETE_bad_timestamp in obj/test_server 19a684d Fix using filter() to meet python2,3 18bb999 Add more comment to authtoken sample options bb24695 updated output for swift stat 1f690df Use ConfigParser instead of SafeConfigParser eea574a Py3: Fixes eventlet ImportError: No module named urllib2 3f00148 Config logABug feature for Swift api-ref 4bcd3d7 Support multi-range GETs for static large objects. 88b14a0 Add functional test for access control (RBAC INFO) with Keystone 5448a55 Add functional test for access control (container ACL) with Keystone Diffstat (except docs and test files) ------------------------------------- .mailmap | 1 + AUTHORS | 540 ++--- CHANGELOG | 62 + REVIEW_GUIDELINES.rst | 2 +- api-ref/source/conf.py | 44 +- api-ref/source/parameters.yaml | 17 +- .../account-containers-list-http-response-json.txt | 3 +- .../account-containers-list-http-response-xml.txt | 3 +- .../samples/objects-list-http-response-json.txt | 3 +- .../samples/objects-list-http-response-xml.txt | 3 +- api-ref/source/storage-account-services.inc | 7 + api-ref/source/storage-container-services.inc | 15 + api-ref/source/storage-object-services.inc | 16 + bin/swift-drive-audit | 3 +- .../api/use_the_content-disposition_metadata.rst | 1 + etc/account-server.conf-sample | 2 +- etc/container-server.conf-sample | 2 +- etc/drive-audit.conf-sample | 1 + etc/object-server.conf-sample | 2 +- etc/proxy-server.conf-sample | 35 +- install-guide/source/controller-install-debian.rst | 2 +- install-guide/source/controller-install-rdo.rst | 2 +- install-guide/source/controller-install-ubuntu.rst | 2 +- install-guide/source/finalize-installation-rdo.rst | 2 +- .../source/finalize-installation-ubuntu-debian.rst | 2 +- install-guide/source/storage-install-rdo.rst | 6 +- .../source/storage-install-ubuntu-debian.rst | 6 +- install-guide/source/verify.rst | 4 +- .../notes/2_11_0_release-ac1d256e455d347e.yaml | 54 + releasenotes/source/conf.py | 345 +++ releasenotes/source/current.rst | 5 + releasenotes/source/index.rst | 10 + releasenotes/source/newton.rst | 6 + requirements.txt | 2 +- swift/cli/recon.py | 8 +- swift/common/constraints.py | 2 +- swift/common/container_sync_realms.py | 2 +- swift/common/daemon.py | 4 +- swift/common/header_key_dict.py | 4 +- swift/common/internal_client.py | 7 +- swift/common/middleware/catch_errors.py | 2 + swift/common/middleware/slo.py | 263 ++- swift/common/middleware/staticweb.py | 37 +- swift/common/middleware/tempurl.py | 34 +- swift/common/middleware/versioned_writes.py | 11 +- swift/common/request_helpers.py | 23 +- swift/common/ring/utils.py | 1 - swift/common/swob.py | 12 +- swift/common/utils.py | 104 +- swift/obj/auditor.py | 12 +- swift/obj/diskfile.py | 255 ++- swift/obj/mem_diskfile.py | 2 +- swift/obj/reconstructor.py | 14 +- swift/obj/server.py | 4 +- swift/obj/ssync_receiver.py | 17 +- swift/obj/ssync_sender.py | 20 +- swift/obj/updater.py | 3 +- swift/proxy/controllers/base.py | 13 +- swift/proxy/controllers/obj.py | 7 +- test-requirements.txt | 27 +- test/functional/__init__.py | 1 + test/functional/swift_test_client.py | 9 +- test/functional/test_access_control.py | 2009 ++++++++++++++++- test/functional/test_account.py | 0 test/functional/test_container.py | 4 +- test/functional/test_object.py | 0 test/probe/test_account_failures.py | 0 .../probe/test_account_get_fake_responses_match.py | 0 test/probe/test_container_failures.py | 0 test/probe/test_container_merge_policy_index.py | 24 +- test/probe/test_empty_device_handoff.py | 0 test/probe/test_object_async_update.py | 2 +- test/probe/test_object_expirer.py | 8 +- test/probe/test_object_failures.py | 0 test/probe/test_object_handoff.py | 122 +- test/probe/test_object_metadata_replication.py | 42 +- test/probe/test_reconstructor_durable.py | 157 -- test/probe/test_reconstructor_rebuild.py | 290 ++- test/probe/test_reconstructor_revert.py | 2 +- test/probe/test_replication_servers_working.py | 8 +- test/probe/test_signals.py | 155 ++ test/probe/test_wsgi_servers.py | 105 - test/unit/__init__.py | 30 + test/unit/cli/test_recon.py | 8 +- .../common/middleware/crypto/test_encrypter.py | 22 +- test/unit/common/middleware/test_except.py | 44 +- test/unit/common/middleware/test_slo.py | 515 +++-- test/unit/common/middleware/test_staticweb.py | 53 + test/unit/common/middleware/test_tempurl.py | 4 +- .../common/middleware/test_versioned_writes.py | 2 +- test/unit/common/test_constraints.py | 114 +- test/unit/common/test_daemon.py | 31 +- test/unit/common/test_direct_client.py | 68 +- test/unit/common/test_header_key_dict.py | 36 + test/unit/common/test_internal_client.py | 6 +- test/unit/common/test_manager.py | 6 +- test/unit/common/test_storage_policy.py | 0 test/unit/common/test_swob.py | 10 +- test/unit/common/test_utils.py | 114 + test/unit/container/test_backend.py | 12 +- test/unit/container/test_updater.py | 79 +- test/unit/obj/common.py | 56 +- test/unit/obj/test_auditor.py | 109 +- test/unit/obj/test_diskfile.py | 2345 +++++++++++++------- test/unit/obj/test_reconstructor.py | 128 +- test/unit/obj/test_server.py | 243 +- test/unit/obj/test_ssync.py | 480 +++- test/unit/obj/test_ssync_receiver.py | 56 +- test/unit/obj/test_ssync_sender.py | 9 +- test/unit/proxy/controllers/test_base.py | 7 +- test/unit/proxy/controllers/test_obj.py | 20 +- test/unit/proxy/test_server.py | 160 +- tox.ini | 4 + 141 files changed, 7921 insertions(+), 2461 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index eed45fa..9f1582e 100644 --- a/requirements.txt +++ b/requirements.txt @@ -12 +12 @@ xattr>=0.4 -PyECLib>=1.2.0 # BSD +PyECLib>=1.3.1 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index b5cc75f..04ea66b 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -6,11 +6,11 @@ -hacking>=0.10.0,<0.11 -coverage -nose -nosexcover -nosehtmloutput -openstackdocstheme>=1.0.3 # Apache-2.0 -oslosphinx -sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3 # BSD -os-api-ref>=0.1.0 # Apache-2.0 -os-testr>=0.4.1 -mock>=1.0 +hacking>=0.11.0,<0.12 # Apache-2.0 +coverage>=3.6 # Apache-2.0 +nose # LGPL +nosexcover # BSD +nosehtmloutput>=0.0.3 # Apache-2.0 +openstackdocstheme>=1.5.0 # Apache-2.0 +oslosphinx>=4.7.0 # Apache-2.0 +sphinx!=1.3b1,<1.4,>=1.2.1 # BSD +os-api-ref>=1.0.0 # Apache-2.0 +os-testr>=0.8.0 # Apache-2.0 +mock>=2.0 # BSD @@ -18 +18,2 @@ python-swiftclient -python-keystoneclient>=1.3.0 +python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0 +reno>=1.8.0 # Apache-2.0 @@ -21 +22 @@ python-keystoneclient>=1.3.0 -bandit>=1.1.0 # Apache-2.0 +bandit>=1.1.0 # Apache-2.0