We are jazzed to announce the release of: swift 2.13.0: OpenStack Object Storage This release is part of the ocata stable release series. Download the package from: https://tarballs.openstack.org/swift/ For more details, please see below. 2.13.0 ^^^^^^ New Features ************ * Improved performance by eliminating an unneeded directory structure hash. * Optimized the common case for hashing filesystem trees, thus eliminating a lot of extraneous disk I/O. * Updated the *hashes.pkl* file format to include timestamp information for race detection. Also simplified hashing logic to prevent race conditions and optimize for the common case. * The erasure code reconstructor will now shuffle work jobs across all disks instead of going disk-by-disk. This eliminates single-disk I/O contention and allows continued scaling as concurrency is increased. * Erasure code reconstruction handles moving data from handoff nodes better. Instead of moving the data to another handoff, it waits until it can be moved to a primary node. * Temporary URLs now support one common form of ISO 8601 timestamps in addition to Unix seconds-since-epoch timestamps. The ISO 8601 format accepted is '%Y-%m-%dT%H:%M:%SZ'. This makes TempURLs more user- friendly to produce and consume. * Listing containers in accounts with json or xml now includes a *last_modified* time. This does not change any on-disk data, but simply exposes the value to offer consistency with the object listings on containers. * I/O priority is now supported on AArch64 architecture. Upgrade Notes ************* * If you upgrade and roll back, you must delete all *hashes.pkl* files. Deprecation Notes ***************** * If using erasure coding with ISA-L in rs_vand mode and 5 or more parity fragments, Swift will emit a warning. This is a configuration that is known to harm data durability. In a future release, this warning will be upgraded to an error unless the policy is marked as deprecated. All data in an erasure code storage policy using isa_l_rs_vand with 5 or more parity should be migrated as soon as possible. Please see https://bugs.launchpad.net/swift/+bug/1639691 for more information. * The erasure code reconstructor *handoffs_first* option has been deprecated in favor of *handoffs_only*. *handoffs_only* is far more useful, and just like *handoffs_first* mode in the replicator, it gives the operator the option of forcing the consistency engine to focus solely on revert (handoff) jobs, thus improving the speed of rebalances. The *handoffs_only* behavior is somewhat consistent with the replicator's *handoffs_first* option (any error on any handoff in the replicator will make it essentially handoff only forever) but the *handoff_only* option does what you want and is named correctly in the reconstructor. * The default for *object_post_as_copy* has been changed to False. The option is now deprecated and will be removed in a future release. If your cluster is still running with post-as-copy enabled, please update it to use the "fast-post" method. Future versions of Swift will not support post-as-copy, and future features will not be supported under post-as-copy. ("Fast-post" is where *object_post_as_copy* is false). Bug Fixes ********* * Fixed a bug where the ring builder would not allow removal of a device when min_part_seconds_left was greater than zero. * PUT subrequests generated from a client-side COPY will now properly log the SSC (server-side copy) Swift source field. See https://docs.openstack.org/developer/swift/logs.html#swift-source for more information. * Fixed a bug where an SLO download with a range request may have resulted in a 5xx series response. * SLO manifest PUT requests can now be properly validated by sending an ETag header of the md5 sum of the concatenated md5 sums of the referenced segments. * Fixed the stats calculation in the erasure code reconstructor. * Rings with min_part_hours set to zero will now only move one partition replica per rebalance, thus matching behavior when min_part_hours is greater than zero. Other Notes *********** * Various other minor bug fixes and improvements. Changes in swift 2.12.0..2.13.0 ------------------------------- 11bd6c8 2.13.0 authors/changelog updates da55701 Deprecate broken handoffs_first in favor of handoffs_only ae236b5 Hisashi Osanai's responsibility has been changed c569e78 [Trivial]Fix some type errors in ../sample.conf 86127ec Add Apache License 2.0 description in ../conf.py 04502a9 Fix test comment and remove extra parameter 5bb3ea1 Ignore cache files when comparing synced parts 9681a83 Fix test_delete_propagate probe test 6249945 Fix misleading hash invalidations test comments 1c4a16a Typo fix: curent => current 44fc037 Fix_typo: "subsitute" -> "substitute" 702c804 Use https instead of http for git.openstack.org b1c36dc Fix test_replicator assertion f713bb1 Fix swift-get-nodes arg parsing for missing ring 67d545e Fix timing test error when rebalancing 5817f80 Handle log message interpolation by the logger aa71d7e Better optimistic lock in get_hashes eadb01b Do not revert fragments to handoffs 90d517f Remove some cruft in test_proxy_logging 13f1fc0 Clean up EC overview docs a bit c7bffd6 Warn about using EC with isa_l_rs_vand and nparity >= 5 e8589b2 Correct ringbuilder's set_weight usage string 2f0ab78 Shuffle disks and parts in reconstructor b5530f4 Bring docs inline with changes to tox envs 98e9554 remove func-fast-post from tox.ini 51727c5 ISO 8601 timestamps for tempurl 9b0a1cc Fix inline tempurl/formpost signature examples 031ba11 IO Priority support on the AArch64 architecture 1b7aabd remove reference to deprecated tool 442cc1d Fix race in new partitions detecting new/invalid suffixes. c33f2b4 Simplify get_different_suffix_df args 4ee20db Default object_post_as_copy to False c4d455c Minor dev guidelines formatting fixes 47901ea Rebalance with min_part_seconds_left > 0 bbefaca Fix unnecessary for-loop and mis docs d4de550 Remove broken links ebf2731 Removed redundant 'is' f0122b6 Cleanup tests from empty suffix quarantined db fix a768563 add swift_source to PUT request part of COPY bcec1f4 Fix object server tests to include content-type headers 95c7c01 Optimize noop case for suffix rehash e772cf9 Extract test pattern to helper 69f7be9 Move documented reclaim_age option to correct location 1a8085f Test current reclaim_age handling 4a2adfe Make timestamp setting to be consistent in test loop 0590d20 Fix download resumption for new SLOs. dcb432b Trivial follow up to addition of last modified in container listings 652276f Support last modified on listing containers 244e640 Tighten the move-one-replica test 8abe249 added a quote 0fd7257 Fix flaky expirer probe test aa17ae1 Allow Hacking H401, H403 check dc0a592 Donagh McCabe has been reassigned to different project. cbb99f9 Replace six iteration methods with standard ones fd6edf7 Confirm receipt of SLO PUT with etag a67bb2e Removes unnecessary utf-8 encoding 9f30c5d Make tempurl functional tests clean up account keys a0345cb Move bandit to pep8 ffd5194 Raise ValueError if a config section does not exist b9f796b Remove duplicate setup code from test_tempurl.py d58064c Update the tag for Swift tox test b09360d Fix stats calculation in object-reconstructor 609b518 Refactor recon to use single md5_hash_for_file function dfc8e61 Add test for object updater user-agent header ce26e78 For any part, only one replica can move in a rebalance Diffstat (except docs and test files) ------------------------------------- .mailmap | 6 +- AUTHORS | 12 +- CHANGELOG | 86 +++ REVIEW_GUIDELINES.rst | 3 + api-ref/source/conf.py | 2 +- api-ref/source/parameters.yaml | 23 +- .../samples/account-containers-list-response.json | 6 +- .../samples/account-containers-list-response.xml | 2 + bin/swift-get-nodes | 45 +- etc/object-server.conf-sample | 23 +- etc/proxy-server.conf-sample | 14 +- install-guide/source/conf.py | 2 +- .../notes/2_13_0_release-875e1fb1ef59f015.yaml | 83 +++ releasenotes/source/conf.py | 12 + swift/account/backend.py | 7 +- swift/account/reaper.py | 2 +- swift/account/utils.py | 18 +- swift/cli/info.py | 32 ++ swift/cli/recon.py | 31 +- swift/cli/ringbuilder.py | 26 +- swift/common/constraints.py | 6 +- swift/common/daemon.py | 10 +- swift/common/db_replicator.py | 3 - swift/common/middleware/copy.py | 23 +- swift/common/middleware/formpost.py | 4 +- swift/common/middleware/recon.py | 37 +- swift/common/middleware/slo.py | 63 ++- swift/common/middleware/tempurl.py | 26 +- swift/common/ring/builder.py | 85 +-- swift/common/storage_policy.py | 22 + swift/common/utils.py | 33 +- swift/common/wsgi.py | 5 +- swift/container/reconciler.py | 3 + swift/obj/auditor.py | 5 +- swift/obj/diskfile.py | 181 ++++--- swift/obj/expirer.py | 5 +- swift/obj/reconstructor.py | 117 ++-- swift/obj/replicator.py | 20 +- swift/obj/ssync_sender.py | 2 +- test/__init__.py | 7 +- test/functional/swift_test_client.py | 7 +- test/functional/test_access_control.py | 2 - test/functional/test_account.py | 78 ++- test/functional/test_dlo.py | 18 +- test/functional/test_slo.py | 84 ++- test/functional/test_tempurl.py | 150 +++-- test/functional/test_versioned_writes.py | 41 +- test/probe/test_object_expirer.py | 3 + test/probe/test_reconstructor_revert.py | 48 +- test/probe/test_replication_servers_working.py | 11 +- test/sample.conf | 10 +- test/unit/account/test_backend.py | 10 +- test/unit/account/test_reaper.py | 4 +- test/unit/account/test_server.py | 114 ++-- test/unit/cli/test_info.py | 388 ++++++++++++- test/unit/cli/test_recon.py | 12 +- test/unit/cli/test_ringbuilder.py | 59 +- .../common/middleware/crypto/test_keymaster.py | 11 +- test/unit/common/middleware/test_copy.py | 51 +- test/unit/common/middleware/test_proxy_logging.py | 8 - test/unit/common/middleware/test_recon.py | 122 +---- test/unit/common/middleware/test_slo.py | 41 ++ test/unit/common/middleware/test_tempurl.py | 101 ++-- test/unit/common/ring/test_builder.py | 31 +- test/unit/common/ring/test_utils.py | 11 +- test/unit/common/test_daemon.py | 9 + test/unit/common/test_db_replicator.py | 228 +++----- test/unit/common/test_storage_policy.py | 108 ++++ test/unit/common/test_utils.py | 146 ++++- test/unit/obj/common.py | 34 +- test/unit/obj/test_auditor.py | 58 +- test/unit/obj/test_diskfile.py | 603 +++++++++++++++++++-- test/unit/obj/test_reconstructor.py | 133 ++++- test/unit/obj/test_replicator.py | 148 ++--- test/unit/obj/test_server.py | 188 ++++++- test/unit/obj/test_ssync.py | 39 +- test/unit/obj/test_ssync_sender.py | 40 +- test/unit/obj/test_updater.py | 111 ++-- tox.ini | 11 +- 92 files changed, 3341 insertions(+), 1369 deletions(-)