[release-announce] swift 2.28.0 (xena)

We are pumped to announce the release of: swift 2.28.0: OpenStack Object Storage This release is part of the xena release series. The source is available from: https://opendev.org/openstack/swift Download the package from: https://tarballs.openstack.org/swift/ Please report issues through: https://bugs.launchpad.net/swift/+bugs For more details, please see below. 2.28.0 ^^^^^^ New Features ************ * "swift-manage-shard-ranges" improvements: * Exit codes are now applied more consistently: * 0 for success * 1 for an unexpected outcome * 2 for invalid options * 3 for user exit As a result, some errors that previously resulted in exit code 2 will now exit with code 1. * Added a new 'repair' command to automatically identify and optionally resolve overlapping shard ranges. * Added a new 'analyze' command to automatically identify overlapping shard ranges and recommend a resolution based on a JSON listing of shard ranges such as produced by the 'show' command. * Added a "--includes" option for the 'show' command to only output shard ranges that may include a given object name. * Added a "--dry-run" option for the 'compact' command. * The 'compact' command now outputs the total number of compactible sequences. * Partition power increase improvements: * The relinker now spawns multiple subprocesses to process disks in parallel. By default, one worker is spawned per disk; use the new "--workers" option to control how many subprocesses are used. Use "--workers=0" to maintain the previous behavior. * The relinker can now target specific storage policies or partitions by using the new "--policy" and "--partition" options. * More daemons now support systemd notify sockets. * The container-reconciler now scales out better with new "processes", "process", and "concurrency" options, similar to the object-expirer. Deprecation Notes ***************** * Container sharding deprecations: * Added a new config option, "shrink_threshold", to specify the absolute size below which a shard will be considered for shrinking. This overrides the "shard_shrink_point" configuration option, which expressed this as a percentage of "shard_container_threshold". "shard_shrink_point" is now deprecated. * Similar to above, "expansion_limit" was added as an absolute- size replacement for the now-deprecated "shard_shrink_merge_point" configuration option. Bug Fixes ********* * Sharding improvements: * When building a listing from shards, any failure to retrieve listings will result in a 503 response. Previously, failures fetching a partiucular shard would result in a gap in listings. * Container-server logs now include the shard path in the referer field when receiving stat updates. * Added a new config option, "rows_per_shard", to specify how many objects should be in each shard when scanning for ranges. The default is "shard_container_threshold / 2", preserving existing behavior. * Added a new config option, "minimum_shard_size". When scanning for shard ranges, if the final shard would otherwise contain fewer than this many objects, the previous shard will instead be expanded to the end of the namespace (and so may contain up to "rows_per_shard + minimum_shard_size" objects). This reduces the number of small shards generated. The default value is "rows_per_shard / 5". * The sharder now correctly identifies and fails audits for shard ranges that overlap exactly. * The sharder and swift-manage-shard-ranges now consider total row count (instead of just object count) when deciding whether a shard is a candidate for shrinking. * If the sharder encounters shard range gaps while cleaving, it will now log an error and halt sharding progress. Previously, rows may not have been moved properly, leading to data loss. * Sharding cycle time and last-completion time are now available via swift-recon. * Fixed an issue where resolving overlapping shard ranges via shrinking could prematurely mark created or cleaved shards as active. * S3 API improvements: * Added an option, "ratelimit_as_client_error", to return 429s for rate-limited responses. Several clients/SDKs have seem to support retries with backoffs on 429, and having it as a client error cleans up logging and metrics. By default, Swift will respond 503, matching AWS documentation. * Fixed a server error in bucket listings when "s3_acl" is enabled and staticweb is configured for the container. * Fixed a server error when a client exceeds "client_timeout" during an upload. Now, a "RequestTimeout" error is correctly returned. * Fixed a server error when downloading multipart uploads/static large objects that have missing or inaccessible segments. This is a state that cannot arise in AWS, so a new "BrokenMPU" error is returned, indicating that retrying the request is unlikely to succeed. * Fixed several issues with the prefix, marker, and delimiter parameters that would be mirrored back to clients when listing buckets. * Partition power increase fixes: * The relinker now performs eventlet-hub selection the same way as other daemons. In particular, "epolls" will no longer be selected, as it seemed to cause occassional hangs. * Partitions that encountered errors during relinking are no longer marked as completed in the relinker state file. This ensures that a subsequent relink will retry the failed partitions. * Partition cleanup is more robust, decreasing the likelihood of leaving behind mostly-empty partitions from the old partition power. * Improved relinker progress logging, and started collecting progress information for swift-recon. * Cleanup is more robust to files and directories being deleted by another process. * The relinker better handles data found from earlier partition power increases. * The relinker better handles tombstones found for the same object but with different inodes. * The reconciler now defers working on policies that have a partition power increase in progress to avoid issues with concurrent writes. * Erasure coding fixes: * Added the ability to quarantine EC fragments that have no (or few) other fragments in the cluster. A new configuration option, "quarantine_threshold", in the reconstructor controls the point at the fragment will be quarantined; the default (0) will never quarantine. Only fragments older than "quarantine_age" (default: "reclaim_age") may be quarantined. Before quarantining, the reconstructor will attempt to fetch fragments from handoff nodes in addition to the usual primary nodes; a new "request_node_count" option (default "2 * replicas") limits the total number of nodes to contact. * Added a delay before deleting non-durable data. A new configuration option, "commit_window" in the "[DEFAULT]" section of object-server.conf, adjusts this delay; the default is 60 seconds. This improves the durability of both back-dated PUTs (from the reconciler or container-sync, for example) and fresh writes to handoffs by preventing the reconstructor from deleting data that the object-server was still writing. * Improved proxy-server and object-reconstructor logging when data cannot be reconstructed. * Fixed an issue where some but not all fragments having metadata applied could prevent reconstruction of missing fragments. * Server-side copying of erasure-coded data to a replicated policy no longer copies EC sysmeta. The previous behavior had no material effect, but could confuse operators examining data on disk. * Python 3 fixes: * Fixed a server error when performing a PUT authorized via tempurl with some proxy pipelines. * Fixed a server error during GET of a symlink with some proxy pipelines. * Fixed an issue with logging setup when /dev/log doesn't exist or is not a UNIX socket. * The dark-data audit watcher now skips objects younger than a new configurable "grace_age" period. This avoids issues where data could be flagged, quarantined, or deleted because of listing consistency issues. The default is one week. * The dark-data audit watcher now requires that all primary locations for an object's container agree that the data does not appear in listings to consider data "dark". Previously, a network partition that left an object node isolated could cause it to quarantine or delete all of its data. * "EPIPE" errors no longer log tracebacks. * The account and container auditors now log and update recon before going to sleep. * The object-expirer logs fewer client disconnects. * "swift-recon-cron" now includes the last time it was run in the recon information. * "EIO" errors during read now cause object diskfiles to be quarantined. * The formpost middleware now properly supports uploading multiple files with different content-types. * Various other minor bug fixes and improvements. Changes in swift 2.27.0..2.28.0 ------------------------------- a8f151286 AUTHORS/CHANGELOG for 2.28.0 5759072d2 s3api: Pass through 409s from SLO eb969fdee container-reconciler: support multiple processes 1760a0cb9 Fix the sysctl parameter used to tune connections 92aef484b Dark Data Watcher: switch to agreement across the whole ring 2696a79f0 reconstructor: retire nondurable_purge_delay option bbaed18e9 diskfile: don't remove recently written non-durables 775ad9a56 Remove redundant usage of collections(.abc).Mapping f911ffac3 trivial: Use own_shard_range instead of doing an extra query 7b38d11c0 trivial: Clean up a DeprecationWarning e00ae0337 Switch get(full)argspec function according to python version 510fe6962 Add some more arm64 non-voting tests 02dd0187e Remove support for --link-check-limit f29ea9d04 reconciler: concurreny follow-up 4e52d946b Add concurrency to reconciler e491693e3 reconciler: PPI aware reconciler 9b94c278f sharder: add more validation checks on config 6709bb889 sharder: If saving own_shard_range use no_default=True da4010ae2 Sharding: root audit epoch reset warning 26c3d819b Drain and close more internal client requests 4c1320c4c Do not use epoll in relinker 2a593174a sharder: avoid small tail shards a87317db6 sharder: support rows_per_shard in config file 2fd5b87dc reconstructor: make quarantine delay configurable 391cc713e Handle ClientDisconnect on s3api object PUT 7c0ff5797 func tests: Better-tolerate deleted containers in account listings f25592f3c Quarantine on IOErrors while reading 45a5ecc8a Have expirer use IC's delete_object 574897ae2 relinker: tolerate existing tombstone with same timestamp e0ba83eca Sharder: Add root_audit overlap stats 95e031645 Make dark data watcher ignore the newly updated objects 69e9a1acb reconciler: Tolerate 503s on HEAD 85e36f712 recon: refactor common recon names into a common location 50ba8e697 bufferedhttp: Tolerate socket being None c2ae60251 Include status code when we fail to parse an SLO-delete response 2934818d6 reconstructor: Delay purging reverted non-durable datafiles 40aace89f Capture logs when running custom daemons in probe tests 0d05a8ff3 Quiet EPIPE tracebacks e40cf1ec5 Return 503 for container listings when shards are deleted 39ad468df Add async_pending_last time to object.recon 8e75faff8 Run in-process func tests under py38 eacebbb16 Allow floats for a couple more intervals bead9f1ca relinker: Remove replication locks for empty parts d183b41c9 Switch IRC references from freenode to OFTC e4c656bc5 Add unit test for missing whole EC fragments 79b263f5e Use a less bogus credit for Melissa Ma Lei 2b5853f41 Quote paths before sending them to swob.Request.blank e495c3bbe sharder: Send broker path as referer when updating root 705a88bb1 docker: Install cffi and cryptography from system packages 09ead6909 Add warn level logging if we hit un-recoverable EC rebuild 1ba17f635 Get TestDarkDataQuarantining passing when policy-0 is erasure-coded 41a3e1ff5 Add non-voting swift python3 unit test jobs f3f918411 Individual content types for multi form upload 6bfd93d88 symlink: Ensure headers are returned as a list on py3 18f20daf3 Add absolute values for shard shrinking config options f7fd99a88 Use ContainerSharderConf class in sharder and manage-shard-ranges 77530136f tempurl: Fix PUT upload to temp url on py3 1b183f221 sharder: report perfectly overlapping shard ranges bbe189b53 s-m-s-r: use argparse mutually exclusive group 5d1d48c77 swift-manage-shard-ranges: output total compactible sequences 6e5620944 swift-get-nodes: move --path-as-is to end of curl commands 1db281611 relinker: Rehash the parts actually touched when relinking 568036f1b manage-shard-ranges: add --dry-run option for compact and repair c20d61101 Drop the unnecessary sudo from probe tests 46ea3aeae Quarantine stale EC fragments after checking handoffs 4ce907a4a relinker: Add /recon/relinker endpoint and drop progress stats 04fab8d9a Document why TestReconstructorRebuildUTF8 is skipped on py3 bcecddd51 Consider tombstone count before shrinking a shard eeaac713f reconstructor: gather rebuild fragments by x-data-timestamp f140de75e relinker: let --workers accept auto f0e909749 Remove test-requirement on fixtures c374a7a85 Allow floats for all intervals 926c61bcc relinker: Only mark partitions "done" if there were no (new) errors ed6586c46 sharder: stall cleaving at shard range gaps 29418998b Fix shrinking making acceptors prematurely active ba00ff437 Add sharding to swift-recon ab8accbb0 reconstructor: extract closure for handle_response 2a312d1cd Cleanup tests' import of debug_logger 7960097f0 reconstructor: log more details when rebuild fails 8823b45b9 docs: Get rid of useless page 5516bf46c Add more detail to ECFragGetter logging f2a4c50dc Include sharding cycle time in recon 9d6006f64 auditors: Log and dump recon *before* sleeping bacef722a Use underscores instead of dashes in setup.cfg 7087fb0d7 object: Plumb logger_thread_locals through _finalize_put 717d21ccb fix not clear cause for invalid username 4a4d89968 Refactor EC multipart/byteranges control flow fa3109ab5 Enable systemd notify sockets for more daemons e76ba2107 relinker: Add start/end logs to parallel_process 7f35b1cc8 swift-manage-shard-ranges: fix exit codes 751deb988 Fix reclaim to use RECLAIM_PAGE_SIZE batches 4cb52b44d Refactor db auditors into a db_auditor base class c8de76c7f swift-account-audit: Log the bad status 122840cc0 probe test: use helper functions more widely c13c9cc67 Update AUTHORS e53c82cd3 sharder: Prevent ValueError when no cleaving contexts 1895213d2 Update some constraints for py2 7bbc73a91 sharding: constrain fill_gaps to own shard range bounds ecb5aa120 relinker: trivial comment and test fixes abfa6bee7 relinker: Parallelize per disk 20336f177 Use debug_logger instead of FakeLogger in relinker tests 3bdd01cf4 relinker: retry links from older part powers 2a672b678 py2: diskfile.valid_suffix should allow unicode 399bda8a4 ec: Don't copy EC metadata to replicated objects d55bfa7fb s-m-s-r: read shard_ranges from stdin fc7c0afbe swift-manage-shard-ranges analyze: accept incomplete shard data ade3b2863 diskfile: Prevent get_hashes from creating missing partition dirs f819fc8c2 Give functional tests another chance to pass 71a4aea31 Update docs to discourage policy names being numbers 2a3a0fb75 Add rolling upgrade job coming from stable/wallaby ef44905b3 Get rid of strip_self a967d4729 relinker: Accept policy names, too 4bc2b3ef9 Prevent instantiation of ShardRangeOuterBound c9c42c07c swift-manage-shard-ranges: add repair and analyze commands 0e4289fbd relinker: Add policy to relinker progress output b17dd7ec7 Make ShardRange.OuterBound a proper singleton 3a41cbe67 relinker: Allow multiple policies to be specified 812ed1ba0 relinker: add --partition option c6e7dac01 Update master for stable/wallaby 4ef7b10e0 relinker: Tolerate missing files when cleaning up partitions 76feb65cb Add unit tests for relinker cleanup 7e278291f relinker: Move filters & hooks to be methods 55386c836 Add --includes to s-m-s-r edc3f4d97 Add test for relinking when the object-server failed to do it a1350a2c2 relinker: make relink step a subset of cleanup step 2b4ec5a45 relinker: add --policy option 9ab9acb03 gate: Make swift-multinode-rolling-upgrade voting again d54083771 Don't require swift be installed to have passing manager tests ec37893b8 Only test with &-delimited query strings 0b129509c relinker: refactor to share common code e572938af Use headless option for cors functional test runner e35365df5 s3api: Add config option to return 429s on ratelimit acf72e880 Make more use of get_partition_for_hash in unit tests 90660c90d relinker: use abs path index in part power replace c7d56fa9c py36: Fix syslog fallback to UDP 48e625478 Fix systemd notify unit test on macos 42f4af45a Add test coverage for audit_location_generator yield_hashes arg e318d15b8 obj: Include timeout value when logging long-running rsyncs d022781bc s3api: Transfer REMOTE_USER when using s3_acl 46a6a09d5 Create a separate doc for audit watcher f0d406127 Test proxy-server.conf-sample e973e15cf Get func tests passing with a minimal proxy pipeline 66da8eaee Remove babel.cfg 10c24e951 s3api: Fix prefix/delimiter/marker quoting f0b8790c1 s3api: Fix blank delimiter handling Diffstat (except docs and test files) ------------------------------------- .mailmap | 1 + .stestr.conf | 2 + .testr.conf | 4 - .zuul.yaml | 113 +- AUTHORS | 9 +- CHANGELOG | 214 ++ CONTRIBUTING.rst | 5 +- README.rst | 2 +- babel.cfg | 2 - bin/swift-account-audit | 10 +- bin/swift-recon-cron | 11 +- etc/account-server.conf-sample | 8 +- etc/container-reconciler.conf-sample | 15 +- etc/container-server.conf-sample | 51 +- etc/container-sync-realms.conf-sample | 2 +- etc/object-expirer.conf-sample | 4 +- etc/object-server.conf-sample | 95 +- etc/proxy-server.conf-sample | 11 +- etc/swift.conf-sample | 42 +- lower-constraints.txt | 2 +- py2-constraints.txt | 4 +- .../notes/2_28_0_release-f2515e07fb61cd01.yaml | 235 ++ releasenotes/source/index.rst | 2 + releasenotes/source/wallaby.rst | 6 + setup.cfg | 20 +- swift/account/auditor.py | 128 +- swift/account/reaper.py | 2 +- swift/cli/info.py | 3 +- swift/cli/manage_shard_ranges.py | 502 +++- swift/cli/recon.py | 75 +- swift/cli/relinker.py | 1136 +++++--- swift/common/bufferedhttp.py | 18 +- swift/common/container_sync_realms.py | 2 +- swift/common/daemon.py | 1 + swift/common/db.py | 119 +- swift/common/db_auditor.py | 170 ++ swift/common/db_replicator.py | 19 +- swift/common/exceptions.py | 6 +- swift/common/internal_client.py | 46 +- swift/common/middleware/crypto/crypto_utils.py | 3 +- swift/common/middleware/formpost.py | 11 +- swift/common/middleware/recon.py | 42 +- .../common/middleware/s3api/controllers/bucket.py | 118 +- .../middleware/s3api/controllers/multi_delete.py | 4 +- swift/common/middleware/s3api/s3api.py | 2 + swift/common/middleware/s3api/s3request.py | 13 +- swift/common/middleware/s3api/s3response.py | 6 + swift/common/middleware/s3api/utils.py | 1 + swift/common/middleware/symlink.py | 2 +- swift/common/middleware/tempauth.py | 6 +- swift/common/middleware/tempurl.py | 2 +- swift/common/recon.py | 30 + swift/common/ring/ring.py | 10 +- swift/common/storage_policy.py | 67 +- swift/common/utils.py | 354 ++- swift/common/wsgi.py | 15 +- swift/container/auditor.py | 111 +- swift/container/backend.py | 137 +- swift/container/reconciler.py | 107 +- swift/container/sharder.py | 424 ++- swift/container/sync.py | 2 +- swift/container/updater.py | 7 +- swift/obj/auditor.py | 7 +- swift/obj/diskfile.py | 110 +- swift/obj/expirer.py | 15 +- swift/obj/reconstructor.py | 470 +++- swift/obj/replicator.py | 16 +- swift/obj/ssync_sender.py | 2 +- swift/obj/updater.py | 7 +- swift/obj/watchers/dark_data.py | 65 +- swift/proxy/controllers/container.py | 12 +- swift/proxy/controllers/obj.py | 129 +- swift/proxy/server.py | 17 +- test-requirements.txt | 3 +- test/cors/main.py | 12 +- test/debug_logger.py | 133 +- test/functional/__init__.py | 3 +- test/functional/s3api/test_bucket.py | 99 + test/functional/swift_test_client.py | 18 +- test/functional/test_access_control.py | 9 +- test/functional/test_staticweb.py | 5 +- test/functional/test_tempurl.py | 10 + test/functional/test_versioned_writes.py | 5 + test/probe/common.py | 26 +- test/probe/test_dark_data.py | 11 +- test/probe/test_object_partpower_increase.py | 19 +- test/probe/test_reconstructor_rebuild.py | 161 +- test/probe/test_reconstructor_revert.py | 8 +- test/probe/test_sharder.py | 402 ++- test/unit/__init__.py | 47 +- test/unit/account/test_auditor.py | 135 +- test/unit/account/test_backend.py | 16 +- test/unit/account/test_reaper.py | 13 +- test/unit/account/test_server.py | 17 +- test/unit/cli/test_info.py | 9 +- test/unit/cli/test_manage_shard_ranges.py | 1074 +++++++- test/unit/cli/test_recon.py | 108 + test/unit/cli/test_relinker.py | 2883 +++++++++++++++++--- .../common/middleware/crypto/test_crypto_utils.py | 4 +- .../common/middleware/crypto/test_decrypter.py | 6 +- .../common/middleware/crypto/test_encrypter.py | 5 +- .../common/middleware/crypto/test_encryption.py | 5 +- test/unit/common/middleware/helpers.py | 5 +- test/unit/common/middleware/s3api/__init__.py | 7 +- test/unit/common/middleware/s3api/test_bucket.py | 82 +- .../common/middleware/s3api/test_multi_delete.py | 46 + test/unit/common/middleware/s3api/test_obj.py | 21 + test/unit/common/middleware/s3api/test_s3_acl.py | 1 + test/unit/common/middleware/s3api/test_s3api.py | 4 +- .../unit/common/middleware/s3api/test_s3request.py | 4 +- test/unit/common/middleware/s3api/test_s3token.py | 63 +- test/unit/common/middleware/s3api/test_utils.py | 2 + test/unit/common/middleware/test_bulk.py | 2 +- test/unit/common/middleware/test_container_sync.py | 6 +- test/unit/common/middleware/test_copy.py | 3 +- test/unit/common/middleware/test_formpost.py | 62 + test/unit/common/middleware/test_keystoneauth.py | 18 +- .../unit/common/middleware/test_listing_formats.py | 2 +- test/unit/common/middleware/test_proxy_logging.py | 2 +- test/unit/common/middleware/test_ratelimit.py | 6 +- test/unit/common/middleware/test_read_only.py | 24 +- test/unit/common/middleware/test_recon.py | 174 +- .../common/middleware/test_subrequest_logging.py | 4 +- test/unit/common/middleware/test_symlink.py | 1 + test/unit/common/middleware/test_tempauth.py | 8 + test/unit/common/middleware/test_tempurl.py | 39 +- test/unit/common/test_bufferedhttp.py | 44 + test/unit/common/test_container_sync_realms.py | 35 +- test/unit/common/test_daemon.py | 2 +- test/unit/common/test_db.py | 160 +- test/unit/common/test_db_auditor.py | 151 + test/unit/common/test_db_replicator.py | 71 +- test/unit/common/test_direct_client.py | 3 +- test/unit/common/test_internal_client.py | 112 +- test/unit/common/test_manager.py | 16 +- test/unit/common/test_memcached.py | 2 +- test/unit/common/test_recon.py | 31 + test/unit/common/test_request_helpers.py | 18 +- test/unit/common/test_storage_policy.py | 31 +- test/unit/common/test_swob.py | 12 +- test/unit/common/test_utils.py | 534 +++- test/unit/common/test_wsgi.py | 9 +- test/unit/container/test_auditor.py | 129 +- test/unit/container/test_backend.py | 344 ++- test/unit/container/test_reconciler.py | 255 +- test/unit/container/test_replicator.py | 10 +- test/unit/container/test_server.py | 104 +- test/unit/container/test_sharder.py | 1106 ++++++-- test/unit/container/test_sync.py | 2 +- test/unit/container/test_sync_store.py | 2 +- test/unit/container/test_updater.py | 8 +- test/unit/helpers.py | 5 +- test/unit/obj/test_auditor.py | 267 +- test/unit/obj/test_diskfile.py | 262 +- test/unit/obj/test_expirer.py | 27 +- test/unit/obj/test_reconstructor.py | 1228 ++++++++- test/unit/obj/test_replicator.py | 18 +- test/unit/obj/test_server.py | 113 +- test/unit/obj/test_ssync.py | 184 +- test/unit/obj/test_ssync_receiver.py | 5 +- test/unit/obj/test_ssync_sender.py | 4 +- test/unit/obj/test_updater.py | 13 +- test/unit/proxy/controllers/test_base.py | 3 +- test/unit/proxy/controllers/test_container.py | 123 +- test/unit/proxy/controllers/test_obj.py | 661 ++++- test/unit/proxy/test_server.py | 125 +- test/unit/proxy/test_sysmeta.py | 5 +- tools/playbooks/common/install_dependencies.yaml | 4 +- tools/playbooks/cors/run.yaml | 2 +- 186 files changed, 14306 insertions(+), 3371 deletions(-) Requirements updates -------------------- diff --git a/test-requirements.txt b/test-requirements.txt index 862a0e921..7eb48af72 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -11 +11 @@ nosehtmloutput>=0.0.3 # Apache-2.0 -os-testr>=0.8.0 # Apache-2.0 +stestr>=2.0.0 # Apache-2.0 @@ -19 +18,0 @@ requests-mock>=1.2.0 # Apache-2.0 -fixtures>=3.0.0 # Apache-2.0/BSD