We are gleeful to announce the release of: swift 2.24.0: OpenStack Object Storage This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/swift Download the package from: https://tarballs.openstack.org/swift/ Please report issues through: https://bugs.launchpad.net/swift/+bugs For more details, please see below. 2.24.0 ^^^^^^ New Features ************ * Added a new object versioning mode, with APIs for querying and accessing old versions. For more information, see the documentation. * Added support for S3 versioning using the above new mode. * Added a new middleware to allow accounts and containers to opt-in to RFC-compliant ETags. For more information, see the documentation. Clients should be aware of the fact that ETags may be quoted for RFC compliance; this may become the default behavior in some future release. * Proxy, account, container, and object servers now support "seamless reloads" via "SIGUSR1". This is similar to the existing graceful restarts but keeps the server socket open the whole time, reducing service downtime. * New buckets created via the S3 API will now store multi-part upload data in the same storage policy as other data rather than the cluster's default storage policy. * Device region and zone can now be changed via "swift-ring- builder". Note that this may cause a lot of data movement on the next rebalance as the builder tries to reach full dispersion. * Added support for Python 3.8. Deprecation Notes ***************** * Per-service "auto_create_account_prefix" settings are now deprecated and may be ignored in a future release; if you need to use this, please set it in the "[swift-constraints]" section of "/etc/swift/swift.conf". Bug Fixes ********* * The container sharder can now handle containers with special characters in their names. * Internal client no longer logs object DELETEs as status 499. * Objects with an "X-Delete-At" value in the far future no longer cause backend server errors. * The bulk extract middleware once again allows clients to specify metadata (including expiration timestamps) for all objects in the archive. * Container sync now synchronizes static symlinks in a way similar to static large objects. * "swift_source" is set for more sub-requests in the proxy-server. See the documentation. * Errors encountered while validating static symlink targets no longer cause "BadResponseLength" errors in the proxy-server. * On Python 3, the KMS keymaster now works with secrets stored in Barbican with a "text/plain" payload-content-type. * On Python 3, the formpost middleware now works with unicode file names. * On Python 3, certain S3 API headers are now lower case as they would be coming from AWS. * Several utility scripts now work better on Python 3: * "swift-account-audit" * "swift-dispersion-populate" * "swift-drive-recon" * "swift-recon" Changes in swift 2.23.0..2.24.0 ------------------------------- 5cd854b78 Authors/changelog for swift 2.24.0 e3ee6ce31 py38: cgi lost some names 6097660f0 s3api: Implement object versioning API ebaf154fc Add a note to the part power increase documentation bd063b678 s3api: use native strings in s3api.auth_details access_key a84e358dc SSYNC: log body of errors in SSYNC subrequests 27fd97cef Middleware that allows a user to have quoted Etags 51c9b1308 Add unittest2 back to lower-constraints.txt 2759d5d51 New Object Versioning mode 26ff2eb1c container-sync: Sync static links similar to how we sync SLOs 2be4716bc More multi-char-delimiter func test coverage cd78ee983 Bump up ceph tests timeout dad8fb7c3 packaging: Build universal wheels 7bba3ad04 Store normalized x-delete-at on PUT/POST ff0753fe1 Allow bulk to fwd some headers at tar extraction 57ca3570e Allow Timestamp comparisons against out-of-range values 090146451 Clean up container-sync docs 9483630ae py3: Fix formpost unicode filename issues 5b26b749b Drop use of unittest2 614019079 Make FakeSwift query param order agnostic 47eb78897 Fix the duplicated words issue like "the the " b1178b4a9 Allow reconciler to handle reserved names 4601548da Deprecate per-service auto_create_account_prefix 0f13db30b Import directly from the inside of the ring module 3f8890701 sharding: Better-handle newlines in container names 286082222 Use less responses from handoffs 55ab08a5f account-server: Add test for leading delimiter e8b654f31 Have slo tell the object-server that it wants whole manifests 40edf07ab sharding: Let swift-manage-shard-ranges accept a relative path a06c0a475 sharder: quote() more Swift paths when logging d246bf20e sharding: Tolerate blank limits when listing 54312b0b7 py3: Clean up some needlessly-noisy tracebacks c94f625fe Forward-port stable-release changelog entries 226e2c120 Fix type on py3 1f7b97ec0 Add normalize_etag() helper function b65d8b10c Early-return on non-Swift get_info requests 6b33cf99f WSGI server workers must drop_privledges 05e4bb30b Update list of experimental upgrade tests 8c0fd3f13 py3: Make seamless reloads work da4e07164 Set swift_source more in s3api middleware 51b7b110a Set swift_source more in versioned_writes c35ab2e1d Set swift_source in account_quotas middleware f33c061ae s3api: Better handle 498/429 responses 698717d88 Allow internal clients to use reserved namespace a2aaf5985 Fix BadResponseLength error when creating symlink e11c58ef8 Turn off dns_compliant_bucket_names for ceph tests b5c9dc1c9 py3: Fix s3api header casing 249a302d0 Remove a bunch of known-failures that moved from boto to boto3 e5dad445b Switch py2 DSVM jobs to only run swift under py2 29019985e Start README.rst with a better title cfb3ae601 Update known-failures and config for up-rev'ed ceph/s3tests d6db7360e Imported Translations from Zanata da5b94914 Add py38 unit test job 3de330ead print deleted flag in account and container info cli 1107f2417 Seamlessly reload servers with SIGUSR1 2f1111a43 proxy: stop sending chunks to objects with a Queue 7f6b09dc9 Add delete_account in InternalClient 30d99f514 Get rid of _delete_db 7cb92810a Fix bug internal client's delete_object logs 499 c512574e1 Make our py2 func tests more explicitly py2 a505d02b3 Tweak Swift AIO docs, update CentOS and Fedora 281ffab2a pin lower-constraints to run against python 2.7 0c1b485ad exclude utf8 tests under py3 dd235b6ac use request_helpers import namespace for tests b75d59393 Refactor timestamp iter in db backend tests e7cd8df5e Add option for debug query logging 739c37041 Imported Translations from Zanata 2ac50079c docs: Fix then/than grammar 86952dd3e tests: Stop invoking python just to get the real source directory d270596b6 Consistently use io.BytesIO 8efaa3556 py38: Provide readinto() interface for RingReader 3bd565387 Use `==` to compare against the empty string, not `is` 2edcd0b7b py3: Fix swift-recon 4c66596e6 py3: fix swift-dispersion-populate 3e90ddb37 swift-account-audit: clean up some error formatting 2f4fe56ca py3: fix swift-account-audit e58840c57 swift-account-audit: work with encryption 85d3658d6 Fix kms_keymaster under Python 3 405a2b2a5 py3: Fix swift-drive-audit 78a4070c9 Fix misleading error msg if swift.conf unreadable d667affb6 Create segment container w/ same policy as primary 98f978109 Add commands to ring-builder to change region / zone bb5fa0ea2 tests: bust md5 of object not footers 3f7ffbc2c Start building py3 docker images cb76e00e9 Imported Translations from Zanata 527a57ffc Update master for stable/train 0217b12b6 PDF Documentation Build tox target f1b44b199 s3api: paginate listings when aborting MPUs 6853616ae ring: Track more properties of the ring 814a76689 Follow up punch_hole patch Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 134 +- AUTHORS | 3 + CHANGELOG | 209 ++ README.rst | 15 +- REVIEW_GUIDELINES.rst | 4 +- bin/swift-account-audit | 17 +- bin/swift-dispersion-populate | 6 +- bin/swift-drive-audit | 17 +- etc/account-server.conf-sample | 5 +- etc/container-server.conf-sample | 8 +- etc/drive-audit.conf-sample | 4 + etc/object-expirer.conf-sample | 1 - etc/object-server.conf-sample | 3 - etc/proxy-server.conf-sample | 37 +- etc/swift.conf-sample | 5 + .../notes/2_24_0_release-1ca244cc959922fc.yaml | 92 + releasenotes/source/index.rst | 2 + .../locale/ko_KR/LC_MESSAGES/releasenotes.po | 75 - releasenotes/source/train.rst | 6 + setup.cfg | 4 + swift/account/backend.py | 24 +- swift/account/reaper.py | 5 +- swift/account/server.py | 62 +- swift/account/utils.py | 9 +- swift/cli/info.py | 2 + swift/cli/manage_shard_ranges.py | 5 +- swift/cli/recon.py | 2 + swift/cli/ringbuilder.py | 167 +- swift/common/constraints.py | 23 +- swift/common/daemon.py | 3 +- swift/common/db.py | 42 +- swift/common/db_replicator.py | 7 +- swift/common/direct_client.py | 10 +- swift/common/header_key_dict.py | 37 +- swift/common/http.py | 1 + swift/common/internal_client.py | 58 +- swift/common/manager.py | 20 + swift/common/middleware/account_quotas.py | 3 +- swift/common/middleware/acl.py | 2 +- swift/common/middleware/bulk.py | 20 +- swift/common/middleware/container_sync.py | 37 +- swift/common/middleware/copy.py | 12 + swift/common/middleware/crypto/encrypter.py | 7 +- swift/common/middleware/crypto/keymaster.py | 4 + swift/common/middleware/crypto/kms_keymaster.py | 5 +- swift/common/middleware/dlo.py | 8 +- swift/common/middleware/etag_quoter.py | 127 + swift/common/middleware/formpost.py | 10 +- swift/common/middleware/gatekeeper.py | 7 + swift/common/middleware/listing_formats.py | 64 +- swift/common/middleware/proxy_logging.py | 2 +- swift/common/middleware/s3api/acl_handlers.py | 12 +- .../common/middleware/s3api/controllers/bucket.py | 314 +- .../middleware/s3api/controllers/multi_delete.py | 39 +- .../middleware/s3api/controllers/multi_upload.py | 50 +- swift/common/middleware/s3api/controllers/obj.py | 77 +- .../middleware/s3api/controllers/versioning.py | 42 +- swift/common/middleware/s3api/s3request.py | 101 +- swift/common/middleware/s3api/s3response.py | 57 +- swift/common/middleware/s3api/utils.py | 4 +- swift/common/middleware/slo.py | 17 +- swift/common/middleware/staticweb.py | 18 +- swift/common/middleware/symlink.py | 40 +- .../common/middleware/versioned_writes/__init__.py | 51 + .../legacy.py} | 29 +- .../versioned_writes/object_versioning.py | 1467 +++++++++ swift/common/middleware/x_profile/html_viewer.py | 8 +- swift/common/request_helpers.py | 104 +- swift/common/ring/builder.py | 48 +- swift/common/ring/ring.py | 134 +- swift/common/swob.py | 51 +- swift/common/utils.py | 99 +- swift/common/wsgi.py | 222 +- swift/container/backend.py | 43 +- swift/container/server.py | 84 +- swift/container/sharder.py | 132 +- swift/container/sync.py | 22 +- swift/locale/de/LC_MESSAGES/swift.po | 278 +- swift/locale/en_GB/LC_MESSAGES/swift.po | 92 +- swift/locale/es/LC_MESSAGES/swift.po | 77 +- swift/locale/fr/LC_MESSAGES/swift.po | 75 +- swift/locale/it/LC_MESSAGES/swift.po | 77 +- swift/locale/ja/LC_MESSAGES/swift.po | 77 +- swift/locale/ko_KR/LC_MESSAGES/swift.po | 178 +- swift/locale/pt_BR/LC_MESSAGES/swift.po | 72 +- swift/locale/ru/LC_MESSAGES/swift.po | 74 +- swift/locale/tr_TR/LC_MESSAGES/swift.po | 71 +- swift/locale/zh_CN/LC_MESSAGES/swift.po | 71 +- swift/locale/zh_TW/LC_MESSAGES/swift.po | 71 +- swift/obj/diskfile.py | 2 +- swift/obj/expirer.py | 16 +- swift/obj/reconstructor.py | 2 +- swift/obj/server.py | 79 +- swift/obj/ssync_receiver.py | 4 +- swift/obj/updater.py | 1 + swift/proxy/controllers/base.py | 55 +- swift/proxy/controllers/container.py | 2 +- swift/proxy/controllers/obj.py | 427 ++- swift/proxy/server.py | 23 +- test/functional/__init__.py | 16 +- test/functional/s3api/__init__.py | 4 +- test/functional/s3api/s3_test_client.py | 12 +- test/functional/s3api/test_acl.py | 4 +- test/functional/s3api/test_bucket.py | 4 +- test/functional/s3api/test_multi_delete.py | 4 +- test/functional/s3api/test_multi_upload.py | 141 +- test/functional/s3api/test_object.py | 28 +- test/functional/s3api/test_service.py | 4 +- test/functional/s3api/test_versioning.py | 166 + test/functional/swift_test_client.py | 26 +- test/functional/test_account.py | 8 +- test/functional/test_container.py | 8 +- test/functional/test_domain_remap.py | 2 +- test/functional/test_object.py | 82 +- test/functional/test_object_versioning.py | 2669 ++++++++++++++++ test/functional/test_staticweb.py | 2 +- test/functional/test_symlink.py | 4 +- test/functional/test_versioned_writes.py | 4 +- test/probe/brain.py | 162 +- test/probe/common.py | 41 +- test/probe/test_account_reaper.py | 23 +- test/probe/test_container_failures.py | 3 + test/probe/test_container_merge_policy_index.py | 132 +- test/probe/test_container_sync.py | 326 ++ test/probe/test_object_versioning.py | 233 ++ test/probe/test_reconstructor_rebuild.py | 8 +- test/probe/test_replication_servers_working.py | 33 +- test/probe/test_reserved_name.py | 131 + test/probe/test_sharder.py | 359 ++- test/probe/test_signals.py | 344 ++- test/s3api/__init__.py | 89 +- test/s3api/test_service.py | 5 +- test/s3api/test_versioning.py | 758 +++++ test/unit/__init__.py | 13 +- test/unit/account/test_auditor.py | 2 +- test/unit/account/test_backend.py | 125 +- test/unit/account/test_reaper.py | 16 +- test/unit/account/test_server.py | 522 +++- test/unit/account/test_utils.py | 84 +- test/unit/cli/test_info.py | 59 +- test/unit/cli/test_ringbuilder.py | 348 ++- .../common/middleware/crypto/test_kms_keymaster.py | 11 +- test/unit/common/middleware/helpers.py | 43 +- test/unit/common/middleware/s3api/__init__.py | 18 +- test/unit/common/middleware/s3api/helpers.py | 100 +- test/unit/common/middleware/s3api/test_acl.py | 2 +- test/unit/common/middleware/s3api/test_bucket.py | 546 +++- .../common/middleware/s3api/test_multi_delete.py | 138 +- .../common/middleware/s3api/test_multi_upload.py | 85 +- test/unit/common/middleware/s3api/test_obj.py | 385 ++- test/unit/common/middleware/s3api/test_s3api.py | 27 + .../unit/common/middleware/s3api/test_s3request.py | 4 +- .../common/middleware/s3api/test_s3response.py | 18 + .../common/middleware/s3api/test_versioning.py | 172 +- test/unit/common/middleware/test_bulk.py | 41 +- test/unit/common/middleware/test_container_sync.py | 3 + test/unit/common/middleware/test_etag_quoter.py | 215 ++ test/unit/common/middleware/test_formpost.py | 126 +- test/unit/common/middleware/test_gatekeeper.py | 33 + .../unit/common/middleware/test_listing_formats.py | 264 +- .../common/middleware/test_object_versioning.py | 3214 ++++++++++++++++++++ test/unit/common/middleware/test_proxy_logging.py | 6 +- test/unit/common/middleware/test_slo.py | 100 +- test/unit/common/middleware/test_symlink.py | 34 + .../common/middleware/test_versioned_writes.py | 102 +- test/unit/common/middleware/test_xprofile.py | 2 +- test/unit/common/ring/test_ring.py | 17 +- test/unit/common/test_constraints.py | 24 +- test/unit/common/test_db.py | 131 +- test/unit/common/test_direct_client.py | 122 +- test/unit/common/test_internal_client.py | 78 +- test/unit/common/test_request_helpers.py | 279 +- test/unit/common/test_swob.py | 454 +-- test/unit/common/test_utils.py | 141 +- test/unit/common/test_wsgi.py | 107 +- test/unit/container/test_backend.py | 609 ++-- test/unit/container/test_reconciler.py | 17 +- test/unit/container/test_replicator.py | 87 +- test/unit/container/test_server.py | 335 +- test/unit/container/test_sharder.py | 473 ++- test/unit/container/test_updater.py | 91 +- test/unit/helpers.py | 3 +- test/unit/obj/test_expirer.py | 14 + test/unit/obj/test_server.py | 235 +- test/unit/obj/test_ssync_receiver.py | 43 +- test/unit/obj/test_ssync_sender.py | 15 +- test/unit/obj/test_updater.py | 18 +- test/unit/proxy/controllers/test_base.py | 63 + test/unit/proxy/controllers/test_container.py | 9 +- test/unit/proxy/controllers/test_obj.py | 29 +- test/unit/proxy/test_server.py | 136 +- tox.ini | 27 +- 224 files changed, 20021 insertions(+), 3860 deletions(-)