We are jazzed to announce the release of:
swift 2.23.0: OpenStack Object Storage
This release is part of the train release series.
The source is available from:
https://opendev.org/openstack/swift
Download the package from:
https://tarballs.openstack.org/swift/
Please report issues through:
https://bugs.launchpad.net/swift/+bugs
For more details, please see below.
2.23.0 ^^^^^^
New Features ************
* Python 3.6 and 3.7 are now fully supported. If you've been testing Swift on Python 3, upgrade at your earliest convenience.
* Added "static symlinks", which perform some validation as they follow redirects and include more information about their target in container listings. For more information, see the symlink middleware section of the documentation.
* Multi-character strings may now be used as delimiters in account and container listings.
Upgrade Notes *************
* **Dependency update**: "eventlet" must be at least 0.25.0. This also dragged forward minimum-supported versions of "dnspython" (1.15.0), "greenlet" (0.3.2), and "six" (1.10.0).
Bug Fixes *********
* Python 3 fixes:
* Removed a request-smuggling vector when running a mixed py2/py3 cluster.
* Allow "fallocate_reserve" to be specified as a percentage.
* Fixed listings for sharded containers.
* Fixed non-ASCII account metadata handling.
* Fixed "rsync" output parsing.
* Fixed some title-casing of headers.
If you've been testing Swift on Python 3, upgrade at your earliest convenience.
* Sharding improvements
* Container metadata related to sharding are now removed when no longer needed.
* Empty container databases (such as might be created on handoffs) now shard much more quickly.
* The "proxy-server" now ignores 404 responses from handoffs that have no data when deciding on the correct response for object requests, similar to what it already does for account and container requests.
* Static Large Object sizes in listings for versioned containers are now more accurate.
* When refetching Static Large Object manifests, non-manifest responses are now handled better.
* S3 API now translates "503 Service Unavailable" responses to a more S3-like response instead of raising an error.
* Improved proxy-to-backend requests to be more RFC-compliant.
* Various other minor bug fixes and improvements.
Changes in swift 2.22.0..2.23.0 -------------------------------
8a4becb12 Authors/changelog for 2.23.0 bf9346d88 Fix some request-smuggling vectors on py3 be4172188 Add experimental job to test upgrades from stein 9a33365f0 py3: Allow percentages in configs ad7f7da32 py3: decode stdout from backgrounded servers e9cd9f74a sharder: Keep cleaving on empty shard ranges f56071e57 Make sharding methods with only one job 50255de0e func tests: Add more UTF8 tests for versioning 6271d88f9 Add func test for changing versionining modes 9495bc000 sharding: Update probe test to verify CleavingContext cleanup 370ac4cd7 Sharding: Use the metadata timestamp as last_modified 291873e78 proxy: Don't trust Content-Length for chunked transfers 81a41da54 Sharding: Clean up old CleaveConext's during audit 20fc16e8d Close leaking opened requests 9698b1bb9 Skip test when object versioning is not enabled 75c9c636f [train][goal] Run 'tempest-ipv6-only' job in gate b4288b4aa versioned_writes: checks for SLO object before copy db8b0b6bc Make ceph tests more portable 3960df983 Remove unneeded Zuul branch matcher 49f62f6ab bufferedhttp: ensure query params are properly quoted 1ded0d6c8 Allow arbitrary UTF-8 strings as delimiters in listings 4cafc3d65 doc: Fix the swift middleware doc needs more info to set s3 api 1d7e1558b py3: (mostly) port probe tests c71bb2506 diskfile: Add some argument validation e6e31410e Find .d pid files with swift-orphans 3e4efb7aa py3: Update Getting Started docs 4d83b9b95 tests/py3: Improve header casing a32fb30c1 Use SOURCE_DATE_EPOCH in docs to make build reproducible 254537205 py3: Switch swift-dsvm-functional-py3 to run tests under py3 ff5ea003b ec: log durability of frags that fail to reconstruct 94366fd00 Add Dockerfile to build a py3 swift docker image e92191306 Rename symlink method 82e427a8b Fix symlink docstring 74db36706 py3: Finish porting func tests a21c0be70 dlo: Respond 200 on multi-range GETs 7f42309a3 py3: Cover account/container func tests 3e7752d8c Update docker image to latest Alpine 3.10.1 1abc9c4f9 Allow "static symlinks" 0c16fd953 Fixing broken links 3750285bc py3: fix up listings on sharded containers 27e7e80e9 py3: fix up swift-orphans 9e9c36336 fix ec overview doc formatting 47c705825 Add params to get_*_metadata internal client methods cb3081191 Allow bulk delete of big SLO manifests 227fe3025 py3: port test/functional/test_versioned_writes.py f05119c16 py3: mostly port s3 func tests c90365f98 Allow Connection: value to be case-insensitive 996aa4547 Consolidate Container-Update-Override headers 3ee6de408 slo: Add X-Manifest-Etag to responses e095bf3da Bring our IPv6 setup more in line with devstack's 7c74dbd25 py3: port dlo func tests 7bb7da6d2 s3api: Translate 503 to S3-style 503s 2c727f65f py3: Add swift-tox-func-encryption-py37 gate job a10d936ee Use `is` to compare against sentinel object a48dd1950 Allow non-default domain to be used in func tests 2e4c8a924 py3: port RBAC func tests ab1924153 Increase node_timeout for real sever unittests 25aeb0ca4 Make GreenAsyncPile not hang 3189410f9 Ignore 404s from handoffs for objects when calculating quorum 7678e94a2 Negative test for non-empty chunked put symlink c0035ed82 Update the bandit.yaml available tests list c19d09f60 Give ECAppIter greenthreads a chance to wrap up a1455b3a3 symlink: Allow symlinks to be created via chunk-encoded PUTs 8635e8cc8 Make backport releases conform to existing CHANGELOG formatting 02cfb6c02 Add integrated gate tempest and grenade job 097a62cad Fix broken API-ref link 71f83963c py3: fix non-ascii metadata handling in account-server 60c27d3ae py3: Fix title-casing in HeaderKeyDict 73ec8f15d Update api-ref location a48104c54 py3: port SLO func tests ef5a37c2b slo: Better handle non-manifest responses when refetching manifest 12a7b4206 Fix test_parse_get_node_args 2926fc96a py3: Bring functional/test_object.py under test; add func-ec-py37 job a65350e88 Make rolling-upgrade job voting again 424a2603d Fix up errno checking ebf561026 py3: fix object-replicator rsync output parsing bc5f4c061 sharding: better handle get_shard_ranges failures 3532df0fd Add Python 3 Train unit tests 0fec28ab1 Fix invalid assert states 03512e001 Update the constraints url 5270da86e Add python3 to setup.cfg 0ebfeddf6 Bump up our minimum eventlet version 4aa71aa25 We don't have to keep the retrieved token anymore fbf6d8479 Make py36 job voting eed76d8be Fix a potential bug
Diffstat (except docs and test files) -------------------------------------
.gitignore | 1 + .zuul.yaml | 124 +++- AUTHORS | 5 + CHANGELOG | 78 ++- Dockerfile | 9 +- Dockerfile-py3 | 76 +++ README.rst | 2 +- api-ref/source/conf.py | 29 +- bandit.yaml | 11 +- bin/swift-orphans | 30 +- .../15_install_python_and_pip_src.sh | 57 -- lower-constraints.txt | 8 +- .../notes/2_23_0_release-2a2d11c1934f0b61.yaml | 74 +++ requirements.txt | 8 +- setup.cfg | 3 + swift/account/backend.py | 10 +- swift/account/server.py | 39 +- swift/account/utils.py | 6 +- swift/common/bufferedhttp.py | 11 +- swift/common/constraints.py | 4 - swift/common/header_key_dict.py | 32 +- swift/common/internal_client.py | 20 +- swift/common/manager.py | 3 + swift/common/middleware/copy.py | 4 +- swift/common/middleware/crypto/decrypter.py | 5 +- swift/common/middleware/crypto/encrypter.py | 12 +- swift/common/middleware/dlo.py | 2 + .../middleware/s3api/controllers/multi_upload.py | 5 +- swift/common/middleware/s3api/controllers/obj.py | 2 +- swift/common/middleware/s3api/s3api.py | 83 ++- swift/common/middleware/s3api/s3request.py | 26 +- swift/common/middleware/s3api/s3token.py | 22 +- swift/common/middleware/s3api/subresource.py | 2 + swift/common/middleware/slo.py | 79 ++- swift/common/middleware/symlink.py | 272 +++++++-- swift/common/middleware/versioned_writes.py | 9 +- swift/common/request_helpers.py | 13 + swift/common/utils.py | 24 +- swift/common/wsgi.py | 39 +- swift/container/backend.py | 44 +- swift/container/replicator.py | 2 +- swift/container/server.py | 14 +- swift/container/sharder.py | 107 +++- swift/obj/diskfile.py | 6 +- swift/obj/reconstructor.py | 30 +- swift/obj/replicator.py | 2 +- swift/obj/server.py | 4 +- swift/obj/ssync_receiver.py | 12 +- swift/proxy/controllers/base.py | 8 +- swift/proxy/controllers/container.py | 26 +- swift/proxy/controllers/obj.py | 68 ++- swift/proxy/server.py | 7 + test/functional/__init__.py | 22 +- test/functional/s3api/s3_test_client.py | 8 +- test/functional/s3api/test_acl.py | 3 +- test/functional/s3api/test_bucket.py | 18 + test/functional/s3api/test_multi_upload.py | 61 +- test/functional/s3api/test_object.py | 52 +- test/functional/s3api/test_presigned.py | 8 +- test/functional/s3api/test_service.py | 4 +- test/functional/s3api/utils.py | 3 +- test/functional/swift_test_client.py | 8 +- test/functional/test_access_control.py | 2 +- test/functional/test_account.py | 23 +- test/functional/test_container.py | 47 +- test/functional/test_dlo.py | 87 +-- test/functional/test_object.py | 100 +-- test/functional/test_slo.py | 369 ++++++----- test/functional/test_symlink.py | 673 +++++++++++++++++++-- test/functional/test_versioned_writes.py | 337 ++++++++--- test/probe/common.py | 18 +- .../probe/test_account_get_fake_responses_match.py | 6 +- test/probe/test_container_failures.py | 2 +- test/probe/test_container_merge_policy_index.py | 59 +- test/probe/test_container_sync.py | 28 +- test/probe/test_empty_device_handoff.py | 4 +- test/probe/test_object_async_update.py | 6 +- test/probe/test_object_conditional_requests.py | 12 +- test/probe/test_object_failures.py | 73 ++- test/probe/test_object_handoff.py | 123 +++- test/probe/test_object_metadata_replication.py | 10 +- test/probe/test_object_partpower_increase.py | 4 +- test/probe/test_reconstructor_rebuild.py | 32 +- test/probe/test_reconstructor_revert.py | 2 +- test/probe/test_sharder.py | 131 ++-- test/probe/test_signals.py | 14 +- test/unit/__init__.py | 3 +- test/unit/account/test_server.py | 169 +++++- test/unit/common/middleware/s3api/test_bucket.py | 19 +- .../common/middleware/s3api/test_multi_delete.py | 22 + test/unit/common/middleware/s3api/test_obj.py | 12 +- test/unit/common/middleware/s3api/test_s3api.py | 80 ++- .../unit/common/middleware/s3api/test_s3request.py | 9 +- test/unit/common/middleware/s3api/test_s3token.py | 15 +- test/unit/common/middleware/test_dlo.py | 6 +- test/unit/common/middleware/test_slo.py | 221 ++++++- test/unit/common/middleware/test_symlink.py | 433 ++++++++++++- .../common/middleware/test_versioned_writes.py | 34 +- test/unit/common/test_bufferedhttp.py | 5 +- test/unit/common/test_header_key_dict.py | 15 + test/unit/common/test_internal_client.py | 4 +- test/unit/common/test_manager.py | 14 +- test/unit/common/test_utils.py | 59 ++ test/unit/common/test_wsgi.py | 8 +- test/unit/container/test_server.py | 138 ++++- test/unit/container/test_sharder.py | 195 +++++- test/unit/container/test_sync.py | 12 +- test/unit/helpers.py | 35 +- test/unit/obj/test_reconstructor.py | 22 +- test/unit/obj/test_replicator.py | 6 +- test/unit/obj/test_updater.py | 15 +- test/unit/proxy/controllers/test_container.py | 199 ++++-- test/unit/proxy/controllers/test_obj.py | 169 +++++- test/unit/proxy/test_server.py | 258 +++++++- tools/playbooks/multinode_setup/make_rings.yaml | 8 +- .../saio_single_node_setup/add_s3api.yaml | 23 + tox.ini | 32 +- 132 files changed, 4961 insertions(+), 1363 deletions(-)
Requirements updates --------------------
diff --git a/requirements.txt b/requirements.txt index 609e540e6..d4c994af2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,3 +5,3 @@ -dnspython>=1.14.0;python_version=='2.7' # http://www.dnspython.org/LICENSE -eventlet>=0.17.4,!=0.23.0 # MIT -greenlet>=0.3.1 +dnspython>=1.15.0;python_version=='2.7' # http://www.dnspython.org/LICENSE +eventlet>=0.25.0 # MIT +greenlet>=0.3.2 @@ -12 +12 @@ requests>=2.14.2 # Apache-2.0 -six>=1.9.0 +six>=1.10.0