[release-announce] swift 2.31.0 (antelope)
no-reply at openstack.org
no-reply at openstack.org
Tue Jan 31 15:44:34 UTC 2023
We are psyched to announce the release of:
swift 2.31.0: OpenStack Object Storage
This release is part of the antelope release series.
The source is available from:
https://opendev.org/openstack/swift
Download the package from:
https://tarballs.openstack.org/swift/
Please report issues through:
https://bugs.launchpad.net/swift/+bugs
For more details, please see below.
2.31.0
^^^^^^
New Features
************
* Added support for Python 3.10.
* Added an optional "backend_ratelimit" middleware for backend
servers. See the backend server sample configuration files for more
information.
* Sharding improvements
* Added a "merge" subcommand to "swift-manage-shard-ranges" to
merge arbitrary shard ranges into a container DB. Minimal safety
checks are performed; it should only be used for emergency shard
range manipulation by expert users.
* Warnings are now emitted when sharding appears to have become
stuck. Use the new "container_sharding_timeout" option to
configure the "stuck" threshold; the default is 48 hours.
* Metrics improvements
* Added timing stats for memcached operations.
* Renamed and improved the granularity of shard range cache and
backend stats. Metrics dashboards may need to be updated.
* Emit stats when backend nodes are error-limited.
* Added the ability to configure a chance to skip checking memcache
when querying account and container information. This allows some
fraction of traffic to go to disk and refresh memcache before the
key ages out. Recommended values for the new
"account_existence_skip_cache_pct" and
"container_existence_skip_cache_pct" options are in the range of 0.0
to 0.01.
* Absolute-form request targets are now accepted. This enables
access for certain clients and SDKs (including some older versions
of rclone that were using an old version of aws-sdk-go).
Upgrade Notes
*************
* Static large object segments may now be deleted asynchronously by
default. Operators may return to the old behavior by disabling the
"allow_async_delete" option in the "[filter:slo]" section in their
proxy-server.conf.
Security Issues
***************
* Fixed a security issue in how "s3api" handles XML parsing that
allowed authenticated S3 clients to read arbitrary files from proxy
servers. Refer to CVE-2022-47950 for more information.
Bug Fixes
*********
* S3 API improvements
* Fixed a server error when handling malformed
CompleteMultipartUpload requests.
* Improved error reporting when attempting to set invalid
"X-Delete- At" or "X-Delete-After" values via the S3 API.
* Sharding improvements
* Sync more shard ranges from the root database to the shards.
This helps ensure shard range repairs effected at the root make
their way to shards that would otherwise be stuck trying to
further divide into sub-shards.
* Improved performance of "delimiter" listings for sharded
containers.
* Added more safety checks to the "repair" subcommand of "swift-
manage-shard-ranges".
* Better handle "EOFError" and "KeyboardInterrupt" when prompting
for input in "swift-manage-shard-ranges".
* Stop warning about transient overlaps when auditing shard
ranges.
* Fixed a path-rewriting bug introduced in Python 3.7.14, 3.8.14,
3.9.14, and 3.10.6 that could cause some "domain_remap" requests to
be routed to the wrong object.
* Fixed a server error when attempting to access data in a deleted
container that had an erasure-coded storage policy.
* Improved error messages to clients that encounter errors using the
"formpost" middleware.
* Removed some inappropriate error-suppression when locking account
and container databases.
* Improved server start-up time when using multiple workers.
* Removed some unnecessary locking when logging.
* Added some basic object-metadata validation; invalid diskfiles
will be quarantined via the auditor or reconstructor.
* Enhanced logging when error-limiting a backend node.
* Various other minor bug fixes and improvements.
Changes in swift 2.30.0..2.31.0
-------------------------------
90f9a479b Authors/ChangeLog for 2.31.0
5de745c2b Add Kota's private email address to mailmap
941b9082d Proxy: move '_get_update_shard' from base class to child class.
2f32d0770 swift_proxy: add memcache skip success/error stats for shard range.
30ffafb3e Clean up some cruft
3550e00dd tests: Ensure XXE injection tests have config loaded
51a944850 Skip coverage reports when running pytest directly
908595ee0 Don't run reno as part of building an sdist
cc1c649de Clean up project URLs for PyPI
dd9b687f7 Fix docs build
b8467e190 s3api: Prevent XXE injections
69b18e3c5 tests: Remove references to soft_lock
930744f36 Declare py310 support
fb477d17a CI: Remove nodeset pin for DSVM jobs
ee12a11e7 Add attrs to lower-constraints
b4124e0cd Memcached: add timing stats to set/get and other public functions
f77172e2a CI: pin tox at the project level
ec9504733 Sharder: add a new probe test for the case of slow parent sharding.
f6ac7d449 Tolerate absolute-form request targets
2d7c1dc6d CI: Fix our usage of tox
20b48a690 Clean up a bunch of deprecation warnings
2d0fea6a5 CI: Add py310 unit test job
ef155bd74 Switch to pytest
884f5538f Inline parse_request from cpython
597887ded Extract SwiftHttpProtocol to its own module
4cba97d7b Malformed CompleteMultipartUpload request should 400
d363236a2 s3api errors for unsupported headers x-delete-at, x-delete-after
001d931e6 sharder: update own_sr stats explicitly
07b909c26 s-m-s-r: handle EOFError and KeyboardInterrupt
861d13c51 DB locks shouldn't squelch errors
662ef2808 trivial: fix flakey account/test_auditor.py assertion
a1939cba0 proxy: add test for ContainerController._GET_using_cache
bd5d09903 proxy: refactor ContainerController._GET_using_cache
7a0118fca proxy-server: include suppression time in error limit log
230b467af sharding: avoid transient overlaps while auditing.
5a9f5f521 Temporary pin the swift-dsvm-functional nodeset to Focal
983622d51 account_auditor: fix warning string
26c86c0e4 db_auditor: remove logging translation
2bcf3d1a8 sharder: merge shard shard_ranges from root while sharding
12df65e37 Imported Translations from Zanata
38124221d Proxy: add metrics related to error limiter.
d1b2bbdcf Mark rolling-upgrade job non-voting
623024848 proxy: extract response error handling to single method
162847d15 tests: Tolerate NoSuchBucket errors when cleaning up
f08b8e0af wsgi: Start workers in parallel, rather than serially
052abed75 Move base CI job to jammy
c4ad66b49 Sharder: add more logs during container cleaving.
3571cdf2f CI: Install pip from pinned upstream for py2 probe tests
a8516e151 Refactor memcache config and MemcacheRing loading
51730f127 proxy: refactor error limiter to a class
a7af17394 Clean up some replication error messages
326e68d70 Fix a formpost reponse bug
79e8f56ac Imported Translations from Zanata
cc033154a py2constraints: pin PasteDeploy version
4ed2b89cb Sharder: warn when sharding appears to have stalled.
e09d4bcf4 Stop locking in our SysLogHandler
07b5f139a playbooks: replace ansible_ssh_user with ansible_user
a46f2324a sharder: always merge child shard ranges fetched from root
e838d8a94 swift-drive-audit: reload systemd after editing fstab
f15b92084 sharder: always get ranges from root while shrinking
429702e30 sharder: refactor _audit_shard_container
7b345e754 Update master for stable/zed
a425f06ee Imported Translations from Zanata
6bcb50985 sharder: trivial tidy up of unit tests
94fb7a0e7 Set irrelevant-files for openstacksdk-functional-devstack job
0860db1f6 Fix test.probe.brain CLI
9bdb77a38 Fixes Storage-Policy AttributeError in proxy
a53270a15 swift-manage-shard-ranges repair: check for parent-child overlaps.
032d7aa05 ShardName: add test and clarify docstrings
73bce6331 ShardRange: add is_child_of method
745a7f04f Add validation method for metadata in ECDiskfile
3e6650f99 Fix misuse of assertTrue
d2edf6646 swift-manage-shard-ranges: add 'merge' subcommand
5c6407bf5 proxy: Add a chance to skip memcache for get_*_info calls
c4ff49c32 CI: Add rolling upgrade job coming from stable/yoga
a9177a4b9 Add note about rsync_bwlimit suffixes
da52e5675 Migrate CentOS Stream 8 FIPS job to CentOS Stream 9
d0feee190 Add MPU to s3api tests
17664cfb9 Use daemon property instead of setDaemon method
61624ab83 swift-manage-shard-ranges repair: ignore recent overlaps
38866a396 Optimize ShardRanges a little
85d063b20 ring: Have RingBuilder.debug return to old state
25463b253 sharding: Skip shards that can't include any new subdir entries
ccaf49a00 Add backend rate limiting middleware
fa1058b6e slo: Default allow_async_delete to true
Diffstat (except docs and test files)
-------------------------------------
.mailmap | 4 +-
.zuul.yaml | 155 +-
AUTHORS | 6 +-
CHANGELOG | 95 ++
CONTRIBUTING.rst | 4 +-
bin/swift-drive-audit | 4 +
bindep.txt | 2 +-
etc/account-server.conf-sample | 14 +-
etc/container-server.conf-sample | 20 +-
etc/object-server.conf-sample | 17 +-
etc/proxy-server.conf-sample | 4 +-
lower-constraints.txt | 8 +-
py2-constraints.txt | 7 +-
py36-constraints.txt | 3 -
.../notes/2_31_0_release-77e6b20dfba3b32c.yaml | 118 ++
releasenotes/source/index.rst | 2 +
.../locale/en_GB/LC_MESSAGES/releasenotes.po | 164 ++-
releasenotes/source/zed.rst | 6 +
setup.cfg | 23 +-
swift/__init__.py | 10 +-
swift/account/auditor.py | 7 +-
swift/account/reaper.py | 17 +-
swift/cli/dispersion_report.py | 7 +-
swift/cli/manage_shard_ranges.py | 222 ++-
swift/common/db.py | 19 +-
swift/common/db_auditor.py | 19 +-
swift/common/error_limiter.py | 93 ++
swift/common/http_protocol.py | 326 +++++
swift/common/memcached.py | 118 +-
swift/common/middleware/backend_ratelimit.py | 86 ++
swift/common/middleware/formpost.py | 26 +-
swift/common/middleware/memcache.py | 96 +-
.../middleware/s3api/controllers/multi_upload.py | 20 +-
swift/common/middleware/s3api/etree.py | 2 +-
swift/common/middleware/s3api/exception.py | 4 -
swift/common/middleware/s3api/s3api.py | 3 +
swift/common/middleware/s3api/s3request.py | 18 +-
swift/common/middleware/slo.py | 4 +-
.../common/middleware/versioned_writes/__init__.py | 3 +
swift/common/ring/builder.py | 4 +-
swift/common/swob.py | 3 +
swift/common/utils.py | 354 ++++-
swift/common/wsgi.py | 266 +---
swift/container/backend.py | 108 +-
swift/container/sharder.py | 510 +++++--
swift/container/updater.py | 12 +-
swift/obj/auditor.py | 4 +
swift/obj/diskfile.py | 14 +
swift/obj/reconstructor.py | 19 +-
swift/obj/replicator.py | 19 +-
swift/obj/ssync_sender.py | 18 +-
swift/obj/updater.py | 19 +-
swift/proxy/controllers/base.py | 162 +--
swift/proxy/controllers/container.py | 297 ++--
swift/proxy/controllers/obj.py | 113 +-
swift/proxy/server.py | 124 +-
test-requirements.txt | 7 +-
test/__init__.py | 55 +-
test/functional/__init__.py | 3 +-
test/functional/s3api/test_object.py | 51 +-
test/functional/s3api/test_presigned.py | 30 +
test/functional/s3api/test_xxe_injection.py | 231 +++
test/probe/brain.py | 5 +-
test/probe/common.py | 22 +
test/probe/test_sharder.py | 335 ++++-
test/s3api/__init__.py | 4 +
test/s3api/test_mpu.py | 140 ++
test/s3api/test_request_target_style.py | 56 +
test/unit/__init__.py | 4 -
test/unit/account/test_auditor.py | 6 +-
test/unit/cli/test_manage_shard_ranges.py | 625 +++++++-
test/unit/common/middleware/helpers.py | 4 +-
test/unit/common/middleware/s3api/__init__.py | 5 +
.../common/middleware/s3api/test_multi_delete.py | 40 +
test/unit/common/middleware/s3api/test_s3token.py | 12 +-
.../common/middleware/test_backend_ratelimit.py | 170 +++
test/unit/common/middleware/test_formpost.py | 32 +-
test/unit/common/middleware/test_memcache.py | 214 +--
test/unit/common/middleware/test_slo.py | 10 +-
test/unit/common/test_db.py | 23 +-
test/unit/common/test_db_replicator.py | 93 +-
test/unit/common/test_error_limiter.py | 99 ++
test/unit/common/test_http_protocol.py | 454 ++++++
test/unit/common/test_manager.py | 6 +-
test/unit/common/test_memcached.py | 276 ++++
test/unit/common/test_storage_policy.py | 5 +-
test/unit/common/test_utils.py | 537 ++++++-
test/unit/common/test_wsgi.py | 335 +----
test/unit/container/test_backend.py | 125 +-
test/unit/container/test_server.py | 15 +-
test/unit/container/test_sharder.py | 1526 +++++++++++++++++---
test/unit/helpers.py | 8 +-
test/unit/obj/common.py | 1 +
test/unit/obj/test_auditor.py | 31 +-
test/unit/obj/test_diskfile.py | 36 +-
test/unit/obj/test_replicator.py | 24 +-
test/unit/obj/test_server.py | 10 +-
test/unit/obj/test_ssync.py | 1 +
test/unit/proxy/controllers/test_base.py | 56 +-
test/unit/proxy/controllers/test_container.py | 583 +++++++-
test/unit/proxy/controllers/test_obj.py | 26 +-
test/unit/proxy/test_server.py | 619 ++++++--
tools/playbooks/common/install_dependencies.yaml | 24 +-
tools/playbooks/multinode_setup/common_config.yaml | 4 +-
tools/playbooks/multinode_setup/make_rings.yaml | 8 +-
tools/playbooks/multinode_setup/run.yaml | 2 +-
.../templates/make_multinode_rings.j2 | 2 +-
.../saio_single_node_setup/setup_saio.yaml | 14 +-
tools/test-setup.sh | 12 +
tox.ini | 43 +-
117 files changed, 8657 insertions(+), 2648 deletions(-)
Requirements updates
--------------------
diff --git a/test-requirements.txt b/test-requirements.txt
index 7eb48af72..488080ae5 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -7,4 +7,3 @@ hacking>=2.0,<2.1.0 # Apache-2.0
-coverage>=3.6 # Apache-2.0
-nose>=1.3.7 # LGPL
-nosexcover>=1.0.10 # BSD
-nosehtmloutput>=0.0.3 # Apache-2.0
+coverage>=5.0.4 # Apache-2.0
+pytest>=4.6.11 # MIT
+pytest-cov>=2.12.1 # MIT
More information about the Release-announce
mailing list