[release-announce] swift 2.29.0 (yoga)
no-reply at openstack.org
no-reply at openstack.org
Mon Feb 14 17:43:48 UTC 2022
We are delighted to announce the release of:
swift 2.29.0: OpenStack Object Storage
This release is part of the yoga release series.
The source is available from:
https://opendev.org/openstack/swift
Download the package from:
https://tarballs.openstack.org/swift/
Please report issues through:
https://bugs.launchpad.net/swift/+bugs
For more details, please see below.
2.29.0
^^^^^^
New Features
************
* S3 API improvements
* CORS preflights are now allowed for pre-signed URLs.
* The "storage_domain" option now accepts a comma-separated list
of storage domains. This allows multiple storage domains to
configured for use with virtual-host style addressing.
* Reduced the overhead of retrieving bucket and object ACLs.
* Replication, reconstruction, and diskfile improvements
* The reconstructor now uses the replication network to fetch
fragments for reconstruction.
* Added the ability to limit how many objects per handoff
partition will be reverted in a reconstructor cycle using the new
"max_objects_per_revert" option. This may be useful to reduce
ssync timeouts and lock contention, ensuring that progress is made
during rebalances.
* Object updater improvements
* Added the ability to ratelimit updates (approximately) per-
container using the new "max_objects_per_container_per_second"
option. This may be used to limit requests to already-overloaded
containers while still making progress on updates to other
containers.
* Added timing stats by response code.
* Updates are now sent over the replication network.
* Memcache improvements
* Added the ability to configure a chance to skip checking
memcache when querying shard ranges. This allows some fraction of
traffic to go to disk and refresh memcache before the key ages
out. Recommended values for the new
"container_updating_shard_ranges_skip_cache_pct" and
"container_listing_shard_ranges_skip_cache_pct" options are in the
range of 0.0 to 0.1.
* Added stats for shard range cache hits, misses, and skips.
* Added object-reconstructor stats to recon.
* Added a new "swift.common.registry" module. This includes helper
functions "register_sensitive_header" and "register_sensitive_param"
which third party middleware authors may use to flag headers and
query parameters for redaction when logging. For more information,
see the documentation.
* Added the ability to configure project-scope read-only roles for
keystoneauth using the new "project_reader_roles" option.
* The "cname_lookup" middleware now works with dnspython 2.0 and
later.
* The internal clients used by the container-reconciler, container-
sharder, container-sync, and object-expirer daemons now use a more-
descriptive "<daemon>-ic" log name, rather than "swift". If you
previously configured the "log_name" option in "internal-
client.conf", you must now use the "set log_name = <value>" syntax
to configure it, even if no value is set in the "[DEFAULT]" section.
This may be done prior to upgrading.
* Removed translations from most logging.
Deprecation Notes
*****************
* The "StatsdClient.set_prefix" method is now deprecated and may be
removed in a future release; by extension, so is the
"LogAdapter.set_statsd_prefix" method. Middleware developers should
use the "statsd_tail_prefix" argument to "get_logger" instead.
Bug Fixes
*********
* S3 API fixes
* Fixed the types of configured values in "/info" response.
* Fixed a server error when trying to copy objects with non-ASCII
names.
* Fixed a server error when uploading objects with very long
names. A "KeyTooLongError" is now returned.
* Fixed an error when multi-deleting MPUs when SLO async-deletes
are enabled.
* Fixed an error that allowed list-uploads and list-parts requests
to return incomplete or out-of-order results.
* Fixed several bugs when dealing with non-ASCII object names and
multipart uploads.
* Replication, reconstruction, and diskfile fixes
* Ensure that non-durable data and .meta files are purged from
handoffs after syncing.
* Fixed tracebacks when there's a race to mark a file durable or
delete it.
* Improved cooperative multitasking during ssync.
* Upon detecting a ring change, the reconstructor now only aborts
the jobs for that ring and continues processing jobs for other
rings.
* Fixed a traceback when logging about a lock timeout in the
replicator.
* Fixed a security issue where tempurl and s3api signatures were
logged in full. This allowed an attacker with access to log data to
perform replay attacks, potentially accessing or overwriting cluster
data. Now, such signatures are redacted in a manner similar to auth
tokens; see the "reveal_sensitive_prefix" option in "proxy-
server.conf".
See CVE-2017-8761 for more information.
* Fixed a race condition where swift would attempt to quarantine
recently-deleted object updates.
* Improved handling of timeouts and other errors when obtaining a
connection to memcached.
* The "swift-recon" tool now queries each object-server IP only once
when reporting disk usage. Previously, each port in the ring would
be queried; when using servers-per-port, this could dramatically
overstate the disk capacity in the cluster.
* Fixed a bug that allowed some statsd metrics to be annotated with
the wrong backend layer.
* Fixed a traceback in the account-server when there's no account
database on disk to receive a container update. The account-server
now correctly 404s.
* The container-updater will quarantine container databases if all
replicas for the account respond 404.
* Fixed a proxy-server error when the read-only middleware tried to
handle non-Swift paths (such as may be used by third-party
middleware).
* Some client behaviors that the proxy previously logged at warning
have been lowered to info.
* Various other minor bug fixes and improvements.
Changes in swift 2.28.0..2.29.0
-------------------------------
975d3dbcf AUTHORS/CHANGELOG for 2.29.0
c4762acaa Quiet more BadStatusLine tracebacks
a9565893f Add docs for registry module
00bb0f8ba read-only: Only act on Swift paths
086aa5c4f CI: fix lower-constraints job
f2c279bae Trim sensitive information in the logs (CVE-2017-8761)
eda7d5fe3 Deprecate LogAdapter.set_statsd_prefix
7f2adb4e8 CI: Use xena on CentOS 8 Stream
589ac355f Move *_swift_info functions into a new registry module
363aa3319 Fix multipart upload listings
5f25e1cc7 s3api: Fix non-ascii MPUs
ab52b2f77 tests: Clean up some dangling timeouts
9bc1c008a Get rid of pipeline_property
114440487 proxy-server: add stats for backend shard_listing requests
a50726f27 Make NodeIter require a logger argument
6942b25cc Fix statsd prefix mutation in proxy controllers
0a1955d88 s3api: tighten register_info unit test
4edf0cde4 s3api: Fix types in /info
8c6ccb5fd proxy: Add a chance to skip memcache when looking for shard ranges
876a7de6a De-clutter container sync sample internal client conf
03be71c44 Add FIPS CI jobs
11d102216 s3api: Allow multiple storage domains
7be5cb0ab Error limit the correct node on ECFragGetter resume
b8d7c3dcb Do not fetch content of container/object to retrieve S3 ACLs
035d91dce Modify log_name in internal clients' pipeline configs
1907594bd reconstructor: Abort just the changed policies
874a5865b tests: Improve FakeMemcache call tracking
de8886298 Finer grained ratelimit for update
f7101f379 tests: Unify FakeMemcaches
5079d8429 internal-client: pass global_conf to loadapp
b1f03149f Attempt fix test timeouts
40e0f577a Add stats for shard range cache hits/misses
54fc8a7de Fix cname_lookup test
ac92b3d8d CI: Add rolling upgrade job coming from stable/xena
d6206e6fe CI: Move py37 and py38 jobs to experimental pipeline
546c9629e Move CI from CentOS 8 to CentOS 8 Stream
219a79a24 updater: Add timing stats
eec595b12 updaters: use replication network
1b3879e0d reconstructor: include partially reverted handoffs in handoffs_remaining
8ee631cce reconstructor: restrict max objects per revert job
9acc44b15 Add some extra logging when EC decode fails
5979b6e15 account: Remove logging translations
7cfacbb25 container: Remove a bunch of logging translations
80d025233 object: Remove a bunch of logging translations
1942262db recon: only query each host once for disk usage
1eaf7474f Fix some imports for py310
346f518d6 Make arm jobs voting (but not the pipeline)
ada9f0eeb reconstructor: purge meta files in pure handoffs
092d409c4 reconstructor: silence traceback when purging
88eb360f5 ssync sender: add context to missing_check error log
e3069e6f7 reconstructor: remove non-durable files on handoffs
591adc845 Fix SSYNC update phase blocking
bb487bb20 Fix SSYNC/missing_check blocking (sender)
3ee262d8b Make cmp_policy_info agree with the API behaviour
3d190dba5 Clean up devstack job definitions
439fbbdc7 Ensure close socket for memcached if got timeout
898a1d790 memcache: Prevent possible pool exhaustion
8ebd0354a Test for PartitionLockTimeout
c4dca39a7 Make SAIO reconciler multiprocess
336599842 Fix log message interpolation
f1c411c52 proxy: Remove a bunch of logging translations
7f5e34a40 sharidng: update doc to only mention auto_shard experimental
d49dc9db5 Update master for stable/xena
4f84e77a6 Bump up timeout on swift-probetests-centos-8-arm64 job
a5fbe6ca4 ec: Use replication network to get frags for reconstruction
1cde3938a cname_lookup: Work with dnspython 2.0+
5b9a90b65 sharder: Make stats interval configurable
85d021127 Get rid of port to node assumptions and their modulo kludges
c15818f1e s3api: fix the copy of non-ASCII objects
47749cd0e Bug: fix s3api multipart parts listings
b4e532a46 func test improvements
9facfdc95 Clean up extra spaces
dfd1bdc9e Add documentation for DELETE method for Swift Object Store API.
930ac1114 Fix docker image builds
7a105b5ef Add and pipe reconstructor stats through recon
927691098 Plumb allow_modify_pipeline through run_wsgi/run_server
2992f1417 container GET: return 503 if policy index mismatches
56510ab3c container-server: return objects of a given policy
bcff1282b Band-aid and test the crash of the account server
619828483 Add a project scope read-only role to keystoneauth
61dd2ee44 s3api: Fix (async) multi-delete of MPUs
2a806296f func tests: plumb tolerate_missing through to listing_empty
c9b0795b9 diskfile: Stop raising ENOENT when we race to mark durable
460dcf756 s3api: Allow CORS preflights for pre-signed URLs
66ab1da99 updater: Stop trying to quarantine missing asyncs
ca0c34445 s3api: Return KeyTooLongError when upload name exceeds constraints
a61d154fa proxy: Downgrade some client problems to info
0d44328c7 Quiet more deprecation warnings on py2
9f9360833 use a context manager to apply --no-auto-shard option
Diffstat (except docs and test files)
-------------------------------------
.zuul.yaml | 119 ++--
AUTHORS | 7 +
CHANGELOG | 143 ++++-
Dockerfile | 2 +-
Dockerfile-py3 | 2 +-
api-ref/source/storage-account-services.inc | 64 +++
.../1.conf} | 2 +
etc/container-server.conf-sample | 3 +
etc/internal-client.conf-sample | 4 +-
etc/object-server.conf-sample | 24 +
etc/proxy-server.conf-sample | 42 +-
lower-constraints.txt | 2 +-
py2-constraints.txt | 1 +
.../notes/2_29_0_release-af71f7efd73109b0.yaml | 167 ++++++
releasenotes/source/index.rst | 2 +
releasenotes/source/xena.rst | 6 +
requirements.txt | 2 +-
swift/__init__.py | 4 +
swift/account/auditor.py | 5 +-
swift/account/server.py | 10 +-
swift/cli/container_deleter.py | 7 +-
swift/cli/recon.py | 126 +++--
swift/common/internal_client.py | 25 +-
swift/common/memcached.py | 32 +-
swift/common/middleware/account_quotas.py | 2 +-
swift/common/middleware/bulk.py | 4 +-
swift/common/middleware/cname_lookup.py | 5 +-
swift/common/middleware/container_quotas.py | 2 +-
swift/common/middleware/container_sync.py | 3 +-
swift/common/middleware/crossdomain.py | 2 +-
swift/common/middleware/crypto/__init__.py | 3 +-
swift/common/middleware/domain_remap.py | 4 +-
swift/common/middleware/etag_quoter.py | 3 +-
swift/common/middleware/formpost.py | 7 +-
swift/common/middleware/keystoneauth.py | 22 +-
swift/common/middleware/name_check.py | 3 +-
swift/common/middleware/proxy_logging.py | 34 +-
swift/common/middleware/ratelimit.py | 3 +-
swift/common/middleware/read_only.py | 10 +-
swift/common/middleware/recon.py | 9 +
swift/common/middleware/s3api/acl_handlers.py | 3 +
.../common/middleware/s3api/controllers/bucket.py | 4 +-
.../middleware/s3api/controllers/multi_delete.py | 11 +-
.../middleware/s3api/controllers/multi_upload.py | 98 ++--
swift/common/middleware/s3api/controllers/obj.py | 8 +-
.../common/middleware/s3api/controllers/s3_acl.py | 2 +-
.../middleware/s3api/controllers/versioning.py | 3 +-
swift/common/middleware/s3api/s3api.py | 66 ++-
swift/common/middleware/s3api/s3request.py | 24 +-
swift/common/middleware/s3api/s3response.py | 7 +-
swift/common/middleware/s3api/utils.py | 2 +-
swift/common/middleware/slo.py | 7 +-
swift/common/middleware/staticweb.py | 3 +-
swift/common/middleware/symlink.py | 3 +-
swift/common/middleware/tempauth.py | 12 +-
swift/common/middleware/tempurl.py | 8 +-
.../common/middleware/versioned_writes/__init__.py | 4 +-
swift/common/registry.py | 154 ++++++
swift/common/swob.py | 6 +-
swift/common/utils.py | 216 ++++----
swift/common/wsgi.py | 88 ++-
swift/container/reconciler.py | 16 +-
swift/container/server.py | 25 +-
swift/container/sharder.py | 63 ++-
swift/container/sync.py | 88 ++-
swift/container/updater.py | 55 +-
swift/obj/auditor.py | 31 +-
swift/obj/diskfile.py | 119 ++--
swift/obj/expirer.py | 42 +-
swift/obj/reconstructor.py | 151 +++--
swift/obj/replicator.py | 70 +--
swift/obj/server.py | 17 +-
swift/obj/ssync_receiver.py | 4 +
swift/obj/ssync_sender.py | 41 +-
swift/obj/updater.py | 239 ++++++--
swift/proxy/controllers/account.py | 7 +-
swift/proxy/controllers/base.py | 135 +++--
swift/proxy/controllers/container.py | 117 +++-
swift/proxy/controllers/info.py | 4 +-
swift/proxy/controllers/obj.py | 221 ++++----
swift/proxy/server.py | 56 +-
test/__init__.py | 4 +
test/cors/test-s3-obj.js | 25 +
test/functional/s3api/__init__.py | 7 +-
test/functional/s3api/s3_test_client.py | 23 +-
test/functional/s3api/test_bucket.py | 12 +-
test/functional/s3api/test_multi_delete.py | 4 +-
test/functional/s3api/test_multi_upload.py | 27 +-
test/functional/s3api/test_object.py | 15 +-
test/functional/swift_test_client.py | 8 +-
test/probe/common.py | 27 +-
test/probe/test_container_failures.py | 30 +-
test/probe/test_container_sync.py | 2 +-
test/probe/test_object_failures.py | 2 +-
test/probe/test_object_handoff.py | 33 +-
test/probe/test_orphan_container.py | 150 +++++
test/probe/test_reconstructor_revert.py | 4 +-
test/probe/test_sharder.py | 70 ++-
test/unit/__init__.py | 58 +-
test/unit/cli/test_container_deleter.py | 10 +
test/unit/cli/test_recon.py | 83 ++-
test/unit/common/middleware/crypto/test_crypto.py | 32 +-
test/unit/common/middleware/helpers.py | 2 +
test/unit/common/middleware/s3api/__init__.py | 2 +-
test/unit/common/middleware/s3api/test_acl.py | 12 +-
.../common/middleware/s3api/test_multi_delete.py | 23 +-
.../common/middleware/s3api/test_multi_upload.py | 175 ++++--
test/unit/common/middleware/s3api/test_obj.py | 37 +-
test/unit/common/middleware/s3api/test_s3api.py | 47 +-
.../unit/common/middleware/s3api/test_s3request.py | 55 +-
test/unit/common/middleware/s3api/test_utils.py | 4 +-
test/unit/common/middleware/test_bulk.py | 8 +-
test/unit/common/middleware/test_cname_lookup.py | 17 +-
test/unit/common/middleware/test_container_sync.py | 3 +-
test/unit/common/middleware/test_domain_remap.py | 10 +-
test/unit/common/middleware/test_keystoneauth.py | 40 +-
test/unit/common/middleware/test_name_check.py | 10 +-
test/unit/common/middleware/test_proxy_logging.py | 297 +++++++---
test/unit/common/middleware/test_ratelimit.py | 52 +-
test/unit/common/middleware/test_read_only.py | 54 +-
test/unit/common/middleware/test_recon.py | 26 +
test/unit/common/middleware/test_slo.py | 10 +-
test/unit/common/middleware/test_symlink.py | 3 +-
test/unit/common/middleware/test_tempauth.py | 68 +--
test/unit/common/middleware/test_tempurl.py | 38 +-
.../common/middleware/test_versioned_writes.py | 10 +-
test/unit/common/test_exceptions.py | 5 +-
test/unit/common/test_internal_client.py | 140 ++++-
test/unit/common/test_memcached.py | 35 ++
test/unit/common/test_registry.py | 296 ++++++++++
test/unit/common/test_utils.py | 401 +++++++-------
test/unit/common/test_wsgi.py | 105 ++--
test/unit/container/test_reconciler.py | 126 ++++-
test/unit/container/test_server.py | 124 +++++
test/unit/container/test_sharder.py | 39 +-
test/unit/container/test_sync.py | 28 +-
test/unit/container/test_updater.py | 54 +-
test/unit/obj/test_diskfile.py | 130 ++++-
test/unit/obj/test_expirer.py | 127 +++--
test/unit/obj/test_reconstructor.py | 451 ++++++++++++++-
test/unit/obj/test_replicator.py | 15 +
test/unit/obj/test_server.py | 4 +
test/unit/obj/test_ssync.py | 2 +-
test/unit/obj/test_ssync_sender.py | 181 +++++-
test/unit/obj/test_updater.py | 437 +++++++++++++--
test/unit/proxy/controllers/test_base.py | 33 +-
test/unit/proxy/controllers/test_container.py | 608 ++++++++++++++++++---
test/unit/proxy/controllers/test_info.py | 61 ++-
test/unit/proxy/controllers/test_obj.py | 146 ++++-
test/unit/proxy/test_server.py | 493 +++++++++++++----
tools/playbooks/multinode_setup/common_config.yaml | 2 +-
tools/test-setup.sh | 2 +-
164 files changed, 7088 insertions(+), 2080 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 223fc617b..3b74c3a87 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -8 +8 @@ netifaces>=0.8,!=0.10.0,!=0.10.1
-PasteDeploy>=1.3.3
+PasteDeploy>=2.0.0
More information about the Release-announce
mailing list