[release-announce] swift 2.32.0 (bobcat)
no-reply at openstack.org
no-reply at openstack.org
Fri Sep 1 13:50:03 UTC 2023
We are stoked to announce the release of:
swift 2.32.0: OpenStack Object Storage
This release is part of the bobcat release series.
The source is available from:
https://opendev.org/openstack/swift
Download the package from:
https://tarballs.openstack.org/swift/
Please report issues through:
https://bugs.launchpad.net/swift/+bugs
For more details, please see below.
2.32.0
^^^^^^
New Features
************
* Python 3.11 is now supported.
* Added the ability for reseller admins to set per-policy account
quotas by posting metadata of the form "X-Account-Quota-Bytes-
Policy-<policy name>".
* Added a "keepalive_timeout" option to the proxy server to limit
how long to wait for a client to initiate a request, separate from
the general "client_timeout" option. Note that this requires
eventlet 0.33.4 (currently unreleased) or later.
* Added a "keep_cache_slo_manifest" option to the object server to
better control whether SLO manifests are dropped from the page
cache.
* WSGI servers now accept a "--test-config" option that may be used
to validate configuration changes before reloading/restarting the
server.
* Metrics improvements:
* Metrics are now emitted for a variety of S3 error responses, in
the form "s3api.<status_int>.<error_class>[.<reason>]"
* Account and container info metrics now include the response
status code when backend requests are made.
* Added timing metrics to the container sharder for various
operations.
* A variety of performance improvements have been made for sharded
container databases.
* Various logging and metrics improvements when talking to memcache.
* Improved formatting of meta and sysmeta for "swift-account-info"
and "swift-container-info".
Upgrade Notes
*************
* Previously, under some circumstances, a non-standard config option
such as "RECLAIM_AGE" might get parsed as "reclaim_age" for some
processes but ignored by others. Now, all config parsing is case-
sensitive; non-standard names will always be ignored.
* The structure of cached shard ranges has changed, improving
performance when listing or writing to sharded containers. Note that
immediately after upgrade, the new structures will all be cache
misses, which may lead to a thundering herd problem. To avoid this,
upgrade just a few nodes first, let them service some fraction of
traffic to populate the cache, then upgrade the rest of the cluster.
Deprecation Notes
*****************
* Removed the hard dependency on netifaces; it may still be used if
the "getifaddrs" C function is not available. This fallback support
may be removed in a future release.
Bug Fixes
*********
* Python 3 fixes:
* Python 3 object servers can now read unencrypted non-ASCII
metadata that was written under Python 2.
* Ssync no longer corrupts unencrypted non-ASCII metadata during
transfers.
* Fixed an encoding issue when writing non-ASCII object names to
sharded containers and shard range caching is not enabled.
* Fixed an encoding issue when handling non-ASCII account names.
* Fixed a "generator already executing" error on client
disconnect.
* Suppressed "RemoteDisconnected" tracebacks.
* Fixed an issue that prevented proxy servers from emitting metrics
and logs for backend requests made when getting account or container
info.
* Fixed ssync's handling of timestamp offsets. Previously, this
could cause ssync to fail with a 409 Conflict, causing the transfer
to fail and preventing handoffs from clearing.
* Fixed an issue where an erasure-coded PUT could prevent other
requests from being processed when network calls rarely or never
blocked.
* Fixed an issue when downloading an SLO manifest would hit a
recoverable error and attempt to resume from another node. This
would manifest as either a pyeclib decode error or an unexpected
empty response.
* The proxy server now applies error-limiting to the correct node
when handling a recoverable node error.
* Account, container, and object log fields are now correctly
identified when returning "BadDigest" responses to S3 requests.
* Reduced the backend load of making "?versions" requests to a
container that has never had object versioning enabled.
* The standard-library "logging" module is no longer monkey-patched
when importing "swift.common.utils", making it easier to re-use
swift code in other contexts.
* Various other minor bug fixes and improvements.
Changes in swift 2.31.1..2.32.0
-------------------------------
191b627b8 Authors/ChangeLog for 2.32.0
f06e53695 Add rolling-upgrade job from Antelope
287fbadc1 tests: Pollute stderr less
1edf7df75 Partially revert "Pull libc-related functions out to a separate module"
336c64338 s3api: set swift.backend_path when returning 422
8f85e27c2 FakeSwift: use env['PATH_INFO'] to index uploaded objects
746daad33 FakeSwift: add tests
8bd21db25 FakeSwift: add docstring
dab7192e1 tests: fix FakeSwift HEAD with query param
ac524832e direct_client: support extra request params for direct_get_container.
9e065e2d2 fix swob HEAD handling for leak tracking
e5d730dc5 proxy: Get rid of iter_nodes helper
6444ef9be ContainerBroker: clarify get_shard_ranges docstring
4a8879a36 proxy-server: fix AttributeError in exception handler
6251539dc proxy-server: de-duplicate implementation of is_good_source
629cd5751 proxy-server: clarify method naming in GET path
fe09ef258 Save some effort when discovering used_source_etag
f8a6dee62 proxy: encapsulate Getter resp, node and parts_iter
20dc88860 s3api: extra unit test assertion for statsd metric prefix
00bfc425c Add FakeStatsdClient to unit tests
0c9b545ea docs: Clean up proxy logging docs
27120b052 docs/SAIO: Make reconciler use the SAIO convention for logs
dd664c47d doc/SAIO: Update the Fedora section
cb468840b Add non-ascii meta values to ssync probe test
6e8d82c97 container-server: use LIMIT in get_own_shard_range() query
bc300a516 proxy: add new metrics to account/container_info cache for skip/miss
b42d90a34 py3: apply str_to_wsgi to includes in _get_shard_ranges
c51e81f64 proxy: Bring back logging/metrics for get_*_info requests
8a5baf8e1 Imported Translations from Zanata
369a72c4c proxy: remove client_chunk_size and skip_bytes from GetOrHeadHandler
8e051c5b8 docs: Fix link to sphinx project page
46eeea11d Add a docstring for utils.Namespace
6d5301050 sharder: use ContainerBroker.has_other_shard_ranges() method
bfbe8f909 Container-server: only check for shard range existence for DB state queries
f7d6d5806 container-server: do [end_]marker filtering in SQL query
bdbe8ce9f s3api: fix statsd prefix mutation
771d605d0 Add more unit tests for ranged GETs
365c0ef00 Encode header in latin-1 with wsgi_to_bytes
cb1e584e6 Object-server: keep SLO manifest files in page cache.
1b7cf2947 Object-server: backfill unit test coverage for keep_cache_private
14a227f30 versioning: Only list versions container if it seems to exist
e82536c64 Properly format meta and sysmeta in db info cli tools
a82d9c58b Rename TestSource which is not a TestCase
f9af0b70b s3api: emit metrics for error responses
20757612c proxy: Pop ignore-range backend header on resume
82b5335fd tests: Stop requiring <1ms test runtime
1c210d2e4 Change getting major:minor of blkdev
5ef169b92 Imported Translations from Zanata
68e48c868 proxy: error limit the correct node when resuming GETs
e290d47c4 unify duplicate code in replication, EC GET paths
0e0db9003 sharder: Fix completed metric emission
483e17d5b py3: Quiet RemoteDisconnected tracebacks
ca3f10770 tests: Fix replicator test for py311
f955e8104 CI: test under py311
0235db3d3 tests: Stop trying to mutate instantiated EntryPoints
c5a3354fe Green GreenDBConnection.execute
b46b735a3 Fix handling of non-ASCII accounts
be0c481e4 Fix proxy traceback for GeneratorExit in py3
4c986d97b Use statsd_client more directly
f150a5035 proxy: print logs when write new shard range cache into memcache.
d2838ada4 proxy: Remove unused arg from set/clear_info_cache
7788c2c92 container: Stop pulling alternate stats when getting policy index
1f298714a docs: Format metrics in fixed-width font, not italics
307315bde docs: Move metric name/description tables out to separate page(s)
149b617c2 Fix GetOrHeadHandler unit tests to use public interface
a25087e1c memcached: add method and time_spent to exception logs
49a5293cb memcached: handle failed incr+add+incr better
effd4931d Use format_exc, not format_stack
d3d503f90 Drop more translations
23fa18d30 Remove hard dependency on netifaces
fd6a1ad65 Refactor for extract closure to method
e29e2c3ae Move IP-address-related functions out to new module
776a9d9a1 Use the db for get_shard_usage
2deab27da s3api: Better logging for non-JSON when trying to do bucket listings
f714f43c9 memcached: encapsulate common command parameters
aa96cb3dc proxy: add periodic zero-time sleep during object PUT
4bc86ec30 Imported Translations from Zanata
66e6ee624 tests: Make dark data probe tests pass with sync_method = ssync
cff092b68 docs: Update versions in Getting Started doc
ca0dd8d70 docs: Remove references to out-dated install guides
c4c9d5a40 move test_GET_pipeline to BaseTestObjectController
9adc3ce42 disable requests_mock pytest plugin
7b4642567 Memcached: emit memcache timing metrics when exceptions raised
647ee8390 Unit test for keepalive timeout
01a998a45 more explicit catch
9d98721e7 backend_ratelimit: Tighten blanket exception handling
744e9a94a Sharder: add timing metrics for individual steps and total time spent.
d393fa482 testing xattr metadata with py3.8
780754096 Properly read py2 object metadata on py3
9fb860880 memcached: log user provided keys in exception error logging.
84b995f27 Don't monkey patch logging on import
8d23dd8ac Log (Watchdog's) Timeouts with duration
ab03e057d Make all config parsing case-sensitive
9dfbac7c4 Error logs changed for ChunkWriteTimeout
88941ebe4 tests: Fix config numbers in test_versioning_with_metadata_replication
716ae48eb docs: Fix broken paste/pastedeploy links
b8f0a0ed5 ring: Centralize device normalization
0518c6023 CI: Use [tox]requires to pin virtualenv
50c4ea032 ECFragGetter: assume policy.fragment_size is non-zero
0f95870c5 ECFragGetter: simplify iter_bytes_from_response_part
61b6ec263 Assert ChunkWriteTimeout errors are logged
74fc18d90 Extract some closures to methods
0a2bfe981 Use instance attr instead of non-local
105727bc8 tests: Fix test_cleanup_ondisk_files_commit_window
bfc138627 CI: fix lower-constraints job
e19a165a7 proxy controller: always pass x-backend-* headers to backend
ed1f5193e docs: Clean up cross-domain doc formatting; call out CWE-942
469c38e9f wsgi: Add keepalive_timeout option
71d507f8e Proxy: restructure cached listing shard ranges
4e74e7f55 ssync: Round-trip offsets in meta/ctype Timestamps
c95f8e6c0 tests for wsgi/daemon config parsing
bbf9687b7 InternalClient: error if allow_modify_pipeline is True
e5105ffa0 internal_client: Remove allow_modify_pipeline option
12bfcb5cd tests: Fix PriorityQueue import
c78a5962b Pull libc-related functions out to a separate module
f264b53a2 ring-builder: Abort cleanly on ^C, ^D
85d68ee49 Test nested (Metrics)PrefixLoggerAdapters
0a4e41701 Add cap_length helper
0a5f0253b Add --test-config option to WSGI servers
c21256d87 Pull timestamp-related functions out to a separate module
91db804e6 Start common.utils refactor
0dff2cae9 Patch policies for more per-policy quota tests
6885b3367 sharder: fix host logged when send_shard_ranges fails
e15d6880e sharder: fix 'Cleaving not completed' warning log
5032b42b1 Update url
56a9f72ec Forward-port stable-release CHANGELOG entries
cbba65ac9 quotas: Add account-level per-policy quotas
29414ab14 Allow internal container POSTs to not update put_timestamp
9a1bfb897 proxy: add tests for listing with shard gaps
acf31a61d Rename ShardRange*Bound to Namespace*Bound
ea6c48321 Update master for stable/2023.1
052bcadb2 tests: Skip s3api functional tests when no s3api user configured
78f13be75 tests: Let func tests run with test users 1 and 2 but not 3
479a01016 Drop bandit check B309
6ff90ea73 Proxy: restructure cached updating shard ranges
2fe18b24c ssync: fix decoding of ts_meta when ts_data has offset
Diffstat (except docs and test files)
-------------------------------------
.zuul.yaml | 30 +-
AUTHORS | 4 +
CHANGELOG | 272 ++
README.rst | 2 +-
bandit.yaml | 3 +-
bin/swift-account-server | 2 +-
bin/swift-container-server | 2 +-
bin/swift-drive-audit | 59 +-
bin/swift-object-server | 2 +-
bin/swift-proxy-server | 2 +-
etc/internal-client.conf-sample | 1 +
etc/object-server.conf-sample | 5 +
etc/proxy-server.conf-sample | 13 +-
.../notes/2_32_0_release-39c8fb77a0a3e72d.yaml | 122 +
releasenotes/source/2023.1.rst | 6 +
releasenotes/source/index.rst | 14 +-
.../locale/en_GB/LC_MESSAGES/releasenotes.po | 482 ++-
requirements.txt | 1 -
swift/cli/info.py | 11 +-
swift/cli/ringbuilder.py | 6 +-
swift/common/container_sync_realms.py | 13 +-
swift/common/daemon.py | 10 +-
swift/common/db.py | 29 +-
swift/common/db_replicator.py | 39 +-
swift/common/direct_client.py | 77 +-
swift/common/internal_client.py | 27 +-
swift/common/manager.py | 87 +-
swift/common/memcached.py | 242 +-
swift/common/middleware/__init__.py | 4 +
swift/common/middleware/account_quotas.py | 138 +-
swift/common/middleware/backend_ratelimit.py | 10 +-
swift/common/middleware/crossdomain.py | 29 +-
swift/common/middleware/proxy_logging.py | 21 +-
.../common/middleware/s3api/controllers/bucket.py | 10 +-
swift/common/middleware/s3api/s3api.py | 13 +-
swift/common/middleware/s3api/s3request.py | 56 +-
swift/common/middleware/s3api/s3response.py | 11 +-
swift/common/middleware/tempauth.py | 19 +-
.../common/middleware/versioned_writes/__init__.py | 3 -
.../versioned_writes/object_versioning.py | 27 +-
swift/common/request_helpers.py | 3 +-
swift/common/ring/ring.py | 41 +-
swift/common/swob.py | 3 +-
swift/common/{utils.py => utils/__init__.py} | 1756 ++++-------
swift/common/utils/ipaddrs.py | 256 ++
swift/common/utils/libc.py | 345 +++
swift/common/utils/timestamp.py | 399 +++
swift/common/wsgi.py | 109 +-
swift/container/backend.py | 144 +-
swift/container/server.py | 13 +-
swift/container/sharder.py | 49 +-
swift/locale/de/LC_MESSAGES/swift.po | 321 +-
swift/locale/en_GB/LC_MESSAGES/swift.po | 348 +--
swift/locale/es/LC_MESSAGES/swift.po | 280 +-
swift/locale/fr/LC_MESSAGES/swift.po | 210 +-
swift/locale/it/LC_MESSAGES/swift.po | 208 +-
swift/locale/ja/LC_MESSAGES/swift.po | 280 +-
swift/locale/ko_KR/LC_MESSAGES/swift.po | 306 +-
swift/locale/pt_BR/LC_MESSAGES/swift.po | 205 +-
swift/locale/ru/LC_MESSAGES/swift.po | 204 +-
swift/locale/tr_TR/LC_MESSAGES/swift.po | 193 +-
swift/locale/zh_CN/LC_MESSAGES/swift.po | 193 +-
swift/locale/zh_TW/LC_MESSAGES/swift.po | 194 +-
swift/obj/diskfile.py | 25 +-
swift/obj/server.py | 26 +-
swift/obj/ssync_receiver.py | 28 +-
swift/obj/ssync_sender.py | 20 +-
swift/proxy/controllers/account.py | 14 +-
swift/proxy/controllers/base.py | 740 ++---
swift/proxy/controllers/container.py | 165 +-
swift/proxy/controllers/obj.py | 644 ++--
swift/proxy/server.py | 11 +-
test/debug_logger.py | 135 +-
test/functional/s3api/__init__.py | 4 +
test/functional/test_dlo.py | 4 +-
test/functional/test_domain_remap.py | 14 +
test/functional/test_object_versioning.py | 3 +-
test/functional/test_slo.py | 9 +-
test/functional/test_versioned_writes.py | 81 +-
test/probe/test_container_failures.py | 73 +-
test/probe/test_dark_data.py | 3 +-
test/probe/test_object_versioning.py | 99 +-
test/probe/test_reconstructor_revert.py | 6 +-
test/unit/__init__.py | 79 +-
test/unit/account/test_auditor.py | 5 +-
test/unit/account/test_backend.py | 28 +-
test/unit/account/test_reaper.py | 9 +-
test/unit/account/test_server.py | 2 +-
test/unit/cli/test_info.py | 19 +-
test/unit/common/middleware/helpers.py | 129 +-
test/unit/common/middleware/s3api/__init__.py | 4 +-
test/unit/common/middleware/s3api/test_bucket.py | 19 +
.../common/middleware/s3api/test_multi_upload.py | 29 +
test/unit/common/middleware/s3api/test_obj.py | 4 +
test/unit/common/middleware/s3api/test_s3api.py | 234 +-
test/unit/common/middleware/test_account_quotas.py | 93 +
.../common/middleware/test_backend_ratelimit.py | 6 +-
test/unit/common/middleware/test_dlo.py | 24 +
test/unit/common/middleware/test_formpost.py | 2 +-
test/unit/common/middleware/test_helpers.py | 450 +++
.../common/middleware/test_object_versioning.py | 133 +-
test/unit/common/middleware/test_proxy_logging.py | 16 +-
test/unit/common/middleware/test_slo.py | 47 +-
test/unit/common/middleware/test_symlink.py | 13 +-
test/unit/common/middleware/test_tempauth.py | 11 +-
test/unit/common/middleware/test_tempurl.py | 2 +-
test/unit/common/ring/test_ring.py | 16 +-
test/unit/common/test_daemon.py | 126 +-
test/unit/common/test_db_replicator.py | 10 +-
test/unit/common/test_direct_client.py | 132 +-
test/unit/common/test_internal_client.py | 82 +
test/unit/common/test_memcached.py | 459 ++-
test/unit/common/test_swob.py | 24 +-
test/unit/common/test_utils.py | 3178 ++++++++------------
test/unit/common/test_wsgi.py | 261 +-
test/unit/common/utils/__init__.py | 0
test/unit/common/utils/test_ipaddrs.py | 216 ++
test/unit/common/utils/test_libc.py | 167 +
test/unit/common/utils/test_timestamp.py | 882 ++++++
test/unit/container/test_backend.py | 282 +-
test/unit/container/test_replicator.py | 27 +-
test/unit/container/test_server.py | 117 +-
test/unit/container/test_sharder.py | 425 ++-
test/unit/helpers.py | 1 +
test/unit/obj/test_diskfile.py | 288 +-
test/unit/obj/test_reconstructor.py | 14 +-
test/unit/obj/test_replicator.py | 20 +-
test/unit/obj/test_server.py | 371 ++-
test/unit/obj/test_ssync.py | 32 +-
test/unit/obj/test_ssync_receiver.py | 50 +-
test/unit/obj/test_ssync_sender.py | 90 +-
test/unit/obj/test_updater.py | 119 +-
test/unit/proxy/controllers/test_base.py | 483 ++-
test/unit/proxy/controllers/test_container.py | 328 +-
test/unit/proxy/controllers/test_obj.py | 328 +-
test/unit/proxy/test_server.py | 433 ++-
tox.ini | 7 +-
177 files changed, 12701 insertions(+), 8746 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index 3b74c3a87..e9d54faaf 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -7 +6,0 @@ greenlet>=0.3.2
-netifaces>=0.8,!=0.10.0,!=0.10.1
More information about the Release-announce
mailing list