[release-announce] swift 2.30.0 (zed)
no-reply at openstack.org
no-reply at openstack.org
Wed Aug 24 13:28:28 UTC 2022
We eagerly announce the release of:
swift 2.30.0: OpenStack Object Storage
This release is part of the zed release series.
The source is available from:
https://opendev.org/openstack/swift
Download the package from:
https://tarballs.openstack.org/swift/
Please report issues through:
https://bugs.launchpad.net/swift/+bugs
For more details, please see below.
2.30.0
^^^^^^
New Features
************
* Sharding improvements
* The "swift-manage-shard-ranges" tool has a new mode to repair
gaps in the namespace.
* Metrics are now emitted for whether databases used for cleaving
were created or already existed, allowing a better understanding
of the reason for handoffs in the cluster.
* Misplaced-record stats are now also emitted to statsd.
Previously, these were only available in logs.
* Logging improvements
* The message template for proxy logging may now include a
"{domain}" field for the client-provided "Host" header.
* Added a "log_rsync_transfers" option to the object-replicator.
Set it to false to disable logging rsync "send" lines; during
large rebalances, such logging can overwhelm log aggregation while
providing little useful information.
* The formpost digest algorithm is now configurable via the new
"allowed_digests" option, and support is added for both SHA-256 and
SHA-512. Supported formpost digests are exposed to clients in
"/info". Additionally, formpost signatures can now be base64
encoded.
* Added metrics to the formpost and tempurl middlewares to monitor
digest usage in signatures.
* Improved compatibility with certain FIPS-mode-enabled systems.
* Added a "ring_ip" option for various object services. This may be
used to find own devices in the ring in a containerized environment
where the "bind_ip" may not appear in the ring at all.
* Account and container replicators can now be configured with a
"handoff_delete" option, similar to object replicators and
reconstructors. See the sample config for more information.
* Developers using Swift's memcache client may now opt in to having
a "MemcacheConnectionError" be raised when no connection succeeded
using a new "raise_on_error" keyword argument to "get"/"set".
* Device names are now included in new database IDs. This provides
more context when examining incoming/outgoing sync tables or
sharding CleaveContexts.
Deprecation Notes
*****************
* SHA-1 signatures are now deprecated for the formpost and tempurl
middlewares. At some point in the future, SHA-1 will no longer be
enabled by default; eventually, support for it will be removed
entirely.
Security Issues
***************
* Constant-time string comparisons are now used when checking S3 API
signatures.
* Fixed a socket leak when clients try to delete a non-SLO as though
it were a Static Large Object.
Bug Fixes
*********
* Sharding improvements
* Misplaced tombstone records are now properly cleaved.
* Fixed a bug where the sharder could fail to find a device to use
for cleaving.
* Databases marked deleted are now processed by the sharder.
* More information is now synced to the fresh database when
sharding. Previously, a database could lose the fact that it had
been marked as deleted.
* Shard ranges with no rows to cleave could previously be left in
the CREATED state after cleaving. Now, they are advanced to
CLEAVED.
* S3 API improvements
* Fixed cross-policy object copies. Previously, copied data would
always be written using the source container's policy. Now, the
destination container's policy will be used, avoiding availability
issues and unnecessary container-reconciler work.
* More headers are now copied from multi-part upload markers to
their completed objects, including "Content-Encoding".
* When running with "s3_acl" disabled, "bucket-owner-full-control"
and "bucket-owner-read" canned ACLs will be translated to the same
Swift ACLs as "private".
* The S3 ACL and Delete Multiple APIs are now less case-sensitive.
* Improved the error message when deleting a bucket that's ever
had versioning enabled and still has versions in it.
* "LastModified" timestamps in listings are now rounded up to
whole seconds, like they are in responses from AWS.
* Proxy logging for Complete Multipart Upload requests is now more
consistent when requests have been retried.
* Logging improvements
* Signal handling is more consistently logged at notice level.
Previously, signal handling would sometimes be logged at info or
error levels.
* The object-replicator now logs successful rsync transfers at
debug instead of info.
* Transaction IDs are now only included in daemon log lines in a
request/response context.
* The tempurl middleware has been updated to return a 503 if storing
a token in memcache fails. Third party authentication middlewares
are encouraged to also use the new "raise_on_error" keyword argument
when storing ephemeral tokens in memcache.
* Database replication connections are now closed following an error
or timeout. This prevents a traceback in some cases when the
replicator tries to reuse the connection.
* "ENOENT" and "ENODATA" errors are better handled in the object
replicator and auditor.
* Improved object update throughput by shifting some shard range
filtering from Python to SQL.
* Include "Vary: Origin" header when CORS responses vary by origin.
* The staticweb middleware now allows empty listings at the root of
a container. Previously, this would result in a 404 response.
* Ring builder output tables better display weights over 1000.
* Various other minor bug fixes and improvements.
Other Notes
***********
* Pickle support has been removed from Swift's memcache client.
Support had been deprecated since Swift 1.7.0.
Changes in swift 2.29.1..2.30.0
-------------------------------
f6196b0a2 AUTHORS/CHANGELOG for 2.30.0
6fd523947 Fix misuse of assertTrue
9abee0e78 Fix docker image building
dd99514e6 remove unicode prefix from code
9aa740f4f Imported Translations from Zanata
d24678dd5 s3api: Be more consistent about CompleteMultipartUpload logging
3a7b89506 Imported Translations from Zanata
3a71df133 Stop using unicode literals in docs conf.py
1c577fed1 ring-builder: Better format large weights
52a4fe37a Various doc formatting cleanups
7e5c78423 Update "Getting Started" requirements
91cfb0d6d Imported Translations from Zanata
517738ac9 sharder/replicator: emit stats for DBs created or existing
c4e00eb89 Sharder: Fall back to local device in get_shard_broker
59508de0c CI: Add nslookup_target to FIPS jobs
e6ee37274 slo: Reduce overhead for 'Not an SLO manifest' responses
38271142e sharder: process deleted DBs
21fab529c sharder: emit misplaced stats to statsd
2d063cd61 formpost: deprecate sha1 signatures
bc3625142 py310: Fix formatdate() call
bf4edefce DB Replicator: Add handoff_delete option
a5c1444fa Drain and close response in StaticLargeObject.get_slo_segments
25b6bd9f2 tempurl: Continue allowing sha1 by default
45e13ff4c Sharding: Sync container_stat table with fresh db
57f7145f7 sharder: always set state to CLEAVED after cleaving
d7c08d8ea Make the object replicator and auditor more robust to ENODATA
ac8f5550a sharder: fix probe tests skipping conditions
a55016e57 Imported Translations from Zanata
475cdba65 Emit metrics for tempurl & formpost digest usage
6af444926 s3api: Better handle 503s in get_container_info, too
1831658b3 proxy-logging: Allow to add domain in log messages
b9f1f4d60 Ensure clean starting state in test
91317ec14 Imported Translations from Zanata
367583c9f s3api: Make grentee types case insensitive
7a996a5c3 Fix s3api cross policy copy
ef31baf3f formpost: Add support for sha256/512 signatures
68e5a0b1c tests: Fix cross-test interaction
24648528a doc: Comment out language option
de13220c6 more tests for canned acls
12bc79bf0 Add ring_ip option to object services
5d9f1f009 s3api tests: allow AWS credential file loading
888142960 object-replicator: Remove some dead code
c33b3d860 s3api: Add best-effort support for more canned ACLs
27db5213d CI: Run s3api test suite
5112cf712 Add Jianjian to authors.
1a5e6d5c1 Update AUTHORS
19855db95 pytest: explicitly set system logger to DEBUG
238dc0353 CI: constrain py36 deps
b45b45fa7 manage-shard-ranges: add gap repair option
d0cf743b6 ceph-tests: Remove known-failure
019c955e1 sharder: ensure that misplaced tombstone rows are moved
d2b0c04d3 Add missing services to sample rsyslog.conf
75c5dbc29 trivial: add comment re sharder misplaced found stat
2f607cd31 Round s3api listing LastModified to integer resolution
52254bb5c Add ceil method to utils.Timestamp
99a4b9c7e AbstractRateLimiter: add option to burst on start-up
5227cb702 Refactor rate-limiting helper into a class
185b11e2f container-server: plumb includes down into _get_shard_range_rows
0b1cc8b0c More tests for rebalance_missing_suppression_count
94226bdd3 Don't give clients made up tokens
ab612dd26 tests: Save ourselves 20s of sleep
05b2e894a Log signal handling at notice
7298038ed Ignore py36 deprecation warnings
9bed525bf memcached: Give callers the option to accept errors
7e6917681 replicator: Log rsync file transfers less
043e0163e Clarify that rsync_io_timeout is also used for contimeout
0708edecd Drop arm64 probe test job
11b9761cd Rip out pickle support in our memcached client
118cf2ba8 tempurl: Deprecate sha1 signatures
0bf5474bf ceph tests: Register output/ceph-s3-summary.log as a job output
bb220f6f4 Doc: Update links in associated projects
5a272421d Swauth is retired
179fc43eb s3api: Improve error message when bucket is not empty
6142ce88c s3api: Use constant-time string comparisons in check_signature
1cee51d52 doc: also add reverse option to pagination doc
5c3bf6d26 replicator: Tolerate ENOENT when calling listdir
f6f474e42 db: Close ReplConnection sockets on errors/timeouts
a5a98d7e3 tests: Fix swiftclient/requests log level adjustment
d496d03b7 api-ref: Document `reverse` param
d29cbc399 CI: Run ceph and rolling upgrade tests under py3
fce7ad5f1 Ring: Change py2 only tests to py3
f92be1bdd Obj Auditor: Quarantine ENODATA
1c4acf2d8 s3api: Copy more headers from MPU marker to final object
fd2dd1156 s3api: Make the 'Quiet' key value case insensitive
ffb173f8a CI: Run CORS tests under py3
f83bfe1df Update master for stable/yoga
471a559a4 Stop partial()ing hashlib.new
8155e69b6 sharder: fix and expand CleavingContext docstrings
bab7f9322 cors: Include `Vary: Origin` when using the request's Origin
d94ab813f diskfile: Quarantine hashdirs on ENODATA
08da83c19 DB: Encode the device to the DB id
d13eeabdb Clear logger txn_id after making requests
57e41685b trivial: Replace assertRegexpMatches with assertRegex
1e410347f trivial: Replace assertRaisesRegexp with assertRaisesRegex
10767e482 staticweb: Allow empty listings at the root of a container
Diffstat (except docs and test files)
-------------------------------------
.mailmap | 2 +
.zuul.yaml | 76 +-
AUTHORS | 7 +-
CHANGELOG | 137 ++++
Dockerfile | 4 +-
Dockerfile-py3 | 4 +-
api-ref/source/parameters.yaml | 8 +
api-ref/source/storage-account-services.inc | 1 +
api-ref/source/storage-container-services.inc | 1 +
.../pseudo-hierarchical-folders-directories.rst | 112 +--
.../api/use_the_content-disposition_metadata.rst | 20 +-
etc/account-server.conf-sample | 13 +-
etc/container-server.conf-sample | 13 +-
etc/memcache.conf-sample | 10 -
etc/object-server.conf-sample | 15 +-
etc/proxy-server.conf-sample | 12 -
etc/swift-rsyslog.conf-sample | 5 +
py2-constraints.txt | 1 +
py36-constraints.txt | 88 +++
.../notes/2_30_0_release-642778c3010848db.yaml | 167 +++++
releasenotes/source/conf.py | 6 +-
releasenotes/source/index.rst | 2 +
.../locale/en_GB/LC_MESSAGES/releasenotes.po | 598 +++++++++++++++-
releasenotes/source/yoga.rst | 6 +
swift/__init__.py | 4 +
swift/account/backend.py | 3 +-
swift/cli/manage_shard_ranges.py | 122 +++-
swift/cli/ringbuilder.py | 8 +-
swift/common/daemon.py | 2 +-
swift/common/db.py | 6 +-
swift/common/db_auditor.py | 7 +-
swift/common/db_replicator.py | 35 +-
swift/common/digest.py | 151 ++++
swift/common/internal_client.py | 5 +-
swift/common/memcached.py | 55 +-
swift/common/middleware/crypto/decrypter.py | 2 +-
swift/common/middleware/crypto/keymaster.py | 8 +-
swift/common/middleware/formpost.py | 45 +-
swift/common/middleware/memcache.py | 15 -
swift/common/middleware/proxy_logging.py | 8 +
swift/common/middleware/s3api/acl_utils.py | 7 +-
.../common/middleware/s3api/controllers/bucket.py | 12 +-
.../middleware/s3api/controllers/multi_delete.py | 5 +-
.../middleware/s3api/controllers/multi_upload.py | 13 +-
swift/common/middleware/s3api/s3request.py | 13 +-
swift/common/middleware/s3api/s3response.py | 6 +
swift/common/middleware/s3api/schema/delete.rng | 2 +-
swift/common/middleware/s3api/subresource.py | 11 +-
swift/common/middleware/s3api/utils.py | 13 +-
swift/common/middleware/slo.py | 14 +-
swift/common/middleware/staticweb.py | 2 +-
swift/common/middleware/tempauth.py | 42 +-
swift/common/middleware/tempurl.py | 122 ++--
swift/common/ring/builder.py | 2 +-
swift/common/storage_policy.py | 4 +-
swift/common/utils.py | 246 ++++---
swift/common/wsgi.py | 9 +-
swift/container/backend.py | 86 ++-
swift/container/replicator.py | 47 +-
swift/container/server.py | 5 +-
swift/container/sharder.py | 510 +++++++------
swift/container/sync.py | 44 +-
swift/container/updater.py | 8 +-
swift/locale/de/LC_MESSAGES/swift.po | 764 +-------------------
swift/locale/en_GB/LC_MESSAGES/swift.po | 796 +--------------------
swift/locale/es/LC_MESSAGES/swift.po | 682 +-----------------
swift/locale/fr/LC_MESSAGES/swift.po | 589 +--------------
swift/locale/it/LC_MESSAGES/swift.po | 575 +--------------
swift/locale/ja/LC_MESSAGES/swift.po | 565 +--------------
swift/locale/ko_KR/LC_MESSAGES/swift.po | 558 +--------------
swift/locale/pt_BR/LC_MESSAGES/swift.po | 566 +--------------
swift/locale/ru/LC_MESSAGES/swift.po | 576 +--------------
swift/locale/tr_TR/LC_MESSAGES/swift.po | 533 +-------------
swift/locale/zh_CN/LC_MESSAGES/swift.po | 535 +-------------
swift/locale/zh_TW/LC_MESSAGES/swift.po | 539 +-------------
swift/obj/auditor.py | 16 +-
swift/obj/diskfile.py | 76 +-
swift/obj/reconstructor.py | 4 +-
swift/obj/replicator.py | 38 +-
swift/obj/server.py | 5 +-
swift/obj/ssync_receiver.py | 68 +-
swift/obj/updater.py | 60 +-
swift/proxy/controllers/base.py | 15 +-
swift/proxy/controllers/container.py | 17 +-
swift/proxy/controllers/info.py | 3 +-
swift/proxy/controllers/obj.py | 6 +-
test/__init__.py | 9 +
test/debug_logger.py | 13 +
test/functional/s3api/test_multi_upload.py | 87 +--
test/functional/s3api/test_object.py | 45 +-
test/functional/s3api/test_presigned.py | 4 +-
test/functional/test_object_versioning.py | 4 +-
test/functional/test_symlink.py | 4 +-
test/functional/test_tempurl.py | 37 +-
test/probe/test_sharder.py | 498 ++++++++++++-
test/s3api/__init__.py | 28 +-
test/unit/__init__.py | 19 +-
test/unit/account/test_backend.py | 37 +-
test/unit/cli/test_ipv6_output.stub | 10 +-
test/unit/cli/test_manage_shard_ranges.py | 392 +++++++++-
test/unit/cli/test_ringbuilder.py | 2 +-
.../common/middleware/crypto/test_keymaster.py | 2 +-
test/unit/common/middleware/helpers.py | 16 +-
.../unit/common/middleware/s3api/test_acl_utils.py | 51 +-
test/unit/common/middleware/s3api/test_bucket.py | 83 ++-
.../common/middleware/s3api/test_multi_delete.py | 40 +-
.../common/middleware/s3api/test_multi_upload.py | 287 +++++---
test/unit/common/middleware/s3api/test_obj.py | 105 +--
test/unit/common/middleware/s3api/test_s3_acl.py | 11 +
test/unit/common/middleware/s3api/test_s3api.py | 6 -
.../unit/common/middleware/s3api/test_s3request.py | 16 +-
test/unit/common/middleware/s3api/test_utils.py | 57 +-
test/unit/common/middleware/test_formpost.py | 227 +++++-
test/unit/common/middleware/test_memcache.py | 68 +-
test/unit/common/middleware/test_proxy_logging.py | 6 +-
test/unit/common/middleware/test_slo.py | 59 +-
test/unit/common/middleware/test_staticweb.py | 20 +
test/unit/common/middleware/test_tempauth.py | 9 +
test/unit/common/middleware/test_tempurl.py | 167 +++--
test/unit/common/ring/test_builder.py | 26 +-
test/unit/common/ring/test_ring.py | 135 ++--
test/unit/common/ring/test_utils.py | 21 +-
test/unit/common/test_daemon.py | 12 +-
test/unit/common/test_db_replicator.py | 71 +-
test/unit/common/test_digest.py | 191 +++++
test/unit/common/test_internal_client.py | 35 +-
test/unit/common/test_memcached.py | 92 ++-
test/unit/common/test_utils.py | 176 ++++-
test/unit/common/test_wsgi.py | 29 +
test/unit/container/test_backend.py | 254 ++++++-
test/unit/container/test_replicator.py | 78 ++
test/unit/container/test_sharder.py | 489 +++++++++++--
test/unit/obj/test_diskfile.py | 136 ++++
test/unit/obj/test_reconstructor.py | 54 +-
test/unit/obj/test_replicator.py | 223 ++++++
test/unit/obj/test_ssync_receiver.py | 2 +-
test/unit/obj/test_updater.py | 51 +-
test/unit/proxy/controllers/test_container.py | 43 +-
test/unit/proxy/controllers/test_info.py | 22 +-
test/unit/proxy/controllers/test_obj.py | 25 +
test/unit/proxy/test_server.py | 23 +
tools/playbooks/common/restart_swift.yaml | 24 +
tools/playbooks/cors/run.yaml | 2 +-
tools/playbooks/multinode_setup/make_rings.yaml | 2 +-
tools/playbooks/multinode_setup/pre.yaml | 1 -
tools/playbooks/multinode_setup/run.yaml | 5 +-
tox.ini | 19 +
182 files changed, 7341 insertions(+), 9491 deletions(-)
More information about the Release-announce
mailing list