[release-announce] swift 2.20.0 (stein)
no-reply at openstack.org
no-reply at openstack.org
Mon Dec 17 21:10:05 UTC 2018
We are satisfied to announce the release of:
swift 2.20.0: OpenStack Object Storage
This release is part of the stein release series.
Download the package from:
https://tarballs.openstack.org/swift/
For more details, please see below.
2.20.0
^^^^^^
New Features
* S3 API compatibility updates
* Swift can now cache the S3 secret from Keystone to use for
subsequent requests. This functionality is disabled by default but
can be enabled by setting the "secret_cache_duration" in the
"[filter:s3token]" section of the proxy server config to a number
greater than 0.
* s3api now mimics the AWS S3 behavior of periodically sending
whitespace characters on a Complete Multipart Upload request to
keep the connection from timing out. Note that since a request
could fail after the initial 200 OK response has been sent, it is
important to check the response body to determine if the request
succeeded.
* s3api now properly handles "x-amz-metadata-directive" headers on
COPY operations.
* s3api now uses concurrency (default 2) to handle multi-delete
requests. This allows multi-delete requests to be processed much
more quickly.
* s3api now mimics some forms of AWS server-side encryption based
on whether Swift's at-rest encryption functionality is enabled.
Note that S3 API users are now able to know more about how the
cluster is configured than they were previously, ie knowledge of
encryption at-rest functionality being enabled or not.
* s3api responses now include a '-' in multipart ETags.
For new multipart-uploads via the S3 API, the ETag that is stored
will be calculated in the same way that AWS uses. This ETag will
be used in GET/HEAD responses, bucket listings, and conditional
requests via the S3 API. Accessing the same object via the Swift
API will use the SLO Etag; however, in JSON container listings the
multipart upload etag will be exposed in a new "s3_etag" key.
Previously, some S3 clients would complain about download
corruption when the ETag did not have a '-'.
* S3 ETag for SLOs now include a '-'.
Ordinary objects in S3 use the MD5 of the object as the ETag, just
like Swift. Multipart Uploads follow a different format, notably
including a dash followed by the number of segments. To that end
(and for S3 API requests *only*), SLO responses via the S3 API
have a literal '-N' added on the end of the ETag.
* The default location is now set to "us-east-1". This is more
likely to be the default region that a client will try when using
v4 signatures.
Deployers with clusters that relied on the old implicit default
location of "US" should explicitly set "location = US" in the
"[filter:s3api]" section of proxy-server.conf before upgrading.
* Add basic support for ?versions bucket listings. We still do not
have support for toggling S3 bucket versioning, but we can at
least support getting the latest versions of all objects.
* Fixed an issue with SSYNC requests to ensure that only one request
can be running on a partition at a time.
* Data encryption updates
* The "kmip_keymaster" middleware can now be configured directly
in the proxy-server config file. The existing behavior of using an
external config file is still supported.
* Multiple keymaster middlewares are now supported. This allows
migration from one key provider to another.
Note that "secret_id" values must remain unique across all
keymasters in a given pipeline. If they are not unique, the right-
most keymaster will take precedence.
When looking for the active root secret, only the right-most
keymaster is used.
* Prevent PyKMIP's kmip_protocol logger from logging at DEBUG.
Previously, some versions of PyKMIP would include all wire data
when the root logger was configured to log at DEBUG; this could
expose key material in logs. Only the "kmip_keymaster" was
affected.
* Fixed an issue where a failed drive could prevent the container
sharder from making progress.
* Storage policy definitions in swift.conf can now define the
diskfile to use to access objects. See the included swift.conf-
sample file for a description of usage.
* The EC reconstructor will now attempt to remove empty directories
immediately, while the inodes are still cached, rather than waiting
until the next run.
* Added a "keep_idle" config option to configure KEEPIDLE time for
TCP sockets. The default value is the old constant of 600.
* Add "databases_per_second" to the account-replicator, container-
replicator, and container-sharder. This prevents them from using a
full CPU core when they are not IO limited.
* Allow direct_client users to overwrite the "X-Timestamp" header.
* Various other minor bug fixes and improvements.
Changes in swift 2.19.0..2.20.0
-------------------------------
fbad538 authors/changelog for 2.20.0 release
defae1d Fix rolling upgrade job
f98d5d1 Py3: Add working tests to tox.ini
7458f23 Python3: fix test_xprofile.py
b0aea93 s3 secret caching
bce2ac6 Fix container sharding doc
fba94b2 fix a typo in docstring
f765f4b Py3: Use BytesIO when we want bytes
00be3f5 s3api: Allow concurrent multi-deletes
8e95a93 s3api: Allow some forms of server-side-encryption
b7e2278 Support long-running multipart uploads
3379235 api-ref: Clean up container and object response code formatting
9ab276d Change openstack-dev to openstack-discuss
c26d67e fixed _check_node() in the container sharder
a883ada Py3: Replace map() with a list comprehension
014d46f Fix SSYNC concurrency on partition
333ae30 Fix in-process func tests to work with newer PasteDeploy
010cdc2 Py3: Fix test_memcache
59d66e3 Py3: fix a simple bytes vs str issue
3a51771 Py3: Use dict.items instead of dict.iteritems
d75d15b Py3: We cannot use len(filter(...))
21a3423 Py3: Fix a relative import
c3cbe69 Py3: Use collections.MutableMapping rather than UserDict.DictMixin
6c8b4c8 Py3: Port test/unit/test_ringcomposer.py
9017db4 Only print "SKIPPING FUNCTIONAL TESTS ..." messages once
34aef9d Add chaging secret key for external KMS section
fda3052 Multi-key KMS keymaster
4809884 Use eventlet.patcher.original to get Python select module in get_hub
42b4cdc s3api: Include '-' in S3 ETags of normal SLOs
582f058 py3: encryption follow-up
37b8146 py3: port encryption
3465d63 Add missing ws seperator between words
370f7d7 Handle non-ASCII characters in S3 v2 listings.
692a034 s3api: Change default location to us-east-1
a85dc0a update docs link address
c1c65a7 Only url-quote Keys when encoding-type=url
662fd81 Fix typo
c5e33b0 Fix typo
a2fb335 s3api: Add basic support for ?versions bucket listings
34e48d5 Remove setup.py check from pep8 job
bc4494f py3: port account/container replicators
c112203 py3: Monkey-patch json.loads to accept bytes on py35
7a21272 Update min tox version to 2.3.2
887ba87 Add "historical document" note to ring background docs
3420921 Clean up HASH_PATH_* patching
11e81cf added note about double url quoting
441df4f Use correct headers in reconstructor requests
e8a7729 Fix usage of multiple accounts in functional tests
06cf5d2 Add databases_per_second to db daemons
2d1c438 SSYNC: stop sharing global available_map/send_map
6b94cf2 SSYNC: Stop sharing a global response
e4ad56a SSYNC: Stop sharing a global connection
46c6fab SSYNC: Remove useless self.failures in sender
d306345 Remove empty directories after a revert job
1663782 Fix up the test for .ismount
654187e py3: adapt common/db_replicator.py
b5a9172 Require gettext for all non-SUSE distros
0a564d8 Check for .ismount stubs with symlinks, too
3bf9f82 Reduce code duplication in SSYNC sender
781da07 Stop accomodating 5+ year old object-server code in unit tests
6676d74 py3: everything account except the real things
b39d2ef Allow multiple keymasters
00f7732 Simplify the decryption of container listings
0d77486 Unify handle_get/handle_head in decrypter
28a582f py3: adapt account/backend.py and utils.py
83a7ce8 Python 3 compatibility: fix xrange/range issues
b90c792 Follow up s3api ensures any string for secret as bytes string
45c840a s3api: Ensure secret is utf8 in check_signature
dd49f17 Add coverage reports to py27/35/36 unit test jobs
ce51893 Tighten status code assertions for object POST
78c9fd9 Change PUT bucket conflict error
4ca605c Move base64-decoding/unpickling into DatabaseBroker
6e7ca26 DB follow-up
3770f3f internal_client: Require that request_tries be positive
e3a5b63 py3: Adapt db.py
5558b92 Properly handle custom metadata upon an object COPY operation
12b6d46 Prevent kmip_protocol logger from logging at DEBUG
27d8523 Give better errors for malformed credentials
c90d34b py3: port tempauth
c8e3b99 Enabling direct_client users to overwrite X-Timestamp
9686680 Tolerate missing containers when trying to clean up
93fb77e Ignore ENOENT and ENOTEMPTY errors in delete_partition
8c7ca01 Unquote URL before using splited parts.
36a3fee update 1space url in associated projects
ae49663 py3: port internal_client
ddd5cc1 py3: port direct_client
b7fda3b Allow kmip_keymaster to be configured in proxy-server.conf
ecebea1 s3api: Increase max body size for Delete Multiple Objects requests
050f879 Use latest eventlet in probe tests
70d31e0 set up a lower constraints job that uses an XFS tmpdir
62f8e64 Use templates for cover and lower-constraints
1b0172f py3: Port listing_formats middleware
78344bf py3: get proxy-server willing and able to respond to some API requests
904e7c9 Add more doc and test for cors_expose_headers option
5d70396 add multinode functional test to swift
0c97888 fixed a manpage and added the manpage syntax checker to pep8 jobs
2ef21ac py3: port request_helpers
a7dd5f6 Replace Chinese quotes with English quotes
b4567e6 py3: include test_base_storage_server
cfeb32c Adding keep_idle config value to socket
dbacdcf Add punch_hole utility function
1142202 py3: port list_endpoints
5a8cfd6 Add another user for s3api func tests
80001aa Add ceph-s3 test non-voting job
123f6f5 Allow EC to ignore invalid request ETag
cbfa585 Refactor obj.server.ObjectController.PUT
52ecbf9 Add a chunks_finished to BaseDiskFileWriter
84b85f0 s3api: Include '-' in multipart ETags
ebcc95a added some bootstrapping info to the top of the SAIO doc
ce257b3 DiskFile(Writer) refactor cleanups
a42c1da DiskFile(Writer) refactor cleanup for unused use_linkat
33c7650 Add writer method to DiskFile
8bc1434 Import quote directly from swift.common.utils
7914826 Let error messages to be normal strings again
560db71 Drop neutron jobs from our gate
2ae00de Add py36 test job
77734bc Disable the docs badge saying the docs aren't accurate
b4be37c Remove post_as_copy deprecated warning from copy middleware
b478a07 Add IPv6 DSVM tests
8cab628 remove legacy-swift-dsvm-functional jobs
1f29508 Allow for backwards compatibility with swift3.
125bb87 Remove install-guide-jobs
d562659 Tighten up pep8 irrelevant-files
6f00d42 bulk: Use make_subrequest to make subrequests
85f4f94 Factor out a proper BaseKeyMaster class
1d65219 Move legacy-swift-dsvm-functional job in-tree
1e800fc switch documentation job to new PTI
90e260e import zuul job settings from project-config
9167e95 Imported Translations from Zanata
df954e2 py3: port bufferedhttp (hopefully)
673fda7 Configure diskfile per storage policy
e547448 Fix warning while building docs
04d3d68 minor updates to changelog and release notes
ae9536e Update reno for stable/rocky
dd313b8 Reference legacy-tempest-dsvm-neutron-full and neutron-grenade in-tree
2ddc979 Add irrelevant-files lists for in-tree check/gate jobs
6994a2e ratelimit: ignore requests with invalid API versions
b2fbc74 Clean up comment about 404s leaking out during COPYs
e9d0a7a Replacing six.iter/dict.iter usages
9295ba8 Add validation for sorting_method values
51622fd Pass empty etag override values through encrypter
Diffstat (except docs and test files)
-------------------------------------
.mailmap | 1 +
.zuul.yaml | 281 ++++++-
AUTHORS | 13 +-
CHANGELOG | 108 ++-
CONTRIBUTING.rst | 2 +-
api-ref/source/storage-container-services.inc | 15 +-
api-ref/source/storage-object-services.inc | 17 +-
bindep.txt | 4 +-
etc/account-server.conf-sample | 4 +
etc/container-server.conf-sample | 7 +
etc/object-server.conf-sample | 1 +
etc/proxy-server.conf-sample | 19 +-
etc/swift.conf-sample | 7 +
.../notes/2_10_0_release-666a76f4975657a5.yaml | 6 +-
.../notes/2_19_0_release-3e6ee3e6a1fcc6bb.yaml | 6 +-
.../notes/2_20_0_release-7b090a5f4bd916e4.yaml | 116 +++
releasenotes/source/index.rst | 2 +
.../locale/ko_KR/LC_MESSAGES/releasenotes.po | 75 ++
releasenotes/source/rocky.rst | 6 +
setup.cfg | 5 +-
swift/__init__.py | 35 +
swift/account/backend.py | 19 +-
swift/account/reaper.py | 13 +-
swift/account/utils.py | 2 +-
swift/cli/manage_shard_ranges.py | 2 +-
swift/common/bufferedhttp.py | 56 +-
swift/common/constraints.py | 38 +-
swift/common/db.py | 34 +-
swift/common/db_replicator.py | 14 +-
swift/common/direct_client.py | 55 +-
swift/common/internal_client.py | 28 +-
swift/common/memcached.py | 4 +-
swift/common/middleware/bulk.py | 52 +-
swift/common/middleware/copy.py | 20 +-
swift/common/middleware/crypto/crypto_utils.py | 8 +-
swift/common/middleware/crypto/decrypter.py | 123 ++--
swift/common/middleware/crypto/encrypter.py | 17 +-
swift/common/middleware/crypto/keymaster.py | 216 +++---
swift/common/middleware/crypto/kmip_keymaster.py | 19 +-
swift/common/middleware/crypto/kms_keymaster.py | 56 +-
swift/common/middleware/listing_formats.py | 30 +-
swift/common/middleware/ratelimit.py | 3 +
swift/common/middleware/s3api/acl_handlers.py | 2 +-
.../common/middleware/s3api/controllers/bucket.py | 125 +++-
.../middleware/s3api/controllers/location.py | 2 +-
.../middleware/s3api/controllers/multi_delete.py | 50 +-
.../middleware/s3api/controllers/multi_upload.py | 173 +++--
swift/common/middleware/s3api/controllers/obj.py | 21 +-
swift/common/middleware/s3api/etree.py | 16 +-
swift/common/middleware/s3api/s3api.py | 92 ++-
swift/common/middleware/s3api/s3request.py | 160 +++-
swift/common/middleware/s3api/s3response.py | 47 +-
swift/common/middleware/s3api/s3token.py | 182 +++--
swift/common/middleware/symlink.py | 4 +-
swift/common/middleware/tempauth.py | 28 +-
swift/common/middleware/x_profile/html_viewer.py | 10 +-
swift/common/middleware/xprofile.py | 16 +-
swift/common/request_helpers.py | 6 +-
swift/common/ring/ring.py | 2 +-
swift/common/storage_policy.py | 39 +-
swift/common/swob.py | 8 +-
swift/common/utils.py | 297 ++++++--
swift/common/wsgi.py | 10 +-
swift/container/backend.py | 22 +-
swift/container/reconciler.py | 8 +-
swift/container/replicator.py | 5 +-
swift/container/server.py | 9 +-
swift/container/sharder.py | 21 +-
swift/locale/en_GB/LC_MESSAGES/swift.po | 10 +-
swift/locale/es/LC_MESSAGES/swift.po | 6 +-
swift/locale/ja/LC_MESSAGES/swift.po | 6 +-
swift/obj/diskfile.py | 234 +++---
swift/obj/mem_diskfile.py | 43 +-
swift/obj/reconstructor.py | 18 +-
swift/obj/replicator.py | 7 +-
swift/obj/server.py | 393 ++++++----
swift/obj/ssync_receiver.py | 4 +-
swift/obj/ssync_sender.py | 292 ++++----
swift/proxy/controllers/base.py | 18 +-
swift/proxy/controllers/container.py | 2 +-
swift/proxy/controllers/obj.py | 80 +-
swift/proxy/server.py | 22 +-
test/functional/__init__.py | 80 +-
test/functional/s3api/s3_test_client.py | 7 +-
test/functional/s3api/test_acl.py | 19 +-
test/functional/s3api/test_bucket.py | 215 +++---
test/functional/s3api/test_multi_delete.py | 21 +-
test/functional/s3api/test_multi_upload.py | 85 ++-
test/functional/s3api/test_object.py | 29 +
test/functional/swift_test_client.py | 19 +-
test/functional/test_access_control.py | 4 +-
test/functional/test_slo.py | 55 +-
test/functional/test_symlink.py | 4 +
test/functional/test_tempurl.py | 41 +-
test/functional/test_versioned_writes.py | 112 +--
test/probe/test_signals.py | 18 +-
test/sample.conf | 8 +-
test/unit/__init__.py | 18 +-
test/unit/account/test_backend.py | 37 +-
test/unit/account/test_reaper.py | 6 +-
test/unit/cli/test_relinker.py | 21 +-
test/unit/cli/test_ringcomposer.py | 2 +-
.../common/middleware/crypto/crypto_helpers.py | 14 +-
.../common/middleware/crypto/test_crypto_utils.py | 199 ++---
.../common/middleware/crypto/test_decrypter.py | 272 +++----
.../common/middleware/crypto/test_encrypter.py | 201 +++--
.../common/middleware/crypto/test_encryption.py | 59 +-
.../common/middleware/crypto/test_keymaster.py | 144 +++-
.../middleware/crypto/test_kmip_keymaster.py | 74 +-
.../common/middleware/crypto/test_kms_keymaster.py | 73 +-
test/unit/common/middleware/helpers.py | 8 +-
test/unit/common/middleware/s3api/__init__.py | 15 +-
test/unit/common/middleware/s3api/helpers.py | 15 +-
test/unit/common/middleware/s3api/test_acl.py | 4 +-
test/unit/common/middleware/s3api/test_bucket.py | 209 ++++--
.../common/middleware/s3api/test_multi_delete.py | 89 ++-
.../common/middleware/s3api/test_multi_upload.py | 241 +++++-
test/unit/common/middleware/s3api/test_obj.py | 114 ++-
test/unit/common/middleware/s3api/test_s3api.py | 285 ++++++--
.../unit/common/middleware/s3api/test_s3request.py | 83 ++-
.../common/middleware/s3api/test_s3response.py | 7 +-
test/unit/common/middleware/s3api/test_s3token.py | 136 ++++
test/unit/common/middleware/s3api/test_service.py | 5 +-
test/unit/common/middleware/test_copy.py | 49 --
test/unit/common/middleware/test_list_endpoints.py | 31 +-
.../unit/common/middleware/test_listing_formats.py | 84 +--
test/unit/common/middleware/test_memcache.py | 2 +-
test/unit/common/middleware/test_ratelimit.py | 48 +-
test/unit/common/middleware/test_tempauth.py | 97 ++-
test/unit/common/middleware/test_xprofile.py | 6 +-
test/unit/common/test_bufferedhttp.py | 37 +-
test/unit/common/test_daemon.py | 4 +-
test/unit/common/test_db.py | 58 +-
test/unit/common/test_db_replicator.py | 8 +-
test/unit/common/test_direct_client.py | 264 +++----
test/unit/common/test_internal_client.py | 206 +++---
test/unit/common/test_request_helpers.py | 57 +-
test/unit/common/test_storage_policy.py | 86 ++-
test/unit/common/test_utils.py | 809 ++++++++++++++-------
test/unit/common/test_wsgi.py | 35 +-
test/unit/container/test_replicator.py | 34 +-
test/unit/container/test_sharder.py | 135 +++-
test/unit/container/test_sync.py | 4 +-
test/unit/container/test_updater.py | 4 +-
test/unit/helpers.py | 42 +-
test/unit/obj/test_auditor.py | 2 +-
test/unit/obj/test_diskfile.py | 215 ++++--
test/unit/obj/test_reconstructor.py | 73 +-
test/unit/obj/test_replicator.py | 4 +-
test/unit/obj/test_server.py | 25 +-
test/unit/obj/test_ssync.py | 13 +-
test/unit/obj/test_ssync_receiver.py | 59 +-
test/unit/obj/test_ssync_sender.py | 487 ++++++-------
test/unit/obj/test_updater.py | 4 +-
test/unit/proxy/controllers/test_info.py | 10 +-
test/unit/proxy/controllers/test_obj.py | 11 +
test/unit/proxy/test_server.py | 88 ++-
tools/playbooks/common/cover-post.yaml | 3 +
tools/playbooks/common/install_dependencies.yaml | 30 +
tools/playbooks/dsvm/post.yaml | 4 +
tools/playbooks/dsvm/pre.yaml | 9 +
tools/playbooks/dsvm/run.yaml | 11 +
tools/playbooks/multinode_setup/common_config.yaml | 75 ++
.../multinode_setup/configure_loopback.yaml | 64 ++
tools/playbooks/multinode_setup/make_rings.yaml | 126 ++++
tools/playbooks/multinode_setup/pre.yaml | 8 +
tools/playbooks/multinode_setup/run.yaml | 42 ++
.../templates/make_multinode_rings.j2 | 38 +
.../multinode_setup/templates/rc.local.j2 | 8 +
.../multinode_setup/templates/test.conf.j2 | 122 ++++
.../install_dependencies.yaml | 24 -
.../saio_single_node_setup/setup_saio.yaml | 2 +-
tox.ini | 33 +-
200 files changed, 8007 insertions(+), 3563 deletions(-)
More information about the Release-announce
mailing list