[release-announce] swift 2.19.0 (rocky)
no-reply at openstack.org
no-reply at openstack.org
Wed Aug 22 09:38:06 UTC 2018
We are satisfied to announce the release of:
swift 2.19.0: OpenStack Object Storage
This release is part of the rocky release series.
Download the package from:
https://tarballs.openstack.org/swift/
For more details, please see below.
2.19.0
^^^^^^
New Features
* TempURLs now support IP range restrictions. Please see
https://docs.openstack.org/swift/latest/middleware.html#client-usage
for more information on how to use this additional restriction.
* Add support for multiple root encryption secrets for the trivial
and KMIP keymasters. This allows operators to rotate encryption keys
over time without needing to re-encrypt all existing data in the
cluster. Please see the included sample config files for
instructions on how to multiple encryption keys.
* The object updater now supports two configuration settings:
"concurrency" and "updater_workers". The latter controls how many
worker processes are spawned, while the former controls how many
concurrent container updates are performed by each worker process.
This should speed the processing of async_pendings.
On upgrade, a node configured with concurrency=N will still handle
async updates N-at-a-time, but will do so using only one process
instead of N.
If you have a config file like this:
[object-updater] concurrency = <N>
and you want to take advantage of faster updates, then do this:
[object-updater] concurrency = 8 # the default; you can omit
this line updater_workers = <N>
If you want updates to be processed exactly as before, do this:
[object-updater] concurrency = 1 updater_workers = <N>
* When listing objects in a container in json format, static large
objects (SLOs) will now include an additional new "slo_etag" key
that matches the etag returned when requesting the SLO. The existing
"hash" key remains unchanged as the MD5 of the SLO manifest. Text
and XML listings are unaffected by this change.
* Log deprecation warnings for "run_pause". This setting was
deprecated in Swift 2.4.0 and is replaced by "interval". It may be
removed in a future release.
* Object reconstructor logs are now prefixed with information about
the specific worker process logging the message. This makes reading
the logs and understanding the messages much simpler.
* Lower bounds of dependencies have been updated to reflect what is
actually tested.
* SSYNC replication mode now removes as much of the directory
structure as possible as soon at it observes that the directory is
empty. This reduces the work needed for subsequent replication
passes.
* The container-updater now reports zero objects and bytes used for
child DBs in sharded containers. This prevents double-counting in
utilization reports.
* Add fallocate_reserve to account and container servers. This
allows disks shared between account/container and object rings to
avoid getting 100% full. The default value of 1% matches the
existing default on object servers.
* Added an experimental "swift-ring-composer" CLI tool to build
composite rings.
* Added an optional "read_only" middleware to make an entire cluster
or individual accounts read only.
* Fixed a bug where zero-byte PUTs would not work properly with "If-
None-Match: *" conditional requests.
* ACLs now work with unicode in user/account names.
* COPY now works with unicode account names.
* Improved S3 API compatibility.
* Lock timeouts in the container updater are now logged at INFO
level, not ERROR.
* Various other minor bug fixes and improvements.
Changes in swift 2.18.0..2.19.0
-------------------------------
9bd84be authors/changelog for 2.19.0 release
c63d5c0 Fix tempurl's example code to be in the code-block
7ca1a67 Add debugging info to SignatureDoesNotMatch responses
0dc1b62 Multi-key KMIP keymaster
2722e49 Add support for multiple root encryption secrets
fc04dc1 Attempt to fix pep8
a7c5ca0 Fix locking in swift-recon-cron
51b885b s3_acl: Require swift_owner authz to create buckets
bd640cd Fix the deletion of non-existent keys
5dca610 Fix up test_static_web_pretend_to_be_giant_json
0887f09 Update saio sample config files
03c1d61 Imported Translations from Zanata
7895718 Fix keymmaster_conf_section typo
dcbdcb8 Disallow uploads with x-amz-tagging headers set
b957940 Add probetests to gate jobs
79539ea Disallow aws-chunked uploads
86d78f6 imported some docs from the old user-guide
6a2bd0f improve prefix/delimiter api docs
6f920fd Add some reno artifacts to .gitignore
7fd5a12 Stop holding on to sys.exc_info tuples quite so much
3c92e3c Move keymaster_config_path parsing out of _get_root_secret
2f9bd85 Define keymaster log routes on the class
ba8447e add lower-constraints job
84758d9 make probe tests voting in the gate
c3d83ba Remove some unnecessary SkipTests
6a03990 Remove contentdir hack
60f1d34 Lower the required version of ipaddress
8e651a2 Add fallocate_reserve to account and container servers.
0cd42a2 Check other params preserved when slo_etag is extracted
f8b9c24 Add unittest for slo_etag
fa4c30d Include s3api schemas in sdists
c4c98eb Include SLO ETag in container updates
4409da6 Update doc building instructions in Development Guidelines
2f4f2dd Add Alpine Linux suuprt to bindep
9a7b46e swift-ring-builder shows hint about composite builder file
7bacd53 Be consistent about capitalizing Swift in the README
f27f1ae Make docs link consistent
26b20ee IP Range restrictions in temp urls
1951dc7 Add keymaster to fetch root secret from KMIP service
91a8cd2 PUT+POST: break out putter specific test classes
8bfb97a Content-Length enforcement fixups
783c7f6 py3: port proxy/controllers/info.py
d03fc9b swob: Stop auto-encoding unicode bodies
600521c Describe separate keymaster config file in docs
108304e Improve README doc building instructions
8918615 Update README related to documentation building
7ae2cc4 Address some review comments
158d4c2 swob.Match: remove quotes when checking __contains__
efcf7e6 swob: Fix up some WSGI string business
01fb3ef Log unexpected errors when trying to clean up empty dirs
306f9a1 py3: port domain_remap tests
081ff36 py3: port crossdomain
50ea4e1 py3: port healthcheck
0edfb87 reorganized existing ansible playbooks
1fa4b9e Remove unnecessary loop in test_keymaster.py
f0e4685 fix misspelling of 'server'
5652dec container-updater: Always report zero objects/bytes used for shards
a5a6a27 rename swob.str_from_wsgi -> swob.wsgi_to_str
3920b88 py3: port catch_errors
e0b4eb2 py3: fix test_proxy_logging.py
5b79ce9 Reverse shard ranges later
1318bac py36: Fix test_get_logger_sysloghandler_plumbing
ba0b3c3 Add clarifying comment about surrogate check
f192f51 Have check_drive raise ValueError on errors
2b19c26 py3: Port common/swob
0694e51 py3: Port common/constraints
2637fec Add py36 testenv
f21bbe0 specify test selector with py3 packages in bindep
a59c5e3 Resolve TODO's in test/probe/test_sharder.py
554c07a Fix the example webhook middleware
a1cb919 Use valid eventlet logger method
642d874 Fix test_wsgi.py tests failing with eventlet <=0.18.4
4a0afa9 Enforce Content-Length in catch_errors
b08355e Rename test_except.py -> test_catch_errors.py
ac69a71 s3api: Stop debug-logging the entire wsgi environment
049dc0c Blacklist eventlet 0.23.0
572b6be Set lower bounds on all requirements and test-requirements
8e89271 Tighten up staticweb redirect test
b39a88c func tests: Rename storage_url to storage_path
7042d2c Use path_qs instead of reinventing it
f6a436a Only try to fetch or sync shard ranges if the remote supports sharding
a718cb3 py3: fix up some NameErrors
6b626f2 Experimental swift-ring-composer CLI to build composite rings
026f706 rectify 'a integer number' to 'an integer number'
2a2ce04 rectify 'a invalid version' to 'an invalid version'
d5c532a object-updater: add concurrent updates
a04234d Functional tests for domain_remap middleware
65e1de2 Fix shard_max_row in ContainerBroker.get_replication_info()
2ee8363 rectify 'a expirting object' to 'an expiring object'
f2295a6 Fix typo in sharding docs
dc8d1c9 Get rid of tpool_reraise
b3ce3c9 func tests: Drop clone_conn arg to Connection
fcd7a05 Fix up insecure behavior for functional tests
0318e7c Rearrange some error handling to reduce indent
beb014b Read-only middleware fixups
e726473 Fix typos replace a OpenStack bucket with an OpenStack bucket
284da56 Fix grammar errors replace a object with an object
8b44c9d Allow domain remap storage_domain to be configured for func tests
caa3c67 functests: Clean up objects better
c88d4d7 Make COPY work with unicode account names
9d0162a Make ACLs work with Unicode in user/account names
7cec5a8 tempauth: Send back url-encoded account names
9ef2a82 copy: Stop hanging per-request data on middleware instance
1424ff7 Refactor object updater's async-pending finder
ecf4755 Make final stats dump after reconstructor runs once
76344e5 Remove redundant elif in tempauth
2044673 Improve user logging
5acefa3 made changelog section breaks more consistent
e748ef4 Verify diff stat is unchanged when syncing only shard ranges
5078bb1 Added S3 compatibility information to docs
bb9e427 Add detail to container server GET docstring
7d76017 Fix marker and end_marker descriptions in API ref
076f20b removed specific reference to two core reviewers
47aff72 Log updater stats once per pass, not per policy
fb3b43f Added PyPI validation check for the readme.
5d601b7 Adds read_only middleware
a954f59 container-updater: log LockTimeout exceptions at INFO, not ERROR
553bef0 Remove unhelpful log message in copy middleware
31623f7 Catch and log errors from process_container, then keep updating
7b5c30d Update SERVER_ADDR/SERVER_PORT from PROXY protocol
854db51 Fix common/test_utils.py on Python 3.5.4+
844bba8 Remove some Python 2.6 compatibility code in ring
7a76778 Use X-Timestamp when checking object expiration
0702d4d Stop overriding _in_sync in container replicator
a19548b Remove empty part dirs during ssync replication
26538d3 Make multiprocess reconstructor's logs more readable.
f8419bf Improve path handling in proxy_logging
748b29e Make If-None-Match:* work properly with 0-byte PUTs
e0d1869 Fix suffix-byte-range responses for zero-byte EC objects.
c44d047 Create policy_stat table in auditor if missing
b0ab7eb Log deprecation warnings for run_pause
f789af6 display swift services in apache2
b29eb13 Add assertion to test_consolidate_hashes_raises_exception
Diffstat (except docs and test files)
-------------------------------------
.gitignore | 2 +
.zuul.yaml | 14 +-
AUTHORS | 8 +
CHANGELOG | 130 +-
CONTRIBUTING.rst | 2 +-
MANIFEST.in | 1 +
README.rst | 17 +-
api-ref/source/parameters.yaml | 25 +-
bandit.yaml | 2 +-
bin/swift-recon-cron | 26 +-
bin/swift-ring-composer | 22 +
bindep.txt | 21 +-
.../pseudo-hierarchical-folders-directories.rst | 155 ++
etc/account-server.conf-sample | 7 +
etc/container-server.conf-sample | 7 +
etc/keymaster.conf-sample | 20 +
etc/object-server.conf-sample | 9 +-
etc/proxy-server.conf-sample | 40 +
examples/apache2/account-server.template | 2 +-
examples/apache2/container-server.template | 2 +-
examples/apache2/object-server.template | 2 +-
examples/apache2/proxy-server.template | 2 +-
lower-constraints.txt | 89 +
.../install_dependencies.yaml | 50 -
playbooks/saio_single_node_setup/make_rings.yaml | 29 -
playbooks/saio_single_node_setup/setup_saio.yaml | 174 --
.../saio_single_node_setup/templates/rc.local.j2 | 8 -
.../notes/2_19_0_release-3e6ee3e6a1fcc6bb.yaml | 103 ++
.../locale/en_GB/LC_MESSAGES/releasenotes.po | 325 +++-
.../locale/ko_KR/LC_MESSAGES/releasenotes.po | 75 -
requirements.txt | 5 +-
setup.cfg | 6 +
swift/account/backend.py | 7 +-
swift/account/reaper.py | 7 +-
swift/account/server.py | 40 +-
swift/cli/ringbuilder.py | 9 +-
swift/cli/ringcomposer.py | 183 ++
swift/common/constraints.py | 110 +-
swift/common/db_replicator.py | 42 +-
swift/common/exceptions.py | 4 +
swift/common/internal_client.py | 2 +-
swift/common/middleware/acl.py | 11 +-
swift/common/middleware/catch_errors.py | 70 +-
swift/common/middleware/copy.py | 40 +-
swift/common/middleware/crossdomain.py | 2 +-
swift/common/middleware/crypto/crypto_utils.py | 26 +-
swift/common/middleware/crypto/decrypter.py | 150 +-
swift/common/middleware/crypto/encrypter.py | 11 +-
swift/common/middleware/crypto/keymaster.py | 197 +-
swift/common/middleware/crypto/kmip_keymaster.py | 158 ++
swift/common/middleware/crypto/kms_keymaster.py | 27 +-
swift/common/middleware/dlo.py | 2 +
swift/common/middleware/domain_remap.py | 2 +-
swift/common/middleware/gatekeeper.py | 11 +-
swift/common/middleware/healthcheck.py | 4 +-
swift/common/middleware/list_endpoints.py | 2 +-
swift/common/middleware/proxy_logging.py | 15 +-
swift/common/middleware/read_only.py | 123 ++
swift/common/middleware/recon.py | 22 +-
swift/common/middleware/s3api/acl_handlers.py | 10 +-
.../common/middleware/s3api/controllers/bucket.py | 4 +-
.../middleware/s3api/controllers/multi_delete.py | 5 +-
.../middleware/s3api/controllers/multi_upload.py | 6 +-
swift/common/middleware/s3api/controllers/obj.py | 10 +-
swift/common/middleware/s3api/etree.py | 9 +-
swift/common/middleware/s3api/s3api.py | 1 -
swift/common/middleware/s3api/s3request.py | 37 +-
swift/common/middleware/s3api/utils.py | 8 +-
swift/common/middleware/slo.py | 92 +-
swift/common/middleware/tempauth.py | 23 +-
swift/common/middleware/tempurl.py | 101 +-
swift/common/ring/ring.py | 5 +-
swift/common/swob.py | 133 +-
swift/common/utils.py | 78 +-
swift/common/wsgi.py | 69 +-
swift/container/backend.py | 27 +-
swift/container/reconciler.py | 8 +
swift/container/replicator.py | 25 +-
swift/container/server.py | 108 +-
swift/container/updater.py | 28 +-
swift/locale/de/LC_MESSAGES/swift.po | 18 +-
swift/locale/en_GB/LC_MESSAGES/swift.po | 42 +-
swift/locale/es/LC_MESSAGES/swift.po | 20 +-
swift/locale/fr/LC_MESSAGES/swift.po | 18 +-
swift/locale/it/LC_MESSAGES/swift.po | 18 +-
swift/locale/ja/LC_MESSAGES/swift.po | 22 +-
swift/locale/ko_KR/LC_MESSAGES/swift.po | 16 +-
swift/locale/pt_BR/LC_MESSAGES/swift.po | 16 +-
swift/locale/ru/LC_MESSAGES/swift.po | 18 +-
swift/locale/tr_TR/LC_MESSAGES/swift.po | 16 +-
swift/locale/zh_CN/LC_MESSAGES/swift.po | 16 +-
swift/locale/zh_TW/LC_MESSAGES/swift.po | 16 +-
swift/obj/diskfile.py | 133 +-
swift/obj/mem_diskfile.py | 19 +-
swift/obj/reconstructor.py | 55 +-
swift/obj/replicator.py | 40 +-
swift/obj/server.py | 20 +-
swift/obj/updater.py | 168 +-
swift/proxy/controllers/base.py | 22 +-
swift/proxy/controllers/info.py | 2 +-
swift/proxy/controllers/obj.py | 69 +-
swift/proxy/server.py | 4 +-
test-requirements.txt | 12 +-
test/functional/__init__.py | 14 +-
test/functional/s3api/test_bucket.py | 6 +-
test/functional/s3api/test_object.py | 11 +-
test/functional/swift_test_client.py | 96 +-
test/functional/test_container.py | 6 +-
test/functional/test_domain_remap.py | 177 ++
test/functional/test_object.py | 6 +-
test/functional/test_slo.py | 85 +-
test/functional/test_staticweb.py | 54 +-
test/functional/test_symlink.py | 50 +-
test/functional/test_tempurl.py | 4 -
test/probe/test_object_conditional_requests.py | 79 +
test/probe/test_sharder.py | 134 +-
test/sample.conf | 5 +
test/unit/__init__.py | 27 +-
test/unit/account/test_backend.py | 76 +-
test/unit/account/test_server.py | 86 +
test/unit/cli/test_ringbuilder.py | 50 +-
test/unit/cli/test_ringcomposer.py | 195 ++
.../common/middleware/crypto/crypto_helpers.py | 31 +-
.../common/middleware/crypto/test_crypto_utils.py | 47 +-
.../common/middleware/crypto/test_decrypter.py | 196 +-
.../common/middleware/crypto/test_encrypter.py | 11 +-
.../common/middleware/crypto/test_encryption.py | 39 +-
.../common/middleware/crypto/test_keymaster.py | 251 ++-
.../middleware/crypto/test_kmip_keymaster.py | 270 +++
.../common/middleware/crypto/test_kms_keymaster.py | 68 +-
test/unit/common/middleware/s3api/__init__.py | 11 +-
test/unit/common/middleware/s3api/helpers.py | 16 +-
test/unit/common/middleware/s3api/test_bucket.py | 6 +
.../common/middleware/s3api/test_multi_delete.py | 14 +-
test/unit/common/middleware/s3api/test_obj.py | 38 +-
test/unit/common/middleware/s3api/test_s3api.py | 27 +-
test/unit/common/middleware/s3api/test_service.py | 5 +-
test/unit/common/middleware/test_catch_errors.py | 199 ++
test/unit/common/middleware/test_crossdomain.py | 32 +-
test/unit/common/middleware/test_dlo.py | 15 +
test/unit/common/middleware/test_domain_remap.py | 80 +-
test/unit/common/middleware/test_except.py | 142 --
test/unit/common/middleware/test_gatekeeper.py | 4 +-
test/unit/common/middleware/test_healthcheck.py | 12 +-
.../unit/common/middleware/test_listing_formats.py | 15 +-
test/unit/common/middleware/test_proxy_logging.py | 113 +-
test/unit/common/middleware/test_read_only.py | 237 +++
test/unit/common/middleware/test_slo.py | 55 +-
.../common/middleware/test_subrequest_logging.py | 11 +-
test/unit/common/middleware/test_tempauth.py | 30 +-
test/unit/common/middleware/test_tempurl.py | 214 ++-
test/unit/common/test_constraints.py | 180 +-
test/unit/common/test_db_replicator.py | 7 +-
test/unit/common/test_swob.py | 356 ++--
test/unit/common/test_utils.py | 85 +-
test/unit/common/test_wsgi.py | 98 +-
test/unit/container/test_backend.py | 35 +
test/unit/container/test_replicator.py | 139 +-
test/unit/container/test_server.py | 89 +-
test/unit/container/test_updater.py | 121 +-
test/unit/obj/test_diskfile.py | 180 +-
test/unit/obj/test_reconstructor.py | 221 ++-
test/unit/obj/test_replicator.py | 43 +-
test/unit/obj/test_server.py | 265 +--
test/unit/obj/test_ssync.py | 30 +-
test/unit/obj/test_updater.py | 89 +-
test/unit/proxy/controllers/test_info.py | 10 +-
test/unit/proxy/controllers/test_obj.py | 1900 ++++++++++----------
test/unit/proxy/test_server.py | 20 +-
.../install_dependencies.yaml | 24 +
.../saio_single_node_setup/make_rings.yaml | 29 +
.../saio_single_node_setup/setup_saio.yaml | 174 ++
.../saio_single_node_setup/templates/rc.local.j2 | 8 +
tox.ini | 24 +-
212 files changed, 9420 insertions(+), 3414 deletions(-)
Requirements updates
--------------------
diff --git a/requirements.txt b/requirements.txt
index c115113..df5d26b 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6 +6 @@ dnspython>=1.14.0;python_version=='2.7' # http://www.dnspython.org/LICENSE
-eventlet>=0.17.4 # MIT
+eventlet>=0.17.4,!=0.23.0 # MIT
@@ -10 +10 @@ PasteDeploy>=1.3.3
-lxml
+lxml>=3.4.1
@@ -15,0 +16 @@ cryptography!=2.0,>=1.6 # BSD/Apache-2.0
+ipaddress>=1.0.16;python_version<'3.3' # PSF
diff --git a/test-requirements.txt b/test-requirements.txt
index 5f515bd..cf97918 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -8,2 +8,2 @@ coverage>=3.6 # Apache-2.0
-nose # LGPL
-nosexcover # BSD
+nose>=1.3.7 # LGPL
+nosexcover>=1.0.10 # BSD
@@ -13 +13 @@ mock>=2.0 # BSD
-python-swiftclient
+python-swiftclient>=3.2.0
@@ -16,2 +16,2 @@ reno>=1.8.0 # Apache-2.0
-python-openstackclient
-boto
+python-openstackclient>=3.12.0
+boto>=2.32.1
@@ -23,0 +24,2 @@ bandit>=1.1.0 # Apache-2.0
+
+docutils>=0.11 # OSI-Approved Open Source, Public Domain
More information about the Release-announce
mailing list