We are stoked to announce the release of: swift 2.16.0: OpenStack Object Storage This release is part of the queens release series. Download the package from: https://tarballs.openstack.org/swift/ For more details, please see below. 2.16.0 ^^^^^^ New Features ************ * Add checksum to object extended attributes. * Let clients request heartbeats during SLO PUTs by including the query parameter "heartbeat=on". With heartbeating turned on, the proxy will start its response immediately with 202 Accepted then send a single whitespace character periodically until the request completes. At that point, a final summary chunk will be sent which includes a "Response Status" key indicating success or failure and (if successful) an "Etag" key indicating the Etag of the resulting SLO. * Added support for retrieving the encryption root secret from an external key management system. In practice, this is currently limited to Barbican. * Move listing formatting out to a new proxy middleware named "listing_formats". "listing_formats" should be just right of the first proxy-logging middleware, and left of most other middlewares. If it is not already present, it will be automatically inserted for you. Note: if you have a custom middleware that makes account or container listings, it will only receive listings in JSON format. * Log deprecation warning for "allow_versions" in the container server config. Configure the "versioned_writes" middleware in the proxy server instead. This option will be ignored in a future release. * Replaced "replication_one_per_device" by custom count defined by "replication_concurrency_per_device". The original config value is deprecated, but continues to function for now. If both values are defined, the old "replication_one_per_device" is ignored. * Fixed a rare issue where multiple backend timeouts could result in bad data being returned to the client. * Cleaned up logged tracebacks when talking to memcached servers. * Account and container replication stats logs now include "remote_merges", the number of times a whole database was sent to another node. * Respond 400 Bad Request when Accept headers fail to parse instead of returning 406 Not Acceptable. * The "domain_remap" middleware now supports the "mangle_client_paths" option. Its default "false" value changes "domain_remap" parsing to stop stripping the "path_root" value from URL paths. If users depend on this path mangling, operators should set "mangle_client_paths" to "True" before upgrading. * Remove "swift-temp-url" script. The functionality has been in swiftclient for a long time and this script has been deprecated since 2.10.0. * Removed all "post_as_copy" related code and configs. The option has been deprecated since 2.13.0. * Fixed XML responses (eg on bulk extractions and SLO upload failures) to be more correct. The enclosing "delete" tag was removed where it doesn't make sense and replaced with "extract" or "upload" depending on the context. * Static Large Object (SLO) manifest may now (again) have zero-byte last segments. * Fixed an issue where background consistency daemon child processes would deadlock waiting on the same file descriptor. * Removed a race condition where a POST to an SLO could modify the X -Static-Large-Object metadata. * Accept a trade off of dispersion for balance in the ring builder that will result in getting to balanced rings much more quickly in some cases. * Fixed using "swift-ring-builder set_weight" with more than one device. * When requesting objects, return 404 if a tombstone is found and is newer than any data found. Previous behavior was to return stale data. Other Notes *********** * Various other minor bug fixes and improvements. Changes in swift 2.15.1..2.16.0 ------------------------------- 1ff3f1e authors changelog for 2.16.0 5a06e3d No longer import nose 53d92c1 Increase connection timeout for object controller tests 728b4ba Add checksum to object extended attributes 77a8a44 Let clients request heartbeats during SLO PUTs 92705bb Fix indent in overview_policies.rst feee399 Use check_drive consistently 29e9ae1 Make xml responses less insane 449d83f Doc uses alias instead of aliases 178d7f3 Added the man page for container-reconciler.conf 83be309 Fix a small typo 8d88209 Return 404 on a GET if tombstone is newer 93e0287 Require lock_path limit to be a positive int e0244dd Cleanup lock_path and replication_concurrency_per_device tests e199192 Replace replication_one_per_device by custom count 9c97c80 Clean up a couple hand-rolled mocks. 9b2779b Clean up memcache tests 0bca0d4 Be more tolerant of exception messages from sqlite e001c02 Stop logging tracebacks on bad xLOs 356b110 Remove swift-temp-url man page. 1aecd1d tighten up drop_privileges unit tests e6cf9b4 Fix some spelling in a docstring 646f750 Quiet test output when running test_utils.py in isolation 0f91c86 Drop group comparison from drop_privileges test 5438fe7 Clean up some leftover Python 2.6-isms. 250da37 Remove swift-temp-url script c118059 Respond 400 Bad Request when Accept headers fail to parse 03e8ab7 functests: don't crash if no second account eaea0c4 Skip cross-account-copy functest if only one account 24188be Remove some leftover threadpool cruft. 1d67485 Move all monkey patching to one function dc8da5b Use "poll" or "selects" Eventlet hub for all Swift daemons. 407f539 Imported Translations from Zanata 45ca39f add mangle_client_paths to example config 37f23b0 Allow SLOs to have zero-byte last segments. 94bac4a domain_remap: stop mangling client-provided paths a4a5494 test account autocreate listing format 8b7f152 Add example to container-sync-realms.conf.5 man page 8163311 Added the man page for container-sync-realms.conf 747b9d9 Fix swift-ring-builder set_weight with >1 device 839c130 Stop clearing params for account_autocreate responses 4665c17 Clean up SLO tests and docs 79905ae Replace SOSO auth prefix in examples with more-standard AUTH 4716d3d swift-account-audit: compare each etag to the hash from container 93fc9d2 Add cautionary note re delay_reaping in account-server.conf-sample c6aea4b Fix intermittent failure in test_x_delete_after 348bd83 Respect co-builder partition moves when min_part_hours is zero 5c76b9e Add concurrent_gets to proxy.conf man page b4f08b6 Fix functest for IPv6 endpoints 36a843b Preserve X-Static-Large-Object from .data file after POST 53ab6f2 Assert memcached connection error is logged c662e5f Add account_autocreate=true to internal-client.conf-sample e501ac7 Fix memcached exception out of range stacktrace 2321966 Accept a trade off of dispersion for balance 69a90dc Remove reference to EC being in beta 64d2407 Follow-up test fixup cc17c99 Stop reloading swift.common.utils in test_daemon 8556b06 Add test to ensure cache cleared after container PUT 6b19ca7 proxy: Use the right ranges when going to multiple object servers d6fcf74 Make gate keeper to save relative location header path 1e79f82 Remove all post_as_copy related code and configes 4806434 Move listing formatting out to proxy middleware df00122 doc migration: update the doc link address[2/3] 5622c1f Test placeholder for dispersion_report module 6305993 Correct the unused doc link address cbddec3 Add bin/swift-dispersion-report b77de53 Make swift-dispersion-report importable f90ba1a Use swift3's check_signature function 8f84338 Add assertion about last-modified to object post test 4ddd4c8 Update ubuntu version to be correct target 079be1d Mock RingBuilder.rebalance when testing ringbuilder cli warnings f70feb3 swob: Raise RuntimeError instead of IndexError c6d00fe api-ref: Fix container PUT response codes 9fd9141 Unify 'Content-Type' header key name in API reference a2f557e Add explanation about Content-Length header limit with DLO to docs 275da4c Fix bytes and name API reference of Container GET 244d7de Delay cache invalidation during container creation 6998d80 doc migration: update the doc link address[1/3] ddec5f5 Update Container-Sync Docs for internal_client.conf 03c54d5 Remove vestigate HUDSON_PUBLISH_DOCS reference 163fb4d Always require device dir for containers 0e51ac0 Cleanup test tempdirs in tearDown 8ed6e66 Stop clearing req.acl in DLO 9f5aa3b api-ref: update docs links fb07863 Small readability change to test_keystoneauth e109c78 Add probe test for ssync of unexpired metadata to an expired object 2e0ca54 Make X-Backend-Replication consistent for HEAD 87eaaeb Fix DirectClientExceptions with utf-8 encoded paths 00ca1ce Tolerate swiftclient *not* mutatinng args 412898a Imported Translations from Zanata 87340e5 Use more specific asserts in test/unit/account tests 7392953 Socket errors don't warrant tracebacks when talking to memcached 63ca3a7 Drop reconstructor stats when worker has no devices 26a2361 Update reno for stable/pike 72ed76d Make test assertion more holistic 9c3c388 Improve domain_remap docs 15e339d Fix more X-Delete-At timing issues 0fead33 Add OpenSuse SAIO build instructions a959d24 Document keystone role element in container ACL 77bd74d Retrieve encryption root secret from Barbican 808ff4f Ignore all auditor_status_*.json files in reconstructor 849d204 domain_remap: be more careful about client-path mangling 5e673a5 Log deprecation warning for allow_versions 8e59dfb Log remote_merges during DB replication c6b9195 More assertion cleanup 3fa742f Add sanity checks to strip_*meta_prefix functions Diffstat (except docs and test files) ------------------------------------- AUTHORS | 6 + CHANGELOG | 132 +++- CONTRIBUTING.rst | 4 +- README.rst | 26 +- api-ref/source/parameters.yaml | 75 +- api-ref/source/storage-account-services.inc | 8 +- api-ref/source/storage-container-services.inc | 11 +- api-ref/source/storage-object-services.inc | 4 +- bin/swift-account-audit | 14 +- bin/swift-dispersion-report | 397 +---------- bin/swift-temp-url | 31 - bindep.txt | 9 +- .../install/storage-install-ubuntu-debian.rst | 2 +- etc/account-server.conf-sample | 6 +- etc/internal-client.conf-sample | 1 + etc/keymaster.conf-sample | 77 ++ etc/object-server.conf-sample | 11 +- etc/proxy-server.conf-sample | 45 +- .../notes/2_16_0_release-d48cb9b2629df8ab.yaml | 107 +++ releasenotes/source/index.rst | 2 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 67 ++ .../source/locale/ja/LC_MESSAGES/releasenotes.po | 42 +- .../locale/ko_KR/LC_MESSAGES/releasenotes.po | 75 ++ releasenotes/source/pike.rst | 6 + requirements.txt | 2 +- setup.cfg | 8 +- swift/account/reaper.py | 8 +- swift/account/server.py | 21 +- swift/account/utils.py | 56 +- swift/cli/dispersion_report.py | 432 ++++++++++++ swift/cli/ringbuilder.py | 5 +- swift/common/constraints.py | 35 +- swift/common/daemon.py | 3 + swift/common/db.py | 4 +- swift/common/db_replicator.py | 13 +- swift/common/direct_client.py | 17 +- swift/common/exceptions.py | 4 + swift/common/internal_client.py | 8 +- swift/common/manager.py | 3 +- swift/common/memcached.py | 24 +- swift/common/middleware/bulk.py | 67 +- swift/common/middleware/copy.py | 68 +- swift/common/middleware/crypto/decrypter.py | 36 +- swift/common/middleware/crypto/keymaster.py | 59 +- swift/common/middleware/crypto/kms_keymaster.py | 114 +++ swift/common/middleware/dlo.py | 4 +- swift/common/middleware/domain_remap.py | 117 ++- swift/common/middleware/gatekeeper.py | 21 + swift/common/middleware/listing_formats.py | 214 ++++++ swift/common/middleware/recon.py | 4 +- swift/common/middleware/slo.py | 433 +++++++----- swift/common/middleware/staticweb.py | 6 +- swift/common/middleware/tempauth.py | 23 +- swift/common/middleware/versioned_writes.py | 6 +- swift/common/request_helpers.py | 50 +- swift/common/ring/builder.py | 43 +- swift/common/ring/composite_builder.py | 16 +- swift/common/storage_policy.py | 8 +- swift/common/swob.py | 8 +- swift/common/utils.py | 100 ++- swift/common/wsgi.py | 7 +- swift/container/server.py | 71 +- swift/container/sync.py | 1 + swift/container/updater.py | 21 +- swift/locale/de/LC_MESSAGES/swift.po | 20 +- swift/locale/es/LC_MESSAGES/swift.po | 20 +- swift/locale/fr/LC_MESSAGES/swift.po | 20 +- swift/locale/it/LC_MESSAGES/swift.po | 20 +- swift/locale/ja/LC_MESSAGES/swift.po | 20 +- swift/locale/ko_KR/LC_MESSAGES/swift.po | 20 +- swift/locale/pt_BR/LC_MESSAGES/swift.po | 20 +- swift/locale/ru/LC_MESSAGES/swift.po | 20 +- swift/locale/tr_TR/LC_MESSAGES/swift.po | 20 +- swift/locale/zh_CN/LC_MESSAGES/swift.po | 20 +- swift/locale/zh_TW/LC_MESSAGES/swift.po | 20 +- swift/obj/auditor.py | 4 +- swift/obj/diskfile.py | 128 ++-- swift/obj/mem_diskfile.py | 5 +- swift/obj/reconstructor.py | 17 +- swift/obj/replicator.py | 23 +- swift/obj/server.py | 14 +- swift/obj/updater.py | 14 +- swift/proxy/controllers/account.py | 10 +- swift/proxy/controllers/base.py | 82 ++- swift/proxy/controllers/container.py | 7 +- swift/proxy/server.py | 11 +- test/functional/__init__.py | 28 +- test/functional/mock_swift_key_manager.py | 59 ++ test/functional/swift_test_client.py | 33 +- test/functional/test_access_control.py | 11 +- test/functional/test_account.py | 3 + test/functional/test_container.py | 12 + test/functional/test_object.py | 11 +- test/functional/test_slo.py | 95 +++ test/functional/test_versioned_writes.py | 5 + test/probe/common.py | 32 +- test/probe/test_container_merge_policy_index.py | 11 +- test/probe/test_container_sync.py | 21 +- test/probe/test_object_async_update.py | 12 +- test/probe/test_object_expirer.py | 8 +- test/probe/test_object_handoff.py | 63 ++ test/probe/test_object_metadata_replication.py | 2 +- test/probe/test_reconstructor_rebuild.py | 79 ++- test/probe/test_reconstructor_revert.py | 3 +- test/probe/test_replication_servers_working.py | 5 +- test/unit/__init__.py | 150 ++-- test/unit/account/test_backend.py | 16 +- test/unit/account/test_reaper.py | 14 +- test/unit/account/test_server.py | 113 +-- test/unit/cli/test_dispersion_report.py | 21 + test/unit/cli/test_info.py | 3 +- test/unit/cli/test_relinker.py | 3 +- test/unit/cli/test_ringbuilder.py | 88 ++- .../common/middleware/crypto/test_decrypter.py | 133 ---- .../common/middleware/crypto/test_encryption.py | 12 +- .../common/middleware/crypto/test_keymaster.py | 8 +- .../common/middleware/crypto/test_kms_keymaster.py | 783 +++++++++++++++++++++ test/unit/common/middleware/test_bulk.py | 8 +- test/unit/common/middleware/test_cname_lookup.py | 4 +- test/unit/common/middleware/test_copy.py | 250 +------ test/unit/common/middleware/test_dlo.py | 90 +-- test/unit/common/middleware/test_domain_remap.py | 86 ++- test/unit/common/middleware/test_gatekeeper.py | 37 +- test/unit/common/middleware/test_keystoneauth.py | 2 +- .../unit/common/middleware/test_listing_formats.py | 345 +++++++++ test/unit/common/middleware/test_memcache.py | 103 +-- test/unit/common/middleware/test_proxy_logging.py | 4 +- test/unit/common/middleware/test_recon.py | 22 +- test/unit/common/middleware/test_slo.py | 541 ++++++++++---- test/unit/common/middleware/test_staticweb.py | 61 +- .../common/middleware/test_subrequest_logging.py | 51 +- test/unit/common/middleware/test_tempauth.py | 52 +- test/unit/common/middleware/test_tempurl.py | 82 +-- .../common/middleware/test_versioned_writes.py | 91 +-- test/unit/common/middleware/test_xprofile.py | 20 +- test/unit/common/ring/test_builder.py | 194 ++++- test/unit/common/ring/test_composite_builder.py | 320 ++++++--- test/unit/common/ring/test_ring.py | 14 +- test/unit/common/ring/test_utils.py | 19 +- test/unit/common/test_bufferedhttp.py | 7 +- test/unit/common/test_constraints.py | 60 +- test/unit/common/test_daemon.py | 36 +- test/unit/common/test_db.py | 38 +- test/unit/common/test_db_replicator.py | 226 ++++-- test/unit/common/test_direct_client.py | 20 +- test/unit/common/test_linkat.py | 7 +- test/unit/common/test_manager.py | 35 +- test/unit/common/test_memcached.py | 164 ++++- test/unit/common/test_request_helpers.py | 20 +- test/unit/common/test_splice.py | 5 +- test/unit/common/test_swob.py | 54 +- test/unit/common/test_utils.py | 340 +++++---- test/unit/common/test_wsgi.py | 130 ++-- test/unit/container/test_replicator.py | 10 +- test/unit/container/test_server.py | 254 ++++--- test/unit/container/test_sync_store.py | 2 +- test/unit/container/test_updater.py | 22 +- test/unit/helpers.py | 6 +- test/unit/obj/test_auditor.py | 30 +- test/unit/obj/test_diskfile.py | 362 ++++++---- test/unit/obj/test_reconstructor.py | 188 +++-- test/unit/obj/test_replicator.py | 18 +- test/unit/obj/test_server.py | 139 ++-- test/unit/obj/test_ssync.py | 6 +- test/unit/obj/test_ssync_receiver.py | 29 +- test/unit/obj/test_ssync_sender.py | 4 +- test/unit/obj/test_updater.py | 62 +- test/unit/proxy/controllers/test_base.py | 41 +- test/unit/proxy/controllers/test_container.py | 20 +- test/unit/proxy/controllers/test_obj.py | 33 +- test/unit/proxy/test_server.py | 135 ++-- test/unit/proxy/test_sysmeta.py | 13 +- 246 files changed, 7894 insertions(+), 3993 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 9f1582e..1001be5 100644 --- a/requirements.txt +++ b/requirements.txt @@ -13 +13 @@ PyECLib>=1.3.1 # BSD -cryptography>=1.0,!=1.3.0 # BSD/Apache-2.0 +cryptography!=2.0,>=1.6 # BSD/Apache-2.0