We are pleased to announce the release of: swift 2.10.0: OpenStack Object Storage This release is part of the newton release series. For more details, please see below. 2.10.0 ^^^^^^ New Features ************ * Object versioning now supports a "history" mode in addition to the older "stack" mode. The difference is in how DELETE requests are handled. For full details, please read http://docs.openstack.org/de veloper/swift/overview_object_versioning.html. * New config variables to change the schedule priority and I/O scheduling class. Servers and daemons now understand *nice_priority*, *ionice_class*, and *ionice_priority* to schedule their relative importance. Please read http://docs.openstack.org/developer/swift/deployment_guide.html for full config details. * On newer kernels (3.15+ when using xfs), Swift will use the O_TMPFILE flag when opening a file instead of creating a temporary file and renaming it on commit. This makes the data path simpler and allows the filesystem to more efficiently optimize the files on disk, resulting in better performance. * Erasure code GET performance has been significantly improved in clusters that are not completely healthy. * Significant improvements to the api-ref doc available at http://developer.openstack.org/api-ref/object-storage/. * A PUT or POST to a container will now update the container's Last- Modified time, and that value will be included in a GET/HEAD response. * Include object sysmeta in POST responses. Sysmeta is still stripped from the response before being sent to the client, but this allows middleware to make use of the information. Upgrade Notes ************* * Update dnspython dependency to 1.14, removing the need to have separate dnspython dependencies for Py2 and Py3. * Deprecate swift-temp-url and call python-swiftclient's implementation instead. This adds python-swiftclient as an optional dependency of Swift. * Moved other-requirements.txt to bindep.txt. bindep.txt lists non- python dependencies of Swift. Bug Fixes ********* * Fixed a bug where a container listing delimiter wouldn't work with encryption. * Fixed a bug where some headers weren't being copied correctly in a COPY request. * Container sync can now copy SLOs more efficiently by allowing the manifest to be synced before all of the referenced segments. This fixes a bug where container sync would not copy SLO manifests. * Fixed a bug where some tombstone files might never be reclaimed. Other Notes *********** * Various other minor bug fixes and improvements. Changes in swift 2.9.0..2.10.0 ------------------------------ 6bf63d8 authors/changelog updates for 2.10.0 3da65be Cleanup tests for auditor invalidating hashes bfaa8e0 Fix ChunkWriteError when running unittests 60a2fe0 Use separate headers for versioned_writes' stack and history modes 81d4673 Delete old tombstones 4f033a6 Fix unstable json.dumps ordering in obj-server test cbfa79a api-ref: fix some header definitions d40ff70 Clean up api-ref examples 9a81b95 Fixed cut and pasted paragraph from commit message in manpage 5ec9a62 api-ref: clean up account listing examples 2355771 api-ref: Move repeated paragraph to an include file d014496 Include object sysmeta in POST responses a741998 Turn on F812 check 70e3cf1 remove comment saying we ignore H233. we actually check it b587d16 Updating Bandit config file 44a8617 Enable object server to return non-durable data acb8971 Update link reference d062af8 DRY out probe.common f679ed0 Make container sync copy SLO manifests 9ce596a Corrections for the API specifications in api-ref 2aa5811 fix string formatting in manager f35829c Bring "egregious range request" comment in line with reality 3f2a4b2 Add test_pop in the test_header_key_dict f285caf delete TODO in test_manager 51925e5 Add man page for swift.conf 0900a2c Fix error handling in proxy/controllers/base.py. ba610e0 add missing partition directory to example ee11955 Add troubleshooting tips to keystoneauth docs e28ff06 Add regions in ring-builder.py dd5afdb simple log information 1cfa335 Fixed Typo in proxy/controller/obj.py 1751db2 Add test for copy using a Range header ba912a1 Add test_valid_api_version in the test_constraints 8b3c04a py3: Port test_splice to Python 3 0f76185 Added missing manpages for all cli tools 350f10b Deprecate swift-temp-url 84b264b Fix api reference of object GET request with Range parameter 1883c1e Make get_info requests useful with recheck_*_existence == 0 a427911 Remove meaningless lines in reconciler unit test c948802 Make versioned_writes docstring more precise 2a58b1e More Test[Account|Container]Controller test updates ff52bb1 Avoid warnings for versioned_writes docs 2825909 Silence "Client disconnected" warnings on reads. 6712542 Correct swift reraising of exception 0c22bc2 Include correct version in install-guide 66c905e Close the iterators in string_along. cd8eb6d Add test to validate the error strings in test_expirer 3b58508 Remove white space between print and () f88e7fc Clarify test case in common/ring/test_builder d68b1bd Remove unnecessary tearDown d2fc261 Authorise versioned write PUTs before copy 4d4885a Tighten header checks for object PUT/POST paths c1ef653 add test for expirer processes == process 01477c7 Fix ValueError information in obj/expirer 773edb4 Make object creation more atomic in Linux b81f53b Improve readability in the obj server's unit tests 89388bf Fix typos and grammer in builder.py d98928c py3: tox.ini: use substituation to py35 commands 06ff865 Allow DLO manifest to be updated when using post-as-copy aa893d9 Get ready for os-api-ref sphinx theme change 8bf2233 Documantation enhancements of nice/ionice feature 1374702 Add test for POST to DLO manifest file b7b77c7 Follow up delayed reap probe test 1fc46a0 py3: Fixes encoding and type error 26633af Reset AccountReaper stats in __init__ 8d6bf36 Remove unnecessary variable ce49a29 Add rm to tox whitelist_externals dfa5523 Add Pros/Cons docs for global cluster consideration 95a5a4a Don't run probe tests if resetswift failed 30b97f2 Drop X-Auth-Token from all versioned_writes subrequests cc2b2cf Improve doc for using container-sync with large objects 65b1820 Ignore auditor status files to prevent replicator reports errors c7283be Add "history" mode to versioned_writes middleware bb87fce Fix typo in the file aab2cee Move other-requirements.txt to bindep.txt f7a820e Wait for a non-empty chunk in WSGIContext._app_call 05b8d9d fix swift_oldies on RH based systems 7e2cb23 Grammer error : swift/doc/source/overview_ring.rst 844d0c9 Corrects spelling error in swift/common/middleware/slo.py 4638171 Fix assertEqual error ed77223 Change schedule priority of daemon/server in config 44ba3c3 Added a test for delayed reap. ddbab05 add reminder how to run debug func tests bd29a3e Remove the duplicated word 'be' 8b5578c [install-guide] Include environment-networking 4aa1ae6 Raise 412 response on expirer c5ff993 NIT: fixing inconsistent naming of OpenStack Swift a9fa5ab Tighten up direct client unit tests ce44ec4 Make log information format 3096dc6 Make the logger information format eadab84 Remove unnecessary open call and related vars in unit tests fb5fcb1 Fix encryption-delimiter interaction 4f94e71 add swift-oldies man page b1b4109 Add unit case 303e4a1 missing parens in functional test 425bb0d Fix stale docstring in SegmentedIterable. 488f88e Use more specific asserts in test/unit/cli tests 5cf24ec Fix link to docs from swift-dispersion manpages e0c7e6b Fix repeated OPTIONS in swift-dispersion-report manpage d819ae0 update swift-dispersion manpages to add policy-name e7f025f made link in README.rst more clear 99a6f91 swift-ring-builder output corrected for ipv6 1533eb3 Reduce code duplication eb53590 modify the home-page info with the developer documentation 0d41b23 Add end_marker and reverse options to direct_client 6652014 Enable in-process func tests to optionally use encryption 5c9732a Moved ipv4 & ipv6 validations to the common utils aa2a84b Check object metadata constraints after authorizing 457cea8 Handle IPv6 addresses in swift-get-nodes. 8bf28c8 Fix broken link in associated projects doc 6575ac0 Simple code and improve the readability of the code 149a331 Additional test for container override etag preferences fbf0e49 monkey_patch_mimetools() now does nothing on py3 7cc2392 Document how to run a single functional test cf1f7af Use self.ts more consistently in obj.test_server a97537e Split up backend/sysmeta header/footer preference tests 77e4763 Mention SWIFT_TEST_DEBUG_LOGS in development guide 7958638 Added quotes to example echo in swift-temp-url a81d604 Fix Python 3 issues in diskfile 6999535 Add doc entry to check partition count c1c18da check _last_part_moves when pretend_min_part_hours_passed bf17d96 Use extract_swift_bytes in override_bytes_from_content_type 5677a04 Python 3: Fix usage of dict methods e677630 Python 3: fix usage of reload() 1eb9639 Fix EC ring validation at ring reload 3a1a198 Raise ValueError if empty value coming into encrypt_header_val ab602a2 Make load_crypto_meta more py3-friendly 2876f59 Cache fragment size for EC policy afe3968 Copy headers correctly when copying object 26d91f2 Python3 eventlet patched httplib _MAXHEADERS 8733311 Prevent CPU spinning when there are no children 49f2507 Python3 fixes generator object issue cdf505a Make swift-oldies py3-compatible d9b7653 Fixed Typo in updater.py f337421 Change assertTrue to assertEqual aaa6315 Imported Translations from Zanata cc00163 Remove red herring from logs c7e5afb Fixed typo in reaper.py bc09be4 Make comparision simplely ecce794 Add log format to i18n 45bde71 Simplify chained comparison e278179 Use more specific asserts in functional tests 75a58a6 Use more specific asserts in test/unit/proxy tests 77d6d01 Use more specific asserts in test/unit/common/ring 5d02b95 Reduce code duplication 90627f9 Add region in ring structure & deployment guide 5cd57dc Improve Keystone v3 token support ffaef48 Add encrypter and decrypter links to middleware.rst c213752 Update dnspython to 1.14 bcd9a58 Fix X-*-Container-Update-Override-* header/footer precedence 3781843 Fix gettext_ calls 401311f Have py35 tox env match py34 57ac316 Fix intermittent SLO unit test failures 3651713 Remove some unnecessary error handling in healthcheck 29b8d2d Avoid docs warning: Duplicate explicit target name fcb6e4c Last-Modified header support on HEAD/GET container Diffstat (except docs and test files) ------------------------------------- .mailmap | 2 + AUTHORS | 17 + CHANGELOG | 59 + README.rst | 2 +- api-ref/source/conf.py | 35 +- api-ref/source/index.rst | 8 +- api-ref/source/metadata_header_encoding.inc | 6 + api-ref/source/metadata_header_syntax.inc | 7 + api-ref/source/parameters.yaml | 592 ++++++---- .../account-containers-list-http-request-json.txt | 2 +- .../account-containers-list-http-request-xml.txt | 3 +- .../source/samples/capabilities-list-response.json | 5 + api-ref/source/storage-account-services.inc | 194 ++-- api-ref/source/storage-container-services.inc | 157 ++- api-ref/source/storage-object-services.inc | 188 +-- api-ref/source/storage_info.inc | 5 + bandit.yaml | 294 ++--- bin/swift-account-audit | 2 +- bin/swift-oldies | 21 +- bin/swift-temp-url | 65 +- bindep.txt | 17 + etc/account-server.conf-sample | 62 +- etc/container-reconciler.conf-sample | 24 + etc/container-server.conf-sample | 72 ++ etc/object-expirer.conf-sample | 22 + etc/object-server.conf-sample | 72 ++ etc/proxy-server.conf-sample | 24 + install-guide/source/conf.py | 5 +- install-guide/source/edit_hosts_file.txt | 19 + install-guide/source/environment-networking.rst | 66 ++ install-guide/source/index.rst | 1 + other-requirements.txt | 17 - .../notes/2_10_0_release-666a76f4975657a5.yaml | 59 + requirements.txt | 3 +- setup.cfg | 2 +- swift/account/auditor.py | 2 +- swift/account/backend.py | 10 +- swift/account/reaper.py | 3 +- swift/cli/info.py | 44 +- swift/cli/ringbuilder.py | 213 ++-- swift/common/bufferedhttp.py | 5 +- swift/common/daemon.py | 3 + swift/common/db_replicator.py | 10 +- swift/common/direct_client.py | 22 +- swift/common/exceptions.py | 4 + swift/common/linkat.py | 76 ++ swift/common/manager.py | 2 +- swift/common/middleware/acl.py | 2 +- swift/common/middleware/container_sync.py | 5 + swift/common/middleware/copy.py | 55 +- swift/common/middleware/crypto/__init__.py | 3 +- swift/common/middleware/crypto/crypto_utils.py | 13 +- swift/common/middleware/crypto/decrypter.py | 45 +- swift/common/middleware/crypto/encrypter.py | 3 +- swift/common/middleware/dlo.py | 14 +- swift/common/middleware/healthcheck.py | 15 +- swift/common/middleware/keystoneauth.py | 6 +- swift/common/middleware/slo.py | 5 +- swift/common/middleware/tempauth.py | 4 +- swift/common/middleware/versioned_writes.py | 510 +++++++-- swift/common/middleware/x_profile/html_viewer.py | 6 +- swift/common/request_helpers.py | 14 +- swift/common/ring/builder.py | 35 +- swift/common/ring/ring.py | 20 +- swift/common/ring/utils.py | 32 +- swift/common/storage_policy.py | 70 +- swift/common/swob.py | 4 +- swift/common/utils.py | 274 ++++- swift/common/wsgi.py | 47 +- swift/container/server.py | 6 +- swift/container/sync.py | 2 - swift/container/updater.py | 2 +- swift/locale/de/LC_MESSAGES/swift.po | 12 +- swift/locale/es/LC_MESSAGES/swift.po | 252 +++- swift/locale/fr/LC_MESSAGES/swift.po | 14 +- swift/locale/it/LC_MESSAGES/swift.po | 13 +- swift/locale/ja/LC_MESSAGES/swift.po | 12 +- swift/locale/ko_KR/LC_MESSAGES/swift.po | 12 +- swift/locale/pt_BR/LC_MESSAGES/swift.po | 12 +- swift/locale/ru/LC_MESSAGES/swift.po | 14 +- swift/locale/tr_TR/LC_MESSAGES/swift.po | 12 +- swift/locale/zh_CN/LC_MESSAGES/swift.po | 12 +- swift/locale/zh_TW/LC_MESSAGES/swift.po | 12 +- swift/obj/auditor.py | 37 +- swift/obj/diskfile.py | 238 +++- swift/obj/expirer.py | 7 +- swift/obj/replicator.py | 21 +- swift/obj/server.py | 60 +- swift/proxy/controllers/base.py | 66 +- swift/proxy/controllers/container.py | 3 +- swift/proxy/controllers/obj.py | 500 ++++++-- swift/proxy/server.py | 15 +- test-requirements.txt | 2 +- test/functional/__init__.py | 70 ++ test/functional/swift_test_client.py | 40 +- test/functional/test_account.py | 52 +- test/functional/test_container.py | 30 +- test/functional/test_object.py | 2 +- test/probe/common.py | 88 +- test/probe/test_account_reaper.py | 82 +- test/probe/test_container_merge_policy_index.py | 9 +- test/probe/test_container_sync.py | 129 ++- test/probe/test_object_expirer.py | 90 +- test/probe/test_object_metadata_replication.py | 3 - test/probe/test_wsgi_servers.py | 4 +- test/unit/__init__.py | 10 + test/unit/account/test_backend.py | 19 +- test/unit/account/test_reaper.py | 39 +- test/unit/account/test_server.py | 53 +- test/unit/cli/test_default_output.stub | 11 + test/unit/cli/test_form_signature.py | 12 +- test/unit/cli/test_info.py | 219 ++-- test/unit/cli/test_ipv6_output.stub | 10 + test/unit/cli/test_recon.py | 40 +- test/unit/cli/test_ring_builder_analyzer.py | 2 +- test/unit/cli/test_ringbuilder.py | 164 ++- test/unit/common/middleware/crypto/test_crypto.py | 65 +- .../common/middleware/crypto/test_decrypter.py | 26 +- .../common/middleware/crypto/test_encrypter.py | 33 + test/unit/common/middleware/helpers.py | 8 +- test/unit/common/middleware/test_container_sync.py | 14 +- test/unit/common/middleware/test_copy.py | 286 ++++- test/unit/common/middleware/test_gatekeeper.py | 20 +- test/unit/common/middleware/test_keystoneauth.py | 105 +- test/unit/common/middleware/test_slo.py | 84 +- .../common/middleware/test_versioned_writes.py | 478 +++++++- test/unit/common/ring/test_builder.py | 52 +- test/unit/common/ring/test_ring.py | 4 +- test/unit/common/ring/test_utils.py | 75 +- test/unit/common/test_constraints.py | 10 + test/unit/common/test_daemon.py | 3 +- test/unit/common/test_db_replicator.py | 8 +- test/unit/common/test_direct_client.py | 126 +- test/unit/common/test_header_key_dict.py | 13 +- test/unit/common/test_linkat.py | 103 ++ test/unit/common/test_manager.py | 5 +- test/unit/common/test_splice.py | 38 +- test/unit/common/test_storage_policy.py | 57 +- test/unit/common/test_swob.py | 1 + test/unit/common/test_utils.py | 269 ++++- test/unit/common/test_wsgi.py | 16 +- test/unit/container/test_backend.py | 20 +- test/unit/container/test_reconciler.py | 2 - test/unit/container/test_server.py | 89 +- test/unit/container/test_sync.py | 77 +- test/unit/obj/test_auditor.py | 204 +++- test/unit/obj/test_diskfile.py | 524 ++++++++- test/unit/obj/test_expirer.py | 144 ++- test/unit/obj/test_replicator.py | 140 ++- test/unit/obj/test_server.py | 611 ++++++++-- test/unit/obj/test_ssync.py | 8 +- test/unit/proxy/controllers/test_account.py | 14 +- test/unit/proxy/controllers/test_base.py | 210 +++- test/unit/proxy/controllers/test_container.py | 12 +- test/unit/proxy/controllers/test_info.py | 74 +- test/unit/proxy/controllers/test_obj.py | 1204 +++++++++++++++++--- test/unit/proxy/test_mem_server.py | 3 + test/unit/proxy/test_server.py | 625 +++++++--- test/unit/proxy/test_sysmeta.py | 6 +- tox.ini | 19 +- 223 files changed, 12283 insertions(+), 3438 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 3c17288..eed45fa 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,2 +5 @@ -dnspython>=1.12.0;python_version<'3.0' -dnspython3>=1.12.0;python_version>='3.0' +dnspython>=1.14.0 # http://www.dnspython.org/LICENSE diff --git a/test-requirements.txt b/test-requirements.txt index 933bc68..b5cc75f 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -21 +21 @@ python-keystoneclient>=1.3.0 -bandit>=0.10.1 +bandit>=1.1.0 # Apache-2.0