We eagerly announce the release of: swift 2.17.0: OpenStack Object Storage This release is part of the queens release series. Download the package from: https://tarballs.openstack.org/swift/ For more details, please see below. 2.17.0 ^^^^^^ New Features * Added symlink objects support. Symlink objects reference one other object. They are created by creating an empty object with an X-Symlink-Target header. The value of the header is of the format <container>/<object>, and the target does not need to exist at the time of symlink creation. Cross- account symlinks can be created by including the X-Symlink-Target- Account header. GET and HEAD requests to a symlink will operate on the referenced object and require appropriate permission in the target container. DELETE and PUT requests will operate on the symlink object itself. POST requests are not forwarded to the referenced object. POST requests sent to a symlink will result in a 307 Temporary Redirect response. * Added support for inline data segments in SLO manifests. Upgrade impact -- during a rolling upgrade, an updated proxy server may write a manifest that an out-of-date proxy server will not be able to read. This will resolve itself once the upgrade completes on all nodes. * The tempurl digest algorithm is now configurable, and Swift added support for both SHA-256 and SHA-512. Supported tempurl digests are exposed to clients in "/info". Additionally, tempurl signatures can now be base64 encoded. * Object expiry improvements * Disallow X-Delete-At header values equal to the X-Timestamp header. * X-Delete-At computation now uses X-Timestamp instead of system time. This prevents clock skew causing inconsistent expiry data. * Deleting an expiring object will now cause less work in the system. The number of async pending files written has been reduced for all objects and greatly reduced for erasure-coded objects. This dramatically reduces the burden on container servers. * Stopped logging tracebacks when receiving an unexpected response. * Allow the expirer to gracefully move past updating stale work items. * When the object auditor examines an object, it will now add any missing metadata checksums. * "swift-ring-builder" improvements * Save the ring when dispersion improves, even if balance doesn't improve. * Improved the granularity of the ring dispersion metric so that small improvements after a rebalance can show changes in the dispersion number. Dispersion in existing and new rings can be recalculated using the new "--recalculate" option to "swift-ring- builder". * Display more info on empty rings. * Fixed rare socket leak on range requests to erasure-coded objects. * The number of container updates on object PUTs (ie to update listings) has been recomputed to be far more efficient while maintaining durability guarantees. Specifically, object PUTs to erasure-coded policies will now normally result in far fewer container updates. * Moved Zuul v3 tox jobs into the Swift code repo. * Changed where liberasurecode-devel for CentOS 7 is referenced and installed as a dependency. * Added container/object listing with prefix to InternalClient. * Added "--swift-versions" to "swift-recon" CLI to compare installed versions in the cluster. * Stop logging tracebacks in the "object-replicator" when it runs out of handoff locations. * Send ETag header in 206 Partial Content responses to SLO reads. * Now "swift-recon-cron" works with conf.d configs. * Improved "object-updater" stats logging. It now tells you all of its stats (successes, failures, quarantines due to bad pickles, unlinks, and errors), and it tells you incremental progress every five minutes. The logging at the end of a pass remains and has been expanded to also include all stats. * If a proxy server is configured to autocreate accounts and the account create fails, it will now return a server error (500) instead of Not Found (404). * Fractional replicas are no longer allowed for erasure code policies. * Various other minor bug fixes and improvements. Changes in swift 2.16.0..2.17.0 ------------------------------- f4cfe81 authors/changelog updates for 2.17.0 release 924a043 Remove some cruft from ratelimit tests 5cac372 Refactor expirer unit tests 1c30666 Imported Translations from Zanata 5a4d3bd tempurl: Make the digest algorithm configurable 11bf9e4 Add support for data segments to SLO and SegmentedIterable 89a5c9d Disallow fractional replicas in EC policies 10542d0 Replace Chinese punctuation with English punctuation fdf817b Include policy in ran-out-of-handoffs log message 05b2a14 Update broken link ddb13aa Remove redundant blank space in README.rst cbcbc77 Internal Client should only retry server errors 12f8745 Add Docstrings to validate_replicas_by_tier d2034cd Keep object-updater stats logging consistent f4cff9f object-server can 409 in response to x-if-delete-at cd2c73f internal_client: Don't retry when we expect the same reponse 222df91 Modify redirection URL and broken URL d1656e3 slo: Send ETag header in 206 responses 88eea33 Recenter builder test expectation around random variance f64c00b Improve object-updater's stats logging 0c27fd2 Reset state every iteration to avoid stale reference 5347c30 Add container-reconciler to source doc tree d924fa7 Remove old post-as-copy leftovers from tests. dfa0c4e Preserve expiring object behaviour with old proxy-server 745581f Don't make async_pendings during object expiration d707fc7 DRY out tests until the stone bleeds ba8f1b1 Fix intermittent unit test failure e747f94 Fix InternalClient to drain response body if the request fails d8f9045 Send correct number of X-Delete-At-* headers cf1a1e8 expirer: unexpected responses don't warrant tracebacks 56b84c9 Minor cleanup in monitoring doc. bc6fb89 Add a note about the cost of COPY for setting metadata 0d324c1 Update http with https 6e394bb Add request_tries option to object-expirer.conf-sample 68906da Further extract a builder method to a util function a1ae142 Merge repeat code for rebalance 5cbd5cf Return HTTPServerError instead of HTTPNotFound 39efb51 Remove the deprecated "giturl" option a9964a7 fix barbican integration 35ad4e8 Add tests for X-Backend-Clean-Expiring-Object-Queue true 7afc6a0 Remove un-needed hack in probetest 55a1b63 Let recon-cron work with conf.d 48da3c1 Limit object-expirer queue updates on object DELETE, PUT, POST 9754a2e Change exit code when displaying empty rings a41c458 proxy: make the right number of container updates b451cee Add pipeline modification test for previously-recommended pipelines e7ffda5 Use _update_x_timestamp method in object controller DELETE method bf13d64 Show devices marked as deleted on empty rings e343452 Support existing builders with None _last_part_moves 94565d9 Disallow x-delete-at equal to x-timestamp 79ac3a3 Fix intermittent check_delete_headers failure 6151554 Correct 400 response message when x-delete-after is zero b22d3c1 fix of type error 582460e Document that x-delete-after takes precedence over x-delete-at 31c294d Fix time skew when using X-Delete-After a13e0ee Ignore directory .stestr 49de7db add swift-ring-builder option to recalculate dispersion 9189f51 Display more info on empty rings 56126b2 Handle EmptyRingError in swift-ring-builder's default command f709eed Fix socket leak on 416 EC GET responses. 7013e70 Represent dispersion worse than one replicanth 61fe6aa Better mock out OSErrors in test_replicator before raising them 0bdec46 Skip symlink + vw functional tests if symlink is not enabled 1449532 Allow InternalClient to container/object listing with prefix 17e6950 Fix manpage docs' daemon names af2c2a6 Fix sometimes-flaky container name functional test. 609c757 functest for symlink + versioned writes bdd4eb6 Install liberasurecode-devel for CentOS 7 dc1c55c Native Zuul v3 tox jobs a7da223 Fix intermittent problem in part_swapping test cd11289 Add a small sleep when trying to predict X-Timestamp fba3fb7 Stop logging tracebacks when the replicator runs out of handoffs 2cf5e7c fix SkipTest imports in functests so they can be run directly by nose aa82d2c Save ring builder if dispersion changes 1984353 Move symlink versioning functional test c579e99 Add more assertions for Symlink + Copy unit tests 097e975 Remove symlink from xml listing response 2f2899e Show missing branches in coverage report. 7647def rename utils function less like stdlib b342a81 Assert X-Newest and X-Backend headers are propagated to symlink target 1d5cf3e add symlink to probetest for reconciler 90134ee add symlink to container sync default and sample config 8df2631 Symlink doc clean up 1b6842d add name to core emeritus 99b89ae Symlink implementation. b86bf15 remove left-over old doc in config file 8182fa7 Fix small error in a doc string dd113ab Refactor proxy-server conf loading to a helper function 84ea58b Ringbuilder: Forbid writing empty rings fc12d63 Remove repeated text from deployment guide bc2e03d Add --swift-versions option to swift-recon man page 206f674 Added swift version to recon cli b808c80 Remove testTooLongName tests 924f0d2 dlo: Move conn2 business to the one test that uses it 3135878 Fix small formatting issues in middleware docs 4f3abec Fix small formatting issue de81861 Fix formatting of SAIO docs c3e73b6 Remove setting of version/release from releasenotes 1962b18 Remove unnecessary commented lines 396380f Better handle missing files in _construct_from_data_file 62d21ec Add metadata checksums to old objects in auditor. 6990d02 added some people to core emeritus in the authors file 31dd95e Add base64decode function to common/utils 5b26516 Increase name-length limits for internal accounts 88c9ba4 Fix bulk heartbeating when emitting XML Diffstat (except docs and test files) ------------------------------------- .coveragerc | 1 + .gitignore | 1 + .mailmap | 3 +- .zuul.yaml | 117 ++ AUTHORS | 18 +- CHANGELOG | 102 ++ README.rst | 6 +- api-ref/source/conf.py | 4 - api-ref/source/parameters.yaml | 93 +- api-ref/source/storage-container-services.inc | 1 + api-ref/source/storage-object-services.inc | 41 +- bin/swift-recon-cron | 9 +- bindep.txt | 13 +- etc/internal-client.conf-sample | 6 +- etc/keymaster.conf-sample | 7 +- etc/object-expirer.conf-sample | 24 +- etc/object-server.conf-sample | 6 + etc/proxy-server.conf-sample | 26 +- .../notes/2_17_0_release-bd35f18c41c5ef18.yaml | 119 ++ releasenotes/source/conf.py | 11 +- .../locale/en_GB/LC_MESSAGES/releasenotes.po | 900 +++++++++- .../locale/ko_KR/LC_MESSAGES/releasenotes.po | 75 - setup.cfg | 3 +- swift/cli/recon.py | 42 +- swift/cli/ringbuilder.py | 84 +- swift/common/constraints.py | 28 +- swift/common/exceptions.py | 4 + swift/common/internal_client.py | 82 +- swift/common/middleware/bulk.py | 4 +- swift/common/middleware/copy.py | 73 +- swift/common/middleware/crypto/decrypter.py | 5 +- swift/common/middleware/crypto/keymaster.py | 17 +- swift/common/middleware/crypto/kms_keymaster.py | 1 + swift/common/middleware/dlo.py | 51 +- swift/common/middleware/keystoneauth.py | 4 +- swift/common/middleware/listing_formats.py | 1 + swift/common/middleware/slo.py | 380 +++-- swift/common/middleware/symlink.py | 579 +++++++ swift/common/middleware/tempurl.py | 117 +- swift/common/middleware/xprofile.py | 4 +- swift/common/request_helpers.py | 66 +- swift/common/ring/builder.py | 57 +- swift/common/ring/ring.py | 13 +- swift/common/ring/utils.py | 32 +- swift/common/storage_policy.py | 4 +- swift/common/utils.py | 45 +- swift/common/wsgi.py | 18 + swift/container/sync.py | 14 +- swift/locale/de/LC_MESSAGES/swift.po | 27 +- swift/locale/en_GB/LC_MESSAGES/swift.po | 1305 ++++++++++++++ swift/locale/es/LC_MESSAGES/swift.po | 31 +- swift/locale/fr/LC_MESSAGES/swift.po | 27 +- swift/locale/it/LC_MESSAGES/swift.po | 27 +- swift/locale/ja/LC_MESSAGES/swift.po | 27 +- swift/locale/ko_KR/LC_MESSAGES/swift.po | 27 +- swift/locale/pt_BR/LC_MESSAGES/swift.po | 27 +- swift/locale/ru/LC_MESSAGES/swift.po | 27 +- swift/locale/tr_TR/LC_MESSAGES/swift.po | 27 +- swift/locale/zh_CN/LC_MESSAGES/swift.po | 25 +- swift/locale/zh_TW/LC_MESSAGES/swift.po | 27 +- swift/obj/auditor.py | 2 +- swift/obj/diskfile.py | 45 +- swift/obj/expirer.py | 24 +- swift/obj/mem_diskfile.py | 2 +- swift/obj/replicator.py | 6 +- swift/obj/server.py | 21 +- swift/obj/updater.py | 123 +- swift/proxy/controllers/account.py | 19 +- swift/proxy/controllers/base.py | 17 + swift/proxy/controllers/container.py | 12 +- swift/proxy/controllers/obj.py | 138 +- test/functional/swift_test_client.py | 5 +- test/functional/test_account.py | 3 +- test/functional/test_container.py | 3 +- test/functional/test_dlo.py | 14 +- test/functional/test_object.py | 7 +- test/functional/test_slo.py | 123 +- test/functional/test_symlink.py | 1781 ++++++++++++++++++++ test/functional/test_tempurl.py | 126 +- test/functional/test_versioned_writes.py | 152 +- test/probe/common.py | 10 +- test/probe/test_container_merge_policy_index.py | 82 +- test/probe/test_container_sync.py | 196 ++- test/probe/test_object_expirer.py | 81 +- test/unit/cli/test_recon.py | 59 + test/unit/cli/test_ringbuilder.py | 232 ++- .../common/middleware/crypto/test_decrypter.py | 22 + .../common/middleware/crypto/test_kms_keymaster.py | 24 +- test/unit/common/middleware/helpers.py | 8 +- test/unit/common/middleware/test_bulk.py | 15 +- test/unit/common/middleware/test_copy.py | 3 + test/unit/common/middleware/test_dlo.py | 11 +- test/unit/common/middleware/test_ratelimit.py | 39 - test/unit/common/middleware/test_slo.py | 331 +++- test/unit/common/middleware/test_symlink.py | 929 ++++++++++ test/unit/common/middleware/test_tempurl.py | 162 +- test/unit/common/ring/test_builder.py | 230 ++- test/unit/common/ring/test_ring.py | 20 + test/unit/common/ring/test_utils.py | 106 +- test/unit/common/test_constraints.py | 277 +-- test/unit/common/test_internal_client.py | 235 ++- test/unit/common/test_storage_policy.py | 36 +- test/unit/common/test_utils.py | 49 + test/unit/common/test_wsgi.py | 255 +++ test/unit/container/test_reconciler.py | 14 +- test/unit/container/test_sync.py | 24 +- test/unit/obj/test_auditor.py | 69 + test/unit/obj/test_diskfile.py | 20 +- test/unit/obj/test_expirer.py | 423 ++--- test/unit/obj/test_replicator.py | 19 +- test/unit/obj/test_server.py | 508 ++++-- test/unit/obj/test_updater.py | 71 +- test/unit/proxy/controllers/test_obj.py | 340 +++- test/unit/proxy/test_server.py | 187 +- tools/test-setup.sh | 30 + tox.ini | 5 - 140 files changed, 11073 insertions(+), 1928 deletions(-)