We are satisfied to announce the release of: openstack-ansible-os_swift 14.0.0: os_swift for OpenStack Ansible This release is part of the newton release series. Download the package from: https://tarballs.openstack.org/openstack-ansible-os_swift/ For more details, please see below. 14.0.0 ^^^^^^ New Features ************ * The os_swift role now supports the ability to configure whether apt/yum tasks install the latest available package, or just ensure that the package is present. The default action is to ensure that the latest package is present. The action taken may be changed to only ensure that the package is present by setting "swift_package_state" to "present". * The "fallocate_reserve` option can now be set (in bytes or as a percentage) for swift by using the ``swift_fallocate_reserve" variable in "/etc/openstack_deploy/user_variables.yml". This value is the amount of space to reserve on a disk to prevent a situation where swift is unable to remove objects due to a lack of available disk space to work with. The default value is 1% of the total disk size. * The "openstack-ansible-os_swift" role will now prevent deployers from changing the "swift_hash_path_prefix" and "swift_hash_path_suffix" variables on clusters that already have a value set in "/etc/swift/swift.conf". You can set the new "swift_force_change_hashes" variable to "True" to force the "swift_hash_path_" variables to be changed. We recommend setting this by running the os-swift.yml playbook with "-e swift_force_change_hashes=True", to avoid changing the "swift_hash_path_" variables unintentionally. Use with caution, changing the "swift_hash_path_" values causes end-user impact. * The "os_swift" role has 3 new variables that will allow a deployer to change the hard, soft and fs.file-max limits. the hard and soft limits are being added to the limits.conf file for the swift system user. The fs.file-max settings are added to storage hosts via kernel tuning. The new options are "swift_hard_open_file_limits" with a default of 10240 "swift_soft_open_file_limits" with a default of 4096 "swift_max_file_limits" with a default of 24 times the value of "swift_hard_open_file_limits". * The "pretend_min_part_hours_passed" option can now be passed to swift-ring-builder prior to performing a rebalance. This is set by the "swift_pretend_min_part_hours_passed" boolean variable. The default for this variable is False. We recommend setting this by running the os-swift.yml playbook with "-e swift_pretend_min_part_hours_passed=True", to avoid resetting "min_part_hours" unintentionally on every run. Setting "swift_pretend_min_part_hours_passed" to True will reset the clock on the last time a rebalance happened, thus circumventing the min_part_hours check. This should only be used with extreme caution. If you run this command and deploy rebalanced rings before a replication pass completes, you may introduce unavailability in your cluster. This has an end-user imapct. * While default python interpreter for swift is cpython, pypy is now an option. This change adds the ability to greatly improve swift performance without the core code modifications. These changes have been implemented using the documentation provided by Intel and Swiftstack. Notes about the performance increase can be seen here (https://software.intel.com/en-us/blogs/2016/05/06/doubling-the- performance-of-openstack-swift-with-no-code-changes). * Change the port for devices in the ring by adjusting the port value for services, hosts, or devices. This will not involve a rebalance of the ring. * Changing the port for a device, or group of devices, carries a brief period of downtime to the swift storage services for those devices. The devices will be unavailable during period between when the storage service restarts after the port update, and the ring updates to match the new port. * Enable rsync module per object server drive by setting the "swift_rsync_module_per_drive" setting to "True". Set this to configure rsync and swift to utilise individual configuration per drive. This is required when disabling rsyncs to individual disks. For example, in a disk full scenario. * The "os_swift" role will now include the swift "staticweb" middleware by default. * The os_swift role now allows the permissions for the log files created by the swift account, container and object servers to be set. The variable is "swift_syslog_log_perms" and is set to "0644" by default. Upgrade Notes ************* * Installation of swift and its dependent pip packages will now only occur within a Python virtual environment. The "swift_venv_enabled", "swift_venv_bin" variables have been removed. * The variable "swift_apt_packages" has been renamed to "swift_distro_packages". * The os_swift role always checks whether the latest package is installed when executed. If a deployer wishes to change the check to only validate the presence of the package, the option "swift_package_state" should be set to "present". * The "swift_fallocate_reserve" default value has changed from 10737418240 (10GB) to 1% in order to match the OpenStack swift default setting. * A new option *swift_pypy_enabled* has been added to enable or disable the pypy interpreter for swift. The default is "false". * A new option *swift_pypy_archive* has been added to allow a pre- built pypy archive to be downloaded and moved into place to support swift running under pypy. This option is a dictionary and contains the URL and SHA256 as keys. * The "swift_max_rsync_connections" default value has changed from 2 to 4 in order to match the OpenStack swift documented value. * When upgrading a Swift deployment from Mitaka to Newton it should be noted that the enabled middleware list has changed. In Newton the "staticweb" middleware will be loaded by default. While the change adds a feature it is non-disruptive in upgrades. Bug Fixes ********* * Previously, the "ansible_managed" var was being used to insert a header into the "swift.conf" that contained date/time information. This meant that swift.conf across different nodes did not have the same MD5SUM, causing "swift-recon --md5" to break. We now insert a piece of static text instead to resolve this issue. * The XFS filesystem is excluded from the daily mlocate crond job in order to conserve disk IO for large IOPS bursts due to updatedb/mlocate file indexing. Changes in openstack-ansible-os_swift 13.0.0..14.0.0 ---------------------------------------------------- 0e7a7a3 Always setup ResellerAdmin role in keystone 9c7c821 Use dictionary for service mappings 0acc370 Update tox.ini tests target for stable/newton c7c2b44 Update ansible-role-requirements to stable/newton 3893207 Update default git branch to stable/newton 27e8a5c Test address setting code paths c3e89df Use centralised test scripts 097eefb Update UPPER_CONSTRAINTS_FILE for stable/newton bc551d3 Update .gitreview for stable/newton 2a3b184 Fix bool evaluation for dedicated repl network ba28ffd Fix variable for replicator config options 768b47a Force Ansible to use dynamic includes a894972 Type min_part_hours,part_power,repl_number as int 9578aa5 Ensure storage/replication_ip are uniform df4d7ec Simplify networking for MR swift 2257044 Use openstack-ansible-tests repository for Swift 98cb749 Store dedicated replication conditions in var 4ea2993 Fix typo in ceilometer ignore_projects filter 05d6c6f Lookup Gnocchi service before proxy is configured 620edb7 Require lookup of Gnocchi identity when needed 0737337 Add "ansible_host" to each host 10d3ee9 Add check for swift_hash_path_ variables 7adb8b2 Fix missing newline in ceilometermiddleware config 1690a5d Compress test execution logs 1d55c37 Filter Gnocchi traffic optionally 80d2d96 Add Multi Region Swift gate scenario 750ccaa Refactor and improve swift-rings.py 599232a Point auditor service at the replicator config 71122a3 Namespace the telemetry vars for swift 702026d remove the path from the auth_url be77d23 Fix gate replication_network setting e31bec0 Rename package lists (and related vars) appropriately 43a6244 Enable log collection after functional testing aad88fd Make all linting tests use upper-constraints a9d1b9e Remove apt update from Vagrantfile b85cd52 Add SNI support via OS packages for os_swift 6fb5535 Updated from global requirements 49a472e Remove requirements.txt install for testing 388a831 Cleanup/standardize usage of tags eca8876 Add python packages for SNI support in tests d3305e4 Relocate Swift configuration docs to os_swift role 17da038 Move other-requirements.txt to bindep.txt 0b5919e Include ansible commands for ansible linting c926f33 Add pypy test Scenario for Swift 10b0fc1 Disable stderr logging dcb2a2a Properly default pretend_min_part_hours_passed 8256654 Allow swift log file permissions to be set 85a657d Add project group to role ae8ca49 Add package requirements for encryption 9790f9e Add ability to change apt/yum package state c9ea779 Ensure that mode is a string 99bfe27 Fixed simple performance issue and deprecations cce137c Ensure that doc linting is included in the linters test 5afbf0c Define region name. 850b60a Remove openstack_hosts from test requirements b51eb2a Optimise pip install tasks 582175d Support service-specific rabbitmq clusters 6753152 Use plugins repo version of the human_log callback plugin a7fb587 Updated from global requirements da2cd24 Remove duplicates from .gitignore 5e3a46a Implement doc8 checks for docs cce0410 Add pretend_min_part_hours_passed to ring build 5f9c83a Update sphinx configuration 022b62f Add options for configuring file limits in swift 5140ede Address Ansible bare variable usage efd2960 Update tox configuration eb04bf1 Refactor Swift ring build & check tasks 9d30bb3 Only install to virtual environment 7657961 Fix ansible-lint complaint about directory permissions 2ba929f Fix swift ring distribute 203743e add bool to var: swift_ceilometer_enabled d0c001c Implement Xenial Support bb17313 Updated swift + pypy 288500c Skip unavailable hosts when distributing keys 5a39413 Clean up container cache prep in tasks d142d36 Pin test-requirements to match OpenStack requirements 2dc3514 Make ansible managed line in swift.conf static 1dcc63d Adding missing / to swift rabbit connection string 6169255 Remove pip_lock_down dependency 9380862 Consistency for multi-os in the includes 38b510c Add staticweb to the default middleware list 69759b2 Updating os_swift to use the Multi-Distro framework daeea3c Set functional-test directory based on var 97a5b6d Verbose option has been deprecated from oslo.log 60b7b83 Enable pypy to be used with swift 53027de Setup rsync module per object server drive 292c443 Set fallocate_reserve to match upstream default be739c5 Refactor os_swift testing 03a5640 Add .swp files to .gitignore cf882fe Change pip install task state to 'latest' 167bf69 Add dependencies for paramiko 2.0 db4ad90 Remove XFS filesystem from the daily mlocate cron job 2c9e0d2 Add fucntional tests for swift c749241 Remove Liberty releasenote index 1a12be6 blacklist Ansible 1.9.6 81aecda Expose and set a default fallocate_reserve value e53ac82 Resolve issues with swift tests fd3aeae Add reno scaffolding for release notes management 0b43cdf Switch defaults/tests to use master branch 462d526 Add ceilometer middleware when ceilometer enabled 1e15882 Clean up deprecation warning in swift proxy 486adbe Include reseller admin role correctly cc81926 Adding Vagrantfile for local developer testing Diffstat (except docs and test files) ------------------------------------- .gitignore | 7 + .gitreview | 1 + CONTRIBUTING.rst | 49 ++- README.rst | 25 +- Vagrantfile | 8 + bindep.txt | 41 +++ defaults/main.yml | 233 +++++++++----- examples/playbook.yml | 8 + handlers/main.yml | 55 +--- manual-test.rc | 33 ++ meta/main.yml | 11 +- other-requirements.txt | 16 - releasenotes/notes/.placeholder | 0 .../notes/add-xenial-support-e285a643a39f0438.yaml | 4 + ...s_swift-only-install-venv-fdd5d41759433cf8.yaml | 5 + ...package-list-name-changes-e6f88d12f3bd9fa0.yaml | 4 + .../notes/package-state-f2309b07440d0ae8.yaml | 13 + .../notes/swift-conf-b8dd5e1199f8e4a8.yaml | 9 + .../swift-fallocate-reserve-ff513025da68bfed.yaml | 11 + .../swift-force-hash-change-45b09eeb8b0368a6.yaml | 14 + .../swift-fs-file-limits-a57ab8b4c3c944e4.yaml | 11 + .../swift-pretend-mph-passed-7e5c15eeb35861c3.yaml | 17 ++ .../notes/swift-pypy-support-9706519c4b88a571.yaml | 15 + ...onfigure-xfs-from-mlocate-e4844e6c0469afd6.yaml | 5 + .../swift-rings-port-change-4a95bbd9b63fb201.yaml | 11 + ...ft-rsync-module-per-drive-79b05af8276e7d6e.yaml | 12 + .../swift-staticweb-support-b280fbebf271820b.yaml | 9 + .../swift-syslog-log-perms-5a116171a1adeae3.yaml | 6 + releasenotes/source/_static/.placeholder | 0 releasenotes/source/_templates/.placeholder | 0 releasenotes/source/conf.py | 281 +++++++++++++++++ releasenotes/source/index.rst | 9 + releasenotes/source/mitaka.rst | 6 + releasenotes/source/unreleased.rst | 5 + setup.py | 11 +- tasks/install-apt.yml | 37 +++ tasks/main.yml | 104 +++++-- tasks/swift_calculate_addresses.yml | 131 ++++++++ tasks/swift_check_hashes.yml | 55 ++++ tasks/swift_command_check.yml | 30 -- tasks/swift_init_common.yml | 33 ++ tasks/swift_init_systemd.yml | 56 ++++ tasks/swift_init_upstart.yml | 26 ++ tasks/swift_install.yml | 162 ++-------- tasks/swift_key_distribute.yml | 9 +- tasks/swift_key_populate.yml | 6 - tasks/swift_key_setup.yml | 7 - tasks/swift_post_install.yml | 39 +-- tasks/swift_pre_install.yml | 53 ++-- tasks/swift_proxy_hosts.yml | 38 ++- tasks/swift_pypy_setup.yml | 74 +++++ tasks/swift_rings.yml | 9 +- tasks/swift_rings_build.yml | 50 +-- tasks/swift_rings_check.yml | 16 +- tasks/swift_rings_distribute.yml | 28 +- tasks/swift_rings_md5sum.yml | 4 - tasks/swift_rings_post_distribution_check.yml | 50 +-- tasks/swift_service_setup.yml | 32 +- tasks/swift_storage_hosts_account.yml | 78 +---- tasks/swift_storage_hosts_container.yml | 102 +------ tasks/swift_storage_hosts_object.yml | 93 +----- tasks/swift_storage_hosts_setup.yml | 24 +- tasks/swift_sync_post_install.yml | 21 +- tasks/swift_upstart_common_init.yml | 44 --- templates/account-server-replicator.conf.j2 | 8 +- templates/account-server.conf.j2 | 9 +- templates/container-reconciler.conf.j2 | 2 + templates/container-server-replicator.conf.j2 | 8 +- templates/container-server.conf.j2 | 13 +- templates/mlocate-crond-daily.sh.j2 | 21 ++ templates/object-expirer.conf.j2 | 2 + templates/object-server-replicator.conf.j2 | 11 +- templates/object-server.conf.j2 | 16 +- templates/proxy-server.conf.j2 | 26 +- templates/ring.contents.j2 | 39 +-- templates/rsyncd.conf.j2 | 38 +++ templates/swift-dispersion.conf.j2 | 1 + templates/swift-rsyslog.conf.j2 | 2 + templates/swift-systemd-init.j2 | 24 ++ templates/swift-systemd-tempfiles.j2 | 4 + templates/swift-upstart-init.j2 | 31 +- templates/swift.conf.j2 | 2 +- templates/swift_rings.py.j2 | 74 ++--- templates/swift_rings_check.py.j2 | 25 +- test-requirements.txt | 19 +- tox.ini | 236 ++++++++++----- vars/debian.yml | 28 ++ 120 files changed, 3537 insertions(+), 1510 deletions(-) Requirements updates -------------------- diff --git a/test-requirements.txt b/test-requirements.txt index 3422d65..8fdd8d8 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -1,4 +1,9 @@ -ansible-lint<=2.3.9 -ansible>=1.9.1,<2.0.0 -bashate -flake8 +# The order of packages is significant, because pip processes them in the order +# of appearance. Changing the order has an impact on the overall integration +# process, which may cause wedges in the gate later. +bashate>=0.2 # Apache-2.0 +flake8<2.6.0,>=2.5.4 # MIT +pyasn1 # BSD +pyOpenSSL>=0.14 # Apache-2.0 +requests>=2.10.0 # Apache-2.0 +ndg-httpsclient>=0.4.2;python_version<'3.0' # BSD @@ -7,2 +12,4 @@ flake8 -sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 -oslosphinx>=2.5.0 # Apache-2.0 +sphinx!=1.3b1,<1.3,>=1.2.1 # BSD +oslosphinx!=3.4.0,>=2.5.0 # Apache-2.0 +doc8 # Apache-2.0 +reno>=1.8.0 # Apache2