We are gleeful to announce the release of: python-watcher 2.0.0: OpenStack Watcher provides a flexible and scalable resource optimization service for multi-tenant OpenStack- based clouds. This release is part of the stein release series. The source is available from: https://git.openstack.org/cgit/openstack/watcher Download the package from: https://tarballs.openstack.org/watcher/ Please report issues through: https://bugs.launchpad.net/watcher/+bugs For more details, please see below. Changes in python-watcher 1.12.0..2.0.0 --------------------------------------- c8bfb3b1 Move client function test to watcher-tempest-plugin e830d379 Access to action's uuid by key a3e5b26c releasenote for data model scope 6da31b42 Fix unittest failed 9435418b Remove unused type check 'int' in audit.py b24bd7a3 Generalize exceptions & structure of strategies f337c67b scope for datamodel eb48cee9 Fix inappropriate description about the audit_state_machine.png 4cdf6a79 improve _collect_aggregates 594039f7 Provide two arguments to exception's message ea728d91 make ceilometer client import optional 4590c47a Fix uniform airflow strategy config parameter dd3c4d55 Fix outlet_temp_control config parameter 9c4b750c remove config parameter 'datasource' a9c02935 Add the define of vm_workload_consolidation job 477b4d01 change config parameter from 'datasource' to 'datasources' 73830387 Move datasources metric mappings out of base.py f8dfdd40 function get_sd return 0 early if len(hosts) is 0 0e46dec6 Update storage_balance job dff99bda Add storage balance job 121ec6e5 Update user guide 1fba9949 Fix E731 error 551f7c8a trivial 1e49203c [Trivial fix] Do not use self in classmethod 103e5b56 Add grenade job a4d31eac Update hacking version f7dcefb5 Add version api ref 6addd6bd update api-ref for audit start/end time d8017c17 Use template for lower-constraints 4f2b4fdd Remove unused modules a19b799d Add host maintenance tempest 8bed8769 Fix mailing list archive URL b0c504cd Fix stop_watcher function 92894237 Deprecate Ceilometer Datasource 216e63c1 Fix doc about nova notifications e251d85f Remove hostname-related playbooks b0a86b42 audit create request can't set scope 4a1d03b9 Change openstack-dev to openstack-discuss channel 0dc9f39a Increase the unit test coverage of host_maintenance.py c032807a update doc for install ubuntu 92f2d1c0 Fix spelling error in the comments of file host_maintenance.py af0c90db Add audit scoper for baremetal data model ddba3573 Increase the unit test coverage of vm_workload_consolidation.py fd8bd4d5 Fix audit_template_uuid description b4b625c9 start tls-proxy (if enabled) before checking for api f1ce2a58 remove older api doc 2cf1187a Add missing ws separator between words 63afd825 Update doc for vm_workload_consolidation strategy 4f99c6be Increase the unit test coverage of cinder_helper.py c7800225 Increase the unit test coverage of nova_helper.py 9f09abd6 Fix version header in the response 69cf0d3e Enhance Watcher Applier Engine 1e8b63e6 Remove unsuitable brackets 4c3bac14 Imported Translations from Zanata b6c24ed4 To avoid ambiguity for flavor_id e8c08e2a Fix accessing to optional cinder pool attributes e4d4a262 Add cover job 456ce5a9 Remove redundant docstring 8a3d9fc4 optimize get_instances_by_node 701a2483 Adjust the same format as above a9393ef2 Make watcherclient-tempest-functional test non-voting d6dc5675 Add detailed unit test documentation for watcher f6c09465 Update min tox version to 2.0 34523ec2 Fix parameter type for cinder pool a8eed9fc update datamodel by nova notifications c4a30153 API Microversioning f9fe6659 remove set_host_offline cbaf3851 Watcher doesn't need paramiko 93890fb2 Don't need nova notifications 62b9282b Fix oslo_versionedobjects warnings d621f727 Add framework for watcher-status upgrade check 7c9a8569 Update documentation regarding DataSource for strategies e5386aa7 Use limit -1 for nova servers list b69fc584 tenant_id should be project_id in instance element c2550e53 add start and end time for continuous audit f41adc7e Remove uses of rpc_backend (oslo_config) a073c42a Don't quote {posargs} in tox.ini f0b96b8a Do not pass www_authenticate_uri to RequestContext 5265b06a remove nova legacy notifications bb2f6d23 Fix link to Watcher API 7cb81ac6 Fix audit creation with named goal and strategy 925b9713 Provide region name while initialize clients 81c241be Add efficacy indicators for workload_stabilization strategy 1f3c96d0 ignore .testrepository 9e00d1bb Fix wrong audit scope bb31cc59 add python 3.6 unit test job 37d9aa52 switch documentation job to new PTI 0b40479d Follow the new PTI for building docs 35e9422e Imported Translations from Zanata 3861f1c8 Remove warning log in common.context d822a0f3 Remove hosts if can't find hosts in host agrregate e772b289 Fix goal method in policy b0957931 import zuul job settings from project-config a1b0c090 Remove -u root as mysql is executed with root user 172d7b04 Improve logs of Workload Stabilization strategy 9686d200 Imported Translations from Zanata cc2962af Add hostname to API Reference 7247d2f9 Update reno for stable/rocky Diffstat (except docs and test files) ------------------------------------- .gitignore | 4 + .zuul.yaml | 124 +++-- api-ref/source/index.rst | 3 +- api-ref/source/parameters.yaml | 117 ++++- .../samples/actionplan-list-detailed-response.json | 3 +- .../source/samples/actionplan-show-response.json | 3 +- .../source/samples/actionplan-start-response.json | 3 +- api-ref/source/samples/api-root-response.json | 30 ++ api-ref/source/samples/api-v1-root-response.json | 80 ++++ api-ref/source/samples/audit-cancel-response.json | 7 +- .../samples/audit-create-request-continuous.json | 6 +- api-ref/source/samples/audit-create-response.json | 7 +- .../samples/audit-list-detailed-response.json | 7 +- api-ref/source/samples/audit-show-response.json | 7 +- api-ref/source/samples/audit-update-response.json | 7 +- api-ref/source/watcher-api-v1-actionplans.inc | 4 + api-ref/source/watcher-api-v1-audits.inc | 26 +- api-ref/source/watcher-api-versions.inc | 81 ++++ devstack/lib/watcher | 25 +- devstack/local.conf.compute | 3 - devstack/local.conf.controller | 3 - devstack/upgrade/from_rocky/upgrade-watcher | 15 + devstack/upgrade/resources.sh | 126 ++++++ devstack/upgrade/settings | 11 + devstack/upgrade/shutdown.sh | 24 + devstack/upgrade/upgrade.sh | 70 +++ .../contributor/api_microversion_history.rst | 1 + .../image_src/plantuml/audit_state_machine.txt | 2 +- .../strategies/vm_workload_consolidation.rst | 12 +- lower-constraints.txt | 2 + .../legacy/grenade-devstack-watcher/post.yaml | 15 + playbooks/legacy/grenade-devstack-watcher/run.yaml | 60 +++ playbooks/orchestrate-tempest.yaml | 14 - playbooks/pre.yaml | 3 - .../add-baremetal-scoper-9ef23f5fb8f0be6a.yaml | 3 + ...time-for-continuous-audit-52c45052cb06d153.yaml | 5 + ...d-upgrade-check-framework-5bb9693c8a78931c.yaml | 13 + .../api-microversioning-7999a3ee8073bf32.yaml | 8 + ...a-versioned-notifications-f98361b37e546b4d.yaml | 60 +++ ...ate-ceilometer-datasource-446b0be70fbce28b.yaml | 7 + ...ce-watcher-applier-engine-86c676ce8f179e68.yaml | 16 + ...nova-legacy-notifications-e1b6d10eff58f30a.yaml | 5 + .../scope-for-data-model-ea9792f90db14343.yaml | 6 + releasenotes/source/index.rst | 1 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 231 +++++++++- releasenotes/source/rocky.rst | 6 + requirements.txt | 3 +- roles/add-hostnames-to-hosts/tasks/main.yaml | 16 - setup.cfg | 23 +- test-requirements.txt | 14 +- tox.ini | 14 +- watcher/api/app.py | 4 +- watcher/api/controllers/base.py | 84 ++++ .../api/controllers/rest_api_version_history.rst | 27 ++ watcher/api/controllers/root.py | 24 +- watcher/api/controllers/v1/__init__.py | 63 +++ watcher/api/controllers/v1/action.py | 12 + watcher/api/controllers/v1/action_plan.py | 11 + watcher/api/controllers/v1/audit.py | 83 +++- watcher/api/controllers/v1/audit_template.py | 11 + watcher/api/controllers/v1/goal.py | 11 + watcher/api/controllers/v1/scoring_engine.py | 11 + watcher/api/controllers/v1/service.py | 15 +- watcher/api/controllers/v1/strategy.py | 11 + watcher/api/controllers/v1/utils.py | 22 + watcher/api/controllers/v1/versions.py | 52 +++ watcher/api/hooks.py | 7 - watcher/api/middleware/__init__.py | 25 - watcher/api/scheduling.py | 4 +- watcher/applier/workflow_engine/base.py | 10 +- watcher/applier/workflow_engine/default.py | 29 +- watcher/cmd/status.py | 53 +++ watcher/common/clients.py | 31 +- watcher/common/context.py | 31 +- watcher/common/exception.py | 22 +- watcher/common/nova_helper.py | 45 +- watcher/common/observable.py | 57 --- watcher/common/policies/goal.py | 2 +- watcher/common/service.py | 8 +- watcher/common/synchronization.py | 22 - watcher/conf/applier.py | 4 +- watcher/conf/ceilometer_client.py | 27 +- watcher/conf/cinder_client.py | 9 +- watcher/conf/glance_client.py | 9 +- watcher/conf/gnocchi_client.py | 7 +- watcher/conf/ironic_client.py | 9 +- watcher/conf/monasca_client.py | 9 +- watcher/conf/neutron_client.py | 9 +- watcher/conf/nova_client.py | 9 +- watcher/datasource/base.py | 50 +- watcher/datasource/ceilometer.py | 20 +- watcher/datasource/gnocchi.py | 16 +- watcher/datasource/manager.py | 7 +- watcher/datasource/monasca.py | 13 +- .../versions/4b16194c56bc_add_start_end_time.py | 24 + watcher/db/sqlalchemy/models.py | 2 + watcher/decision_engine/audit/continuous.py | 24 +- watcher/decision_engine/goal/__init__.py | 2 +- watcher/decision_engine/goal/efficacy/base.py | 3 +- .../decision_engine/goal/efficacy/indicators.py | 74 ++- watcher/decision_engine/goal/efficacy/specs.py | 46 +- watcher/decision_engine/goal/goals.py | 2 +- watcher/decision_engine/model/collector/base.py | 1 + watcher/decision_engine/model/collector/cinder.py | 16 +- watcher/decision_engine/model/collector/ironic.py | 5 +- watcher/decision_engine/model/collector/nova.py | 158 ++++++- watcher/decision_engine/model/element/instance.py | 2 + watcher/decision_engine/model/element/volume.py | 2 +- watcher/decision_engine/model/model_root.py | 29 +- watcher/decision_engine/model/notification/nova.py | 354 +++------------ .../planner/workload_stabilization.py | 2 +- watcher/decision_engine/scope/baremetal.py | 31 +- watcher/decision_engine/scope/compute.py | 19 +- watcher/decision_engine/scope/storage.py | 2 + .../scoring/dummy_scoring_container.py | 2 +- .../strategy/strategies/__init__.py | 13 +- .../strategy/strategies/actuation.py | 6 +- .../decision_engine/strategy/strategies/base.py | 22 + .../strategy/strategies/basic_consolidation.py | 18 +- .../strategy/strategies/dummy_strategy.py | 2 +- .../strategy/strategies/dummy_with_resize.py | 2 +- .../strategy/strategies/dummy_with_scorer.py | 2 +- .../strategy/strategies/host_maintenance.py | 28 +- .../strategy/strategies/noisy_neighbor.py | 16 +- .../strategy/strategies/outlet_temp_control.py | 41 +- .../strategy/strategies/saving_energy.py | 23 +- .../strategies/storage_capacity_balance.py | 11 +- .../strategy/strategies/uniform_airflow.py | 41 +- .../strategies/vm_workload_consolidation.py | 72 ++- .../strategy/strategies/workload_balance.py | 40 +- .../strategy/strategies/workload_stabilization.py | 75 ++- .../strategy/strategies/zone_migration.py | 25 +- watcher/locale/de/LC_MESSAGES/watcher.po | 12 +- watcher/locale/en_GB/LC_MESSAGES/watcher.po | 55 ++- watcher/objects/audit.py | 5 +- watcher/objects/fields.py | 9 +- .../test_default_workflow_engine.py | 68 ++- .../test_taskflow_action_container.py | 5 +- .../decision_engine/audit/test_audit_handlers.py | 55 ++- .../decision_engine/cluster/test_cinder_cdmc.py | 3 +- .../decision_engine/cluster/test_nova_cdmc.py | 14 +- .../decision_engine/model/ceilometer_metrics.py | 2 + .../decision_engine/model/data/scenario_1.xml | 70 +-- .../data/scenario_1_with_1_node_unavailable.xml | 72 +-- .../data/scenario_1_with_all_nodes_disable.xml | 9 + .../model/data/scenario_1_with_metrics.xml | 4 +- .../model/data/scenario_2_with_metrics.xml | 12 +- .../model/data/scenario_3_with_2_nodes.xml | 4 +- .../model/data/scenario_3_with_metrics.xml | 8 +- .../model/data/scenario_5_with_instance_disk_0.xml | 2 +- .../model/data/scenario_6_with_2_nodes.xml | 8 +- .../model/data/scenario_7_with_2_nodes.xml | 8 +- .../model/data/scenario_8_with_4_nodes.xml | 12 +- ...nario_9_with_3_active_plus_1_disabled_nodes.xml | 12 +- .../model/data/storage_scenario_1.xml | 18 +- .../model/faker_cluster_and_metrics.py | 40 +- .../decision_engine/model/faker_cluster_state.py | 51 ++- .../notification/data/instance-create-end.json | 105 +++++ .../model/notification/data/instance-create.json | 70 --- .../notification/data/instance-delete-end.json | 2 +- ...instance-live_migration_force_complete-end.json | 98 ++++ .../instance-live_migration_post_dest-end.json | 97 ++++ .../model/notification/data/instance-lock.json | 97 ++++ .../notification/data/instance-pause-end.json | 97 ++++ .../notification/data/instance-power_off-end.json | 97 ++++ .../notification/data/instance-power_on-end.json | 97 ++++ .../notification/data/instance-rebuild-end.json | 101 +++++ .../notification/data/instance-rescue-end.json | 98 ++++ .../data/instance-resize_confirm-end.json | 97 ++++ .../notification/data/instance-restore-end.json | 97 ++++ .../notification/data/instance-resume-end.json | 97 ++++ .../notification/data/instance-shelve-end.json | 97 ++++ .../notification/data/instance-shutdown-end.json | 82 ++++ .../data/instance-soft_delete-end.json | 97 ++++ .../notification/data/instance-suspend-end.json | 97 ++++ .../model/notification/data/instance-unlock.json | 97 ++++ .../notification/data/instance-unpause-end.json | 97 ++++ .../notification/data/instance-unrescue-end.json | 97 ++++ .../notification/data/instance-unshelve-end.json | 97 ++++ .../model/notification/data/instance-update.json | 4 +- .../data/scenario3_instance-create.json | 70 --- .../data/scenario3_instance-delete-end.json | 49 -- .../data/scenario3_legacy_instance-create-end.json | 62 --- .../data/scenario3_legacy_instance-delete-end.json | 46 -- .../scenario3_legacy_instance-rebuild-end.json | 59 --- ...enario3_legacy_instance-resize-confirm-end.json | 58 --- .../data/scenario3_legacy_instance-update.json | 52 --- ...enario3_legacy_livemigration-post-dest-end.json | 61 --- .../data/scenario_1_bootable-volume-create.json | 4 +- .../data/scenario_1_error-volume-create.json | 4 +- .../data/scenario_1_volume-attach.json | 4 +- .../data/scenario_1_volume-create.json | 4 +- .../scenario_1_volume-create_pool_notfound.json | 4 +- .../data/scenario_1_volume-delete.json | 4 +- .../data/scenario_1_volume-detach.json | 4 +- .../data/scenario_1_volume-resize.json | 4 +- .../data/scenario_1_volume-update.json | 4 +- .../model/notification/data/service-create.json | 24 + .../model/notification/data/service-delete.json | 24 + .../model/notification/fake_managers.py | 11 +- .../notification/test_cinder_notifications.py | 12 +- .../model/notification/test_nova_notifications.py | 504 ++++++++++++++------- .../strategy/context/test_strategy_context.py | 15 +- .../strategy/strategies/test_actuator.py | 6 +- .../strategy/strategies/test_base.py | 70 +++ .../strategies/test_basic_consolidation.py | 93 ++-- .../strategy/strategies/test_dummy_strategy.py | 33 +- .../strategy/strategies/test_dummy_with_scorer.py | 21 +- .../strategy/strategies/test_host_maintenance.py | 125 ++--- .../strategy/strategies/test_noisy_neighbor.py | 82 +--- .../strategies/test_outlet_temp_control.py | 82 +--- .../strategy/strategies/test_saving_energy.py | 43 +- .../strategies/test_storage_capacity_balance.py | 27 +- .../strategy/strategies/test_uniform_airflow.py | 90 +--- .../strategies/test_vm_workload_consolidation.py | 150 +++--- .../strategy/strategies/test_workload_balance.py | 84 +--- .../strategies/test_workload_stabilization.py | 68 ++- .../strategy/strategies/test_zone_migration.py | 103 ++--- 258 files changed, 6926 insertions(+), 3011 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index c6e2ce93..39d429b2 100644 --- a/requirements.txt +++ b/requirements.txt @@ -24,0 +25 @@ oslo.service>=1.30.0 # Apache-2.0 +oslo.upgradecheck>=0.1.0 # Apache-2.0 @@ -48 +49 @@ networkx>=1.11 # BSD - +microversion_parse>=0.2.1 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 3a5c09a4..a6810b6a 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -8 +8 @@ freezegun>=0.3.10 # Apache-2.0 -hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 +hacking>=1.1.0,<1.2.0 # Apache-2.0 @@ -15,7 +14,0 @@ stestr>=2.0.0 # Apache-2.0 - -# Doc requirements -openstackdocstheme>=1.20.0 # Apache-2.0 -sphinx>=1.6.5,!=1.6.6,!=1.6.7 # BSD -sphinxcontrib-pecanwsme>=0.8.0 # Apache-2.0 - -# api-ref @@ -23,5 +15,0 @@ os-api-ref>=1.4.0 # Apache-2.0 - -# releasenotes -reno>=2.7.0 # Apache-2.0 - -# bandit