We are gleeful to announce the release of:
python-watcher 3.0.0: OpenStack Watcher provides a flexible and scalable resource optimization service for multi-tenant OpenStack- based clouds.
This release is part of the train release series.
The source is available from:
https://opendev.org/openstack/python-watcher
Download the package from:
https://tarballs.openstack.org/watcher/
For more details, please see below.
Changes in python-watcher 2.0.0..3.0.0 --------------------------------------
c02ddd58 Imported Translations from Zanata 7abb57dc Update TOX/UPPER_CONSTRAINTS_FILE for stable/train 963d026d Update .gitreview for stable/train 2b63a35e Fix damodel list return None error When has a compute node 48694492 Fix misspelling 42c1babf skip deleted instance when creating datamodel 519ca2c9 Fix unit test failed a88e0766 Watcher planner slector releasenote 0559cd7a Set strategy planner 0c191a2d Get planner from solution ffd8e27b Build pdf docs 31460ac4 update test about cinderclient v1 86af6ab8 correct watcher project for oslo_config 605d7f22 Add node resource consolidation planner aa2a084d Watcher Planner Selector 03a6216d Add releasenote about bp show-datamodel-api f1fe4b6c node resource consolidation 944fda33 Fix misspell word 020a99f0 Remove redundant word 'strategy' 10d8a5fb Add node_resource_consolidation doc 3c0ee0ca Add watcher-specs link to readme.rst 7eb10eee Add get node used and free resources d0a20fb0 Implement watcher datamodel list in watcher-api 49fba60e Implement watcher datamodel list in watcher-decision-engine 5aa4637f Add api-ref doc for data model api b4a02a6d add audit parameter to do_execute 8ca5d655 improve strategies tempest 6ac6ab24 add placement min_microversion d6fbc797 set compute min_microversion 26cce968 [train][goal] Define new 'watcher-tempest-functional-ipv6-only' job 99cd0098 Remove unused disk_capacity field c522e881 Don't revert Migrate action 5a26ceb1 update workload_balance strategy 33d7de12 update node resource capacity for basic_consolidation 3bcd2572 update host_maintenance strategy 60ef8776 update noisy_neighbor strategy 93b40e92 update outlet_temp_control strategy 4b2238f9 add releasenote for bp improve-compute-data-model 74209152 update vm_workload_consolidation strategy 4e4cfc95 Remove resource used fields from ComputeNode cbc2b9eb Fix var src_extra_specs error 251ad35c Remove stale comment in method execute() 0b25c884 Add resource capacity property 86d9cf17 Getting data from placement when updating datamodel 86ea9c8e replace disk_capacity by disk 6cc9ea7c set disk field to disk capacity 0986168f Check resource class before using 4b83bf33 remove id field from CDM 54344208 Update api-ref location 3d741d05 Improve Compute Data Model cadc000f Add call_retry for ModelBuilder for error recovery 361f2263 Remove useless gconfig process in watcher/api/scheduling.py 458ad086 Optimize method list_opts() in watcher/conf/opts.py d5126174 Fix watcher/conf/applier.py default worker value 904720b0 Remove useless _opts.py 933bc59b Baseclass for ModelBuilder with audit scope 433eabb8 Move datasources folder into decision_engine 0727c0e3 Add reource_name for save_energy in action input parameter field 3009716d Add get_compute_node_by_uuid 80e8d000 Resolve aggregate error in workload_stabilization 6dd35a00 Remove redundant human_id fields when creating and updating datamodel 8cb71587 Replace human_id with name in grafana doc 502ed741 Add marker option for get_instance_list() 256104a3 remove baremetal nodes when building CDM 612fc12a Add reource_name for zone_migration in action input parameter field 0541d8c2 Grafana proxy datasource to retrieve metrics dc2c361d Add reource_name in action input parameter field a3c49cf8 Add get_node_by_name 1e8b17ac Reduce the query time of the instances when call get_instance_list() ac53dbf0 remove baremetal nodes from hypversior list 8bddafbd Remove notifier_driver option in Watcher devstack 052fae4b Improve logging in building of nova data model a45f5abe Releasenote for grafana datasource cebee2c4 improve OptGroup consistency across configuration dd1800fb Blacklist sphinx 2.1.0 (autodoc bug) 148ae368 Add Python 3 Train unit tests 8fc4a9cb Fix invalid assert states 0a435f0b Add name field for test data c1a5e443 Add uWSGI support b62965c2 Add name for instance in Watcher datamodel fa1642e3 Documentation configuring grafana datasource 9c9f336f Configure nova notification_format for grenade 5f521471 Fix placement_client group help docs generation d08d7c39 Improve the configuration parameters for grafana 966a4dfa Configure nova notification format in non-grenade CI jobs dd321e9f improve the process of instance_created.end 9b8d1445 remove tail_log 90291923 Update strategy doc 06f8aa71 Implement the configuration for Grafana datasource 37b11fa4 Fix missing print format e4fc5a08 typo ceilometer url 15754a14 Replace removed exceptions and prevent regression a4d978b8 Define a new InstanceNotMapped exception 584eeefd Move datasource query_retry into baseclass. 28df60e2 Fix base enable_plugin branch for grenade run 7281f618 Remove dead code 79a57f67 Map instance to its node 731d4bfd update contraints url dd119ca1 Backwards compatibility for node parameter 00f20ab1 Fix property access in test_global_preference* tests b57feba5 Add Placement helper 251264b1 Cleanup ConfFixture 2d4bc095 Fix string formatting f9e267fa check instance state for instance.update c3e0e41f add strategy tempest job 7f37f7b9 Remove apidoc 374fd279 Optimize NovaHelper.get_compute_node_by_hostname 3f76f9cf Optimize hypervisor API calls 9c1b83e6 Add missing ws separator between words a00daf9f Group instance methods together in nova_helper 2afd0dfc Audit API supports new force option fdea38fb Optimize NovaClusterDataModelCollector.add_instance_node 5c492ea8 Fix test_metric_file_override metric from backend 62d181d9 Add force field to Audit a09cb3fa Remove 2.56 version compatibility check 7489126d Require nova_client.api_version >= 2.56 e76c20d1 Improve exceptions and logging in ds manager 5a35b307 Improve DevStack documentation to support metrics 84cb589a formal datasource interface implementation dea32c5e Improve Gnocchi and Monasca datasource tests b6200817 Allow using file to override metric map f1318256 support-keystoneclient-option f92f77f6 Fix typo in ceilometer datasource 8a206a6a Handle no nova CDM in notification code 76367afd Remove unused utilities file 6d965121 Update migration notification 86a537fe Remove bandit from lower-constraints 15c842fb Update Sphinx requirement 3edafc9b Fix Stein version in watcher-status docs efb4aaa0 Add doc/requirements.txt to venv tox target 4cd8a2f4 Remove dead code from NovaClusterDataModelCollector aa7442d7 Enhance the collector_plugins option help text 0df6c0d9 Use base_strategy's add_action_migrate method d9eb9253 Fix_inappropriate_name a40892c3 update api version history 241df0d5 allow building docs without ceilometer client dce23d7e pass default_config_dirs variable for config initialization. 14d4cb54 docs: fix link to install guide from user guide 5c2939f2 Remove watcher.openstack.common=WARN from _DEFAULT_LOG_LEVELS 76f0ce1d Add force field to api-ref 606f20b6 Fix API version header 567e0798 Remove unused exceptions 838768c7 Fix bandit runs with 1.6.0 bd8636f3 Allow for global datasources preference from config 173bf11a Use the common logging setup function in devstack runs af0f02d1 Fix reraising of exceptions e2285777 Using node replace resource_id in method add_action_disable_node() 4d480d5c Put the method add_migration() in base.py 8fa89344 update wsme types c99e051a Add tempest voting d84f8c50 Resolve problems with audit scope and add tests d2e1d69d Replace git.openstack.org with opendev.org 4db39c52 Add hardware.cpu_util in workload_stabilization 647b5e94 Drop use of git.openstack.org 62da2984 OpenDev Migration Patch 0def7b4d separate launching audit scheduler f36c25b5 Replace HOST_IP to SERVICE_HOST f52716fc remove py35 8cb4c8e4 Uncap jsonschema 2df5ab92 Fix docs gate failed 6dfeeb73 Adapt Watcher to Python3.7 ac3aa945 Move eventlet monkey patch code 68a90b2c Fix lower-constraint deps handling 7f495e8e Fix openstack-tox-lower-constraint TIMED_OUT Error 3b80b352 Update meeting schedule to new bi-weekly format c8e4efcd Make datasource methods match names of metrics b023c907 Replace openstack.org git:// URLs with https:// ab03bf9b Imported Translations from Zanata 871a01f5 Update master for stable/stein a4865b64 Migrate legacy jobs to Ubuntu Bionic
Diffstat (except docs and test files) -------------------------------------
.gitreview | 3 +- .zuul.yaml | 56 +- README.rst | 3 +- api-ref/source/index.rst | 1 + api-ref/source/parameters.yaml | 119 +++- api-ref/source/samples/audit-cancel-response.json | 1 + .../samples/audit-create-request-continuous.json | 1 + .../samples/audit-create-request-oneshot.json | 3 +- api-ref/source/samples/audit-create-response.json | 1 + .../samples/audit-list-detailed-response.json | 1 + api-ref/source/samples/audit-show-response.json | 1 + api-ref/source/samples/audit-update-response.json | 1 + .../source/samples/datamodel-list-response.json | 38 ++ api-ref/source/watcher-api-v1-audits.inc | 6 + api-ref/source/watcher-api-v1-datamodel.inc | 55 ++ devstack/lib/watcher | 88 +-- devstack/local.conf.compute | 11 +- devstack/local.conf.controller | 13 +- devstack/override-defaults | 9 + devstack/upgrade/settings | 11 +- devstack/upgrade/upgrade.sh | 13 + .../strategies/basic-server-consolidation.rst | 14 +- .../strategies/node_resource_consolidation.rst | 95 +++ .../strategies/vm_workload_consolidation.rst | 9 +- etc/apache2/watcher | 2 +- lower-constraints.txt | 16 +- playbooks/legacy/grenade-devstack-watcher/run.yaml | 10 +- .../add-force-field-to-audit-4bcaeedfe27233ad.yaml | 7 + .../notes/api-call-retry-fef741ac684c58dd.yaml | 9 + .../datasource-query-retry-00cba5f7e68aec39.yaml | 17 + .../file-based-metric-map-c2af62b5067895df.yaml | 11 + ...-interface-implementation-222769d55a127d33.yaml | 17 + ...bal-datasource-preference-3ab47b4be09ff3a5.yaml | 11 + .../notes/grafana-datasource-b672367c23ffa0c6.yaml | 10 + ...mprove-compute-data-model-b427c85e4ed2b6fb.yaml | 23 + .../min-required-nova-train-71f124192d88ae52.yaml | 8 + ...de-resource-consolidation-73bc0c0abfeb0b03.yaml | 7 + .../notes/show-datamodel-api-6945b744fd5d25d5.yaml | 12 + ...ort-keystoneclient-option-b30d1ff45f86a2e7.yaml | 5 + .../support-placement-api-58ce6bef1bbbe98a.yaml | 8 + .../notes/uwsgi-support-8dcea6961e56dad0.yaml | 12 + .../watcher-planner-selector-84d77549d46f362a.yaml | 5 + releasenotes/source/index.rst | 1 + .../locale/en_GB/LC_MESSAGES/releasenotes.po | 651 --------------------- releasenotes/source/stein.rst | 6 + requirements.txt | 20 +- setup.cfg | 8 +- test-requirements.txt | 2 +- tox.ini | 34 +- watcher/api/app.wsgi | 23 +- .../api/controllers/rest_api_version_history.rst | 5 + watcher/api/controllers/v1/__init__.py | 12 + watcher/api/controllers/v1/action.py | 8 +- watcher/api/controllers/v1/action_plan.py | 17 +- watcher/api/controllers/v1/audit.py | 51 +- watcher/api/controllers/v1/audit_template.py | 14 +- watcher/api/controllers/v1/data_model.py | 68 +++ watcher/api/controllers/v1/goal.py | 3 +- watcher/api/controllers/v1/scoring_engine.py | 3 +- watcher/api/controllers/v1/service.py | 11 +- watcher/api/controllers/v1/strategy.py | 11 +- watcher/api/controllers/v1/utils.py | 9 + watcher/api/controllers/v1/versions.py | 4 +- watcher/api/scheduling.py | 3 +- watcher/api/wsgi.py | 41 ++ watcher/applier/actions/base.py | 7 +- .../applier/actions/change_nova_service_state.py | 7 +- watcher/applier/actions/migration.py | 6 +- watcher/applier/actions/volume_migration.py | 11 +- watcher/applier/messaging/trigger.py | 4 +- watcher/applier/workflow_engine/default.py | 4 +- watcher/cmd/__init__.py | 20 + watcher/cmd/status.py | 21 +- watcher/common/clients.py | 69 ++- watcher/common/config.py | 7 +- watcher/common/exception.py | 53 +- watcher/common/nova_helper.py | 133 +++-- watcher/common/placement_helper.py | 179 ++++++ watcher/common/policies/__init__.py | 2 + watcher/common/policies/data_model.py | 37 ++ watcher/common/scheduling.py | 27 + watcher/common/service.py | 9 +- watcher/common/utils.py | 5 +- watcher/conf/__init__.py | 10 + watcher/conf/_opts.py | 58 -- watcher/conf/api.py | 2 +- watcher/conf/applier.py | 4 +- watcher/conf/ceilometer_client.py | 2 +- watcher/conf/cinder_client.py | 2 +- watcher/conf/clients_auth.py | 2 +- watcher/conf/collector.py | 22 +- watcher/conf/datasources.py | 59 ++ watcher/conf/db.py | 2 +- watcher/conf/decision_engine.py | 25 +- watcher/conf/glance_client.py | 2 +- watcher/conf/gnocchi_client.py | 13 +- watcher/conf/grafana_client.py | 146 +++++ watcher/conf/grafana_translators.py | 44 ++ watcher/conf/ironic_client.py | 2 +- watcher/conf/keystone_client.py | 38 ++ watcher/conf/monasca_client.py | 2 +- watcher/conf/neutron_client.py | 2 +- watcher/conf/nova_client.py | 15 +- watcher/conf/opts.py | 31 +- watcher/conf/placement_client.py | 41 ++ watcher/conf/planner.py | 2 +- watcher/datasource/base.py | 100 ---- watcher/datasource/ceilometer.py | 289 --------- watcher/datasource/gnocchi.py | 197 ------- watcher/datasource/manager.py | 77 --- watcher/datasource/monasca.py | 221 ------- .../versions/609bec748f2a_add_force_field.py | 22 + watcher/db/sqlalchemy/models.py | 1 + watcher/decision_engine/audit/base.py | 36 +- watcher/decision_engine/audit/continuous.py | 33 +- .../datasources}/__init__.py | 0 watcher/decision_engine/datasources/base.py | 227 +++++++ watcher/decision_engine/datasources/ceilometer.py | 273 +++++++++ watcher/decision_engine/datasources/gnocchi.py | 200 +++++++ watcher/decision_engine/datasources/grafana.py | 251 ++++++++ .../datasources/grafana_translator}/__init__.py | 0 .../datasources/grafana_translator/base.py | 125 ++++ .../datasources/grafana_translator/influxdb.py | 88 +++ watcher/decision_engine/datasources/manager.py | 156 +++++ watcher/decision_engine/datasources/monasca.py | 171 ++++++ .../decision_engine/goal/efficacy/indicators.py | 2 +- watcher/decision_engine/manager.py | 7 +- .../decision_engine/messaging/audit_endpoint.py | 4 +- .../messaging/data_model_endpoint.py | 60 ++ watcher/decision_engine/model/collector/base.py | 52 +- watcher/decision_engine/model/collector/cinder.py | 30 +- watcher/decision_engine/model/collector/ironic.py | 22 +- watcher/decision_engine/model/collector/nova.py | 342 ++++------- .../model/element/compute_resource.py | 1 - watcher/decision_engine/model/element/instance.py | 5 +- watcher/decision_engine/model/element/node.py | 22 +- watcher/decision_engine/model/element/volume.py | 2 +- watcher/decision_engine/model/model_root.py | 58 +- watcher/decision_engine/model/notification/nova.py | 172 ++++-- watcher/decision_engine/planner/manager.py | 10 +- .../planner/node_resource_consolidation.py | 163 ++++++ watcher/decision_engine/rpcapi.py | 5 + watcher/decision_engine/scope/compute.py | 21 +- watcher/decision_engine/solution/base.py | 6 +- .../decision_engine/strategy/context/default.py | 10 +- .../strategy/strategies/__init__.py | 6 +- .../strategy/strategies/actuation.py | 8 +- .../decision_engine/strategy/strategies/base.py | 92 ++- .../strategy/strategies/basic_consolidation.py | 115 ++-- .../strategy/strategies/dummy_strategy.py | 2 +- .../strategy/strategies/dummy_with_resize.py | 2 +- .../strategy/strategies/dummy_with_scorer.py | 2 +- .../strategy/strategies/host_maintenance.py | 21 +- .../strategies/node_resource_consolidation.py | 291 +++++++++ .../strategy/strategies/noisy_neighbor.py | 33 +- .../strategy/strategies/outlet_temp_control.py | 51 +- .../strategy/strategies/saving_energy.py | 37 +- .../strategies/storage_capacity_balance.py | 3 +- .../strategy/strategies/uniform_airflow.py | 81 +-- .../strategies/vm_workload_consolidation.py | 93 +-- .../strategy/strategies/workload_balance.py | 68 +-- .../strategy/strategies/workload_stabilization.py | 133 +++-- .../strategy/strategies/zone_migration.py | 47 +- watcher/hacking/checks.py | 1 + watcher/locale/de/LC_MESSAGES/watcher.po | 51 +- watcher/locale/en_GB/LC_MESSAGES/watcher.po | 45 +- watcher/objects/audit.py | 9 +- watcher/objects/utils.py | 151 ----- .../actions/test_change_nova_service_state.py | 2 +- .../test_taskflow_action_container.py | 12 +- .../decision_engine/audit/test_audit_handlers.py | 11 +- .../decision_engine/cluster/test_cinder_cdmc.py | 2 + .../decision_engine/cluster/test_nova_cdmc.py | 339 ++++++++++- .../datasources/grafana_translators/__init__.py | 0 .../datasources/grafana_translators/test_base.py | 106 ++++ .../grafana_translators/test_influxdb.py | 176 ++++++ .../datasources/test_ceilometer_helper.py | 171 ++++++ .../datasources/test_gnocchi_helper.py | 178 ++++++ .../datasources/test_grafana_helper.py | 305 ++++++++++ .../decision_engine/datasources/test_manager.py | 158 +++++ .../datasources/test_monasca_helper.py | 116 ++++ .../messaging/test_data_model_endpoint.py | 54 ++ .../decision_engine/model/ceilometer_metrics.py | 211 ++++--- .../decision_engine/model/data/scenario_1.xml | 80 +-- .../decision_engine/model/data/scenario_10.xml | 27 + .../data/scenario_1_with_1_node_unavailable.xml | 84 +-- .../data/scenario_1_with_all_instances_exclude.xml | 80 +-- .../data/scenario_1_with_all_nodes_disable.xml | 8 +- .../model/data/scenario_1_with_metrics.xml | 8 +- .../model/data/scenario_2_with_metrics.xml | 20 +- .../model/data/scenario_3_with_2_nodes.xml | 8 +- .../model/data/scenario_3_with_metrics.xml | 12 +- .../data/scenario_4_with_1_node_no_instance.xml | 2 +- .../model/data/scenario_5_with_instance_disk_0.xml | 4 +- .../model/data/scenario_6_with_2_nodes.xml | 12 +- .../model/data/scenario_7_with_2_nodes.xml | 12 +- .../model/data/scenario_8_with_4_nodes.xml | 20 +- ...nario_9_with_3_active_plus_1_disabled_nodes.xml | 20 +- .../model/faker_cluster_and_metrics.py | 104 ++-- .../decision_engine/model/faker_cluster_state.py | 10 + ....json => instance-live_migration_post-end.json} | 2 +- .../data/scenario3_service-update-disabled.json | 2 +- .../data/scenario3_service-update-enabled.json | 2 +- .../model/notification/data/service-create.json | 2 +- .../model/notification/data/service-delete.json | 2 +- .../model/notification/data/service-update.json | 26 +- .../model/notification/test_nova_notifications.py | 193 ++++-- .../planner/test_node_resource_consolidation.py | 245 ++++++++ .../planner/test_planner_manager.py | 4 +- .../solution/test_default_solution.py | 9 +- .../strategy/strategies/test_base.py | 70 +++ .../strategies/test_basic_consolidation.py | 84 +-- .../strategy/strategies/test_host_maintenance.py | 31 +- .../strategies/test_node_resource_consolidation.py | 345 +++++++++++ .../strategy/strategies/test_noisy_neighbor.py | 2 +- .../strategies/test_outlet_temp_control.py | 14 +- .../strategy/strategies/test_saving_energy.py | 78 +-- .../strategies/test_vm_workload_consolidation.py | 84 +-- .../strategy/strategies/test_workload_balance.py | 16 +- .../strategies/test_workload_stabilization.py | 125 ++-- .../strategy/strategies/test_zone_migration.py | 158 +++-- 266 files changed, 9304 insertions(+), 4679 deletions(-)
Requirements updates --------------------
diff --git a/requirements.txt b/requirements.txt index 39d429b2..cc65597b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9 +9 @@ keystoneauth1>=3.4.0 # Apache-2.0 -jsonschema<3.0.0,>=2.6.0 # MIT +jsonschema>=2.6.0 # MIT @@ -12,0 +13 @@ croniter>=0.3.20 # MIT License +os-resource-classes>=0.4.0 @@ -16 +17 @@ oslo.config>=5.2.0 # Apache-2.0 -oslo.context>=2.20.0 # Apache-2.0 +oslo.context>=2.21.0 # Apache-2.0 @@ -20 +21 @@ oslo.log>=3.37.0 # Apache-2.0 -oslo.messaging>=5.36.0 # Apache-2.0 +oslo.messaging>=8.1.2 # Apache-2.0 @@ -30 +31 @@ pbr>=3.1.1 # Apache-2.0 -pecan>=1.2.1 # BSD +pecan>=1.3.2 # BSD @@ -39 +40 @@ python-neutronclient>=6.7.0 # Apache-2.0 -python-novaclient>=10.1.0 # Apache-2.0 +python-novaclient>=14.1.0 # Apache-2.0 @@ -41 +42 @@ python-openstackclient>=3.14.0 # Apache-2.0 -python-ironicclient>=2.3.0 # Apache-2.0 +python-ironicclient>=2.5.0 # Apache-2.0 @@ -46 +47 @@ taskflow>=3.1.0 # Apache-2.0 -WebOb>=1.7.4 # MIT +WebOb>=1.8.5 # MIT @@ -48 +49,3 @@ WSME>=0.9.2 # MIT -networkx>=1.11 # BSD +# NOTE(fdegir): NetworkX 2.3 dropped support for Python 2 +networkx>=1.11,<2.3;python_version<'3.0' # BSD +networkx>=1.11;python_version>='3.4' # BSD @@ -49,0 +53 @@ microversion_parse>=0.2.1 # Apache-2.0 +futurist>=1.8.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index a6810b6a..1a5854f6 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -16 +16 @@ os-api-ref>=1.4.0 # Apache-2.0 -bandit>=1.1.0 # Apache-2.0 +bandit>=1.6.0 # Apache-2.0