We high-spiritedly announce the release of: python-watcher 0.34.0: OpenStack Watcher provides a flexible and scalable resource optimization service for multi-tenant OpenStack- based clouds. This release is part of the ocata release series. The source is available from: https://github.com/openstack/watcher Download the package from: https://tarballs.openstack.org/watcher/ Please report issues through launchpad: http://bugs.launchpad.net/watcher For more details, please see below. 0.34.0 ^^^^^^ New Features * Add notifications related to Audit object. * Watcher can continuously optimize the OpenStack cloud for a specific strategy or goal by triggering an audit periodically which generates an action plan and run it automatically. * Centralize all configuration options for Watcher. * Watcher database can now be upgraded thanks to Alembic. * Provides a generic way to define the scope of an audit. The set of audited resources will be called "Audit scope" and will be defined in each audit template (which contains the audit settings). * The graph model describes how VMs are associated to compute hosts. This allows for seeing relationships upfront between the entities and hence can be used to identify hot/cold spots in the data center and influence a strategy decision. * Watcher supports multiple metrics backend and relies on Ceilometer and Monasca. * Watcher can now run specific actions in parallel improving the performances dramatically when executing an action plan. * Add superseded state for an action plan if the cluster data model has changed after it has been created. * Provide a notification mechanism into Watcher that supports versioning. Whenever a Watcher object is created, updated or deleted, a versioned notification will, if it's relevant, be automatically sent to notify in order to allow an event-driven style of architecture within Watcher. Moreover, it will also give other services and/or 3rd party softwares (e.g. monitoring solutions or rules engines) the ability to react to such events. * Add a service supervisor to watch Watcher deamons. * all Watcher objects have been refactored to support OVO (oslo.versionedobjects) which was a prerequisite step in order to implement versioned notifications. Changes in python-watcher 0.33.0..0.34.0 ---------------------------------------- 4f1d758 add Ocata release notes e03f56e Add period input parameter to basic strategy d925166 Fix invalid mock on ceilometerclient e9d8a28 Updated from global requirements 6e09cdb New Applier Workflow Engine edd3d21 Remove obsolete Resource element d433d6b Graph cluster model instead of mapping one c5d4f9c Fix building of model with a scoped exclusion rule 41f579d Fix broken gates because of wrong pip command 2d7a375 Fix test_clients_monasca failure 45b82e1 Updated from global requirements 0e440d3 New default planner 23442a4 Modify the field in tox.ini a48a165 Add action plan SUPERSEDED state 8f4a856 Fix dummy strategy to use input parameters 85a46ce Updated from global requirements 35066df Update Server Consolidation global efficacy 5e077f3 Fix a typo in watcher/objects/base.py 2ec8bc1 resolve KeyError exception 2dd00a2 Enable notification for vm task state update 59e13fd Should use glanceclient to get images 4235ef7 Multi datasource support for Basic Consolidation a015af1 Added Monasca Helper dad90b6 Removed unnecessary utf-8 encoding 1341c0e Updated from global requirements 6e99fcf Enable coverage report in console output 0adc7d9 Fix TypeError if no input_parameters added 0d7ded0 Update configuration document 80dfbd6 Add additional depencencies of CentOS 7 7d40b3d Fix reference http 87b494d remove incorrect inline comment d0bca1f Add auto_trigger support to watcher 068178f Fix variable name error 99e6c4a Updated from global requirements 48fc90d Updated from global requirements 1bc6b0e Fix bad CDMC update on reception of service.update 1a17c4b remove unused log c4dfbd5 Repair log parameter error 1981f39 multinode devstack update for live-migration d792e3c Function call pass parameter error f66eb46 Improve the instruction of vm_workload_consolidation. 6a323ed Fix method name in doc/source/dev/plugin/action-plugin.rst d252d47 Fix some incorrect description in doc. Diffstat (except docs and test files) ------------------------------------- devstack/lib/watcher | 1 + devstack/local.conf.compute | 3 + devstack/local.conf.controller | 3 + .../strategies/basic-server-consolidation.rst | 3 + .../strategies/vm_workload_consolidation.rst | 2 +- etc/watcher/README-watcher.conf.txt | 2 +- ...rsioned-notifications-api-bca7738e16954bad.yaml | 3 + ...utomatic-triggering-audit-8a9b0540d547db60.yaml | 5 + .../centralise-config-opts-95670987dfbdb0e7.yaml | 3 + .../notes/db-migration-e1a705a8b54ccdd2.yaml | 3 + .../define-the-audit-scope-e89edc5051dcf3f2.yaml | 5 + ...graph-based-cluster-model-523937a6f5e66537.yaml | 6 + .../notes/monasca-support-0b0486b8572ac38b.yaml | 4 + ...anner-storage-action-plan-26ef37893c5e8648.yaml | 4 + .../notes/stale-action-plan-b6a6b08df873c128.yaml | 4 + ...watcher-notifications-ovo-7b44d52ef6400dd0.yaml | 8 + .../watcher-service-list-7b2f4b64f71e9b89.yaml | 3 + ...watcher-versioned-objects-fc5abf5c81c4590c.yaml | 5 + releasenotes/source/conf.py | 2 - requirements.txt | 15 +- setup.cfg | 5 +- tox.ini | 10 +- watcher/__init__.py | 2 - watcher/api/controllers/v1/action.py | 52 +- watcher/api/controllers/v1/action_plan.py | 30 +- watcher/api/controllers/v1/audit.py | 11 +- watcher/api/controllers/v1/types.py | 2 - watcher/applier/actions/resize.py | 106 +++ watcher/applier/workflow_engine/default.py | 50 +- watcher/common/ceilometer_helper.py | 179 ---- watcher/common/clients.py | 38 +- watcher/common/exception.py | 15 + watcher/common/nova_helper.py | 90 +- watcher/conf/__init__.py | 2 + watcher/conf/applier.py | 2 +- watcher/conf/monasca_client.py | 36 + watcher/conf/planner.py | 2 +- watcher/datasource/__init__.py | 0 watcher/datasource/ceilometer.py | 179 ++++ watcher/datasource/monasca.py | 124 +++ watcher/db/sqlalchemy/api.py | 6 + watcher/db/sqlalchemy/models.py | 5 +- watcher/decision_engine/audit/base.py | 27 +- watcher/decision_engine/audit/continuous.py | 3 - watcher/decision_engine/audit/oneshot.py | 7 + watcher/decision_engine/cluster/__init__.py | 0 .../decision_engine/cluster/history/__init__.py | 0 watcher/decision_engine/cluster/history/base.py | 80 -- .../decision_engine/cluster/history/ceilometer.py | 45 - .../decision_engine/goal/efficacy/indicators.py | 14 + watcher/decision_engine/goal/efficacy/specs.py | 7 +- watcher/decision_engine/model/base.py | 4 + watcher/decision_engine/model/collector/base.py | 5 + watcher/decision_engine/model/collector/nova.py | 370 ++++++-- watcher/decision_engine/model/element/__init__.py | 12 +- watcher/decision_engine/model/element/base.py | 40 +- .../model/element/compute_resource.py | 36 +- watcher/decision_engine/model/element/disk_info.py | 59 -- watcher/decision_engine/model/element/instance.py | 22 +- watcher/decision_engine/model/element/node.py | 35 +- watcher/decision_engine/model/element/resource.py | 63 -- watcher/decision_engine/model/mapping.py | 101 --- watcher/decision_engine/model/model_root.py | 242 +++--- watcher/decision_engine/model/notification/nova.py | 141 +-- watcher/decision_engine/planner/default.py | 169 ---- watcher/decision_engine/planner/weight.py | 233 +++++ .../planner/workload_stabilization.py | 301 +++++++ watcher/decision_engine/scope/default.py | 46 +- watcher/decision_engine/solution/default.py | 3 + .../strategy/strategies/basic_consolidation.py | 229 +++-- .../strategy/strategies/dummy_strategy.py | 4 +- .../strategy/strategies/dummy_with_resize.py | 121 +++ .../strategy/strategies/outlet_temp_control.py | 74 +- .../strategy/strategies/uniform_airflow.py | 81 +- .../strategies/vm_workload_consolidation.py | 227 +++-- .../strategy/strategies/workload_balance.py | 86 +- .../strategy/strategies/workload_stabilization.py | 103 +-- watcher/objects/action.py | 5 +- watcher/objects/action_plan.py | 5 +- watcher/objects/audit.py | 4 +- watcher/objects/base.py | 2 +- watcher/objects/fields.py | 8 +- .../action_plan/test_default_action_handler.py | 1 - .../actions/loading/test_default_actions_loader.py | 1 - .../actions/test_change_nova_service_state.py | 1 - .../test_default_workflow_engine.py | 97 ++- .../decision_engine/audit/test_audit_handlers.py | 61 ++ .../decision_engine/cluster/test_nova_cdmc.py | 24 +- .../decision_engine/model/ceilometer_metrics.py | 260 ++++++ .../decision_engine/model/data/scenario_1.xml | 80 +- .../model/data/scenario_1_with_metrics.xml | 8 +- .../model/data/scenario_2_with_metrics.xml | 20 +- .../model/data/scenario_3_with_2_nodes.xml | 8 +- .../model/data/scenario_3_with_metrics.xml | 12 +- .../data/scenario_4_with_1_node_no_instance.xml | 2 +- .../model/data/scenario_5_with_instance_disk_0.xml | 4 +- .../model/data/scenario_6_with_2_nodes.xml | 12 +- .../model/data/scenario_7_with_2_nodes.xml | 12 +- .../model/data/scenario_8_with_4_nodes.xml | 20 +- ...nario_9_with_3_active_plus_1_disabled_nodes.xml | 20 +- .../model/faker_cluster_and_metrics.py | 28 +- .../decision_engine/model/faker_cluster_state.py | 104 +-- .../model/faker_metrics_collector.py | 255 ------ .../data/scenario3_service-update-disabled.json | 21 + .../data/scenario3_service-update-enabled.json | 21 + .../data/scenario3_service-update.json | 21 - .../model/notification/test_nova_notifications.py | 165 ++-- .../planner/test_default_planner.py | 206 ----- .../planner/test_planner_manager.py | 6 +- .../decision_engine/planner/test_weight_planner.py | 943 +++++++++++++++++++++ .../planner/test_workload_stabilization_planner.py | 378 +++++++++ .../solution/test_default_solution.py | 15 + .../strategies/test_basic_consolidation.py | 71 +- .../strategies/test_outlet_temp_control.py | 14 +- .../strategy/strategies/test_uniform_airflow.py | 17 +- .../strategies/test_vm_workload_consolidation.py | 48 +- .../strategy/strategies/test_workload_balance.py | 15 +- .../strategies/test_workload_stabilization.py | 28 +- watcher/version.py | 1 - 161 files changed, 5498 insertions(+), 3021 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 8ac1edd..08bb48e 100644 --- a/requirements.txt +++ b/requirements.txt @@ -8,3 +8,3 @@ jsonpatch>=1.1 # BSD -keystoneauth1>=2.14.0 # Apache-2.0 -keystonemiddleware!=4.5.0,>=4.2.0 # Apache-2.0 -lxml>=2.3 # BSD +keystoneauth1>=2.18.0 # Apache-2.0 +keystonemiddleware>=4.12.0 # Apache-2.0 +lxml!=3.7.0,>=2.3 # BSD @@ -15 +15 @@ oslo.context>=2.9.0 # Apache-2.0 -oslo.db!=4.13.1,!=4.13.2,>=4.11.0 # Apache-2.0 +oslo.db>=4.15.0 # Apache-2.0 @@ -33,0 +34 @@ python-keystoneclient>=3.8.0 # Apache-2.0 +python-monascaclient>=1.1.0 # Apache-2.0 @@ -35 +36 @@ python-neutronclient>=5.1.0 # Apache-2.0 -python-novaclient!=2.33.0,>=2.29.0 # Apache-2.0 +python-novaclient!=7.0.0,>=6.0.0 # Apache-2.0 @@ -40 +41 @@ stevedore>=1.17.1 # Apache-2.0 -taskflow>=1.26.0 # Apache-2.0 +taskflow>=2.7.0 # Apache-2.0 @@ -42,0 +44,2 @@ WSME>=0.8 # MIT +networkx>=1.10 # BSD +