[openstack-announce] [new][watcher] python-watcher 0.29.0 release (newton)

no-reply at openstack.org no-reply at openstack.org
Wed Aug 31 10:46:05 UTC 2016


We are frolicsome to announce the release of:

python-watcher 0.29.0: Watcher takes advantage of CEP and ML
algorithms/metaheuristics to improve physical resources usage through
better VM placement. Watcher can improve your cloud optimization by
reducing energy footprint and increasing profits.

This release is part of the newton release series.

With source available at:

    https://github.com/openstack/watcher

Please report issues through launchpad:

    http://bugs.launchpad.net/watcher

For more details, please see below.

0.29.0
^^^^^^

New Features

* Added a standard way to both declare and fetch configuration
  options so that whenever the administrator generates the Watcher
  configuration sample file, it contains the configuration options of
  the plugins that are currently available.

* Added a generic scoring engine module, which will standarize
  interactions with scoring engines through the common API. It is
  possible to use the scoring engine by different Strategies, which
  improve the code and data model re-use.

* Added an in-memory cache of the cluster model built up and kept
  fresh via notifications from services of interest in addition to
  periodic syncing logic.

* Added a way to add a new action without having to amend the source
  code of the default planner.

* Added a way to create periodic audit to be able to optimize
  continuously the cloud infrastructure.

* Added a way to compare the efficacy of different strategies for a
  give optimization goal.

* Added a way to return the of available goals depending on which
  strategies have been deployed on the node where the decison engine
  is running.

* Allow decision engine to pass strategy parameters, like
  optimization threshold, to selected strategy, also strategy to
  provide parameters info to end user.

* Copy all audit templates parameters into audit instead of having a
  reference to the audit template.

* Added a strategy that monitors if there is a higher load on some
  hosts compared to other hosts in the cluster and re-balances the
  work across hosts to minimize the standard deviation of the loads in
  the cluster.

* Added a new strategy based on the airflow of servers. This
  strategy makes decisions to migrate VMs to make the airflow uniform.

* Added policies to handle user rights to access Watcher API.

* Added a strategy based on the VM workloads of hypervisors. This
  strategy makes decisions to migrate workloads to make the total VM
  workloads of each hypervisor balanced, when the total VM workloads
  of hypervisor reaches threshold.

Changes in python-watcher 0.28.0..0.29.0
----------------------------------------

6e8dc52 Merge scoring base files
7cce4b9 Add release notes for Newton blueprints
deb5cb3 TrivialFix: Remove cfg import unused
720884c TrivialFix: Remove logging import unused
249cd11 Remove unused LOG
e4ba59e Update configuration section for notifications
f238167 Doc on how to add notification endpoints
77b7fae Notification and CDM partial update
103f541 Remove unreachable line
c617938 Added start/end date params on ceilometer queries
ecea228 Correct watcher reraising of exception
1fafcc5 Check unspecified parameters create audit
32c13d0 Fix loading of plugin configuration parameters
a1cb142 Add Scoring Module implementation
da67b24 Add unit tests for continuous.py
6d41c23 Updated from global requirements
7b22834 Fixed flaky tempest test
e741728 Remove pot files
35201c8 Updated from global requirements
6be758b Added strategy ID + Action Plan syncing
9450a70 Fixes to get cluster data model
64f45ad Updated from global requirements
4e3593a Fix double self._goal definition
01164b0 Scheduler of decision_engine fix
9933955 Updated from global requirements
17d2d75 Clean imports in code
2e55f4e Modify libvirt_opts="-d -l" to libvirtd_opts="-d -l"
ea01031 Rename (pre/post)condition to (pre/post)_condition
6144551 Add unit tests for nova_helper.py
1b2672a Updated from global requirements
31de0e3 Removed unused function in uniform airflow
8145906 Update the home-page info with the developer documentation
9d2d218 Updated from global requirements
31c3734 Refactored the compute model and its elements
cc2e805 Use more specific asserts
0a6841f Merged metrics_engine package into decision_engine
4f8591c Updated DE architecture doc + 'period' param
06c6c46 Added DE Background Scheduler w/ model sync jobs
b94677c Cluster data model collector plugin documentation
5a2a94f Loadable Cluster Data Model Collectors
61e581f Updated from global requirements
26d84e3 Add scoring engines to database and API layers
52ffc2c Implement goal_id, strategy_id and host_aggregate into Audit api
5dee934 use parameters to set the threshold
0769e53 Updated from global requirements
107bd0b Fixed Basic optim tempest test
926b790 Fix 2 occurrences of typo: "occured" --> "occurred"
7d704db Add hacking checks to watcher
5c08095 Update docs links to docs.openstack.org
b579a41 Remove discover from test-requirements
051810d Updated from global requirements
6730202 Fix typos and messages in strategies
fd7c41f Remove unused columns parameters in watcher/db/api
aef1eba test_context_hook_before_method failed
cd60336 Fix dict.keys() PY3 compatible
e7a1ba7 There are some spelling errors in the code.
1b0801a Add Python 3.5 classifier and venv
ff43753 Add installation from Debian packages section
21d1610 Update executor to eventlet


Diffstat (except docs and test files)
-------------------------------------

.coveragerc                                        |   4 +-
devstack/lib/watcher                               |   5 +
.../plantuml/sequence_architecture_cdmc_sync.txt   |  41 +
.../sequence_trigger_audit_in_decision_engine.txt  |  54 +-
.../images/sequence_architecture_cdmc_sync.png     | Bin 0 -> 46795 bytes
.../sequence_trigger_audit_in_decision_engine.png  | Bin 83923 -> 73345 bytes
etc/watcher/policy.json                            |   4 +
.../add-plugins-parameters-376eb6b0b8978b44.yaml   |   8 +
.../notes/add-scoring-module-fa00d013ed2d614e.yaml |   7 +
...ter-model-objects-wrapper-9c799ea262c56a5b.yaml |   6 +
...e-weights-default-planner-3746b33160bc7347.yaml |   4 +
...continuously-optimization-35364f4d2c0b81fc.yaml |   4 +
.../notes/efficacy-indicator-95380ad7b84e3be2.yaml |   4 +
.../get-goal-from-strategy-396c9b13a38bb650.yaml   |   5 +
.../optimization-threshold-21ad38f0470d0e1a.yaml   |   5 +
...rsistent-audit-parameters-ae41dd7252ba9672.yaml |   6 +
...andard-deviation-strategy-cd1d0c443fdfde9c.yaml |   7 +
.../uniform-airflow-strategy-68cdba1419c3f770.yaml |   5 +
.../notes/watcher-policies-1e86a30f0f11c6fa.yaml   |   4 +
...alance-migration-strategy-a0b05148a57815c0.yaml |   7 +
requirements.txt                                   |  23 +-
setup.cfg                                          |  14 +-
test-requirements.txt                              |   3 +-
tox.ini                                            |   3 +-
watcher/api/controllers/v1/__init__.py             |  13 +
watcher/api/controllers/v1/action.py               |   4 +-
watcher/api/controllers/v1/action_plan.py          |  99 ++-
watcher/api/controllers/v1/audit.py                | 286 ++++---
watcher/api/controllers/v1/collection.py           |   4 +-
watcher/api/controllers/v1/goal.py                 |   4 -
watcher/api/controllers/v1/scoring_engine.py       | 246 ++++++
watcher/api/controllers/v1/strategy.py             |   4 -
watcher/api/controllers/v1/types.py                |   4 +-
watcher/api/middleware/parsable_error.py           |   5 +-
watcher/applier/action_plan/default.py             |  10 +-
watcher/applier/actions/base.py                    |   8 +-
.../applier/actions/change_nova_service_state.py   |  17 +-
watcher/applier/actions/migration.py               |  60 +-
watcher/applier/actions/nop.py                     |   8 +-
watcher/applier/actions/sleep.py                   |  10 +-
watcher/applier/manager.py                         |   4 +-
watcher/applier/rpcapi.py                          |   7 +-
watcher/applier/workflow_engine/base.py            |   3 +-
watcher/applier/workflow_engine/default.py         |  24 +-
watcher/cmd/api.py                                 |   3 +-
watcher/cmd/applier.py                             |   5 +-
watcher/cmd/dbmanage.py                            |   6 +-
watcher/cmd/decisionengine.py                      |   9 +-
watcher/cmd/sync.py                                |  39 +
watcher/common/ceilometer_helper.py                |  55 +-
watcher/common/context.py                          | 118 ++-
watcher/common/exception.py                        |  37 +-
watcher/common/loader/default.py                   |   2 +-
watcher/common/loader/loadable.py                  |  31 +
watcher/common/messaging/messaging_handler.py      |   2 +-
watcher/common/nova_helper.py                      |  84 +-
watcher/common/rpc.py                              |  41 +-
watcher/common/scheduling.py                       |  44 +
watcher/common/service.py                          |  88 +-
watcher/common/utils.py                            |  33 +-
watcher/db/api.py                                  |  85 +-
watcher/db/purge.py                                |  58 +-
watcher/db/sqlalchemy/api.py                       | 105 ++-
watcher/db/sqlalchemy/models.py                    |  34 +-
watcher/decision_engine/audit/base.py              |   3 +-
watcher/decision_engine/audit/continuous.py        |  10 +-
watcher/decision_engine/audit/oneshot.py           |   4 +-
watcher/decision_engine/cluster/__init__.py        |   0
.../decision_engine/cluster/history/__init__.py    |   0
watcher/decision_engine/cluster/history/base.py    |  80 ++
.../decision_engine/cluster/history/ceilometer.py  |  45 +
watcher/decision_engine/goal/base.py               |   4 -
watcher/decision_engine/goal/efficacy/base.py      |   4 +-
.../decision_engine/goal/efficacy/indicators.py    |   8 +-
watcher/decision_engine/goal/efficacy/specs.py     |   6 +-
watcher/decision_engine/goal/goals.py              |  20 +-
watcher/decision_engine/loading/default.py         |  20 +
watcher/decision_engine/manager.py                 |  33 +-
.../decision_engine/messaging/audit_endpoint.py    |   1 +
.../decision_engine/model/collector/__init__.py    |   0
watcher/decision_engine/model/collector/base.py    | 178 ++++
watcher/decision_engine/model/collector/manager.py |  61 ++
watcher/decision_engine/model/collector/nova.py    | 113 +++
watcher/decision_engine/model/compute_resource.py  |  50 --
watcher/decision_engine/model/disk_info.py         |  53 --
watcher/decision_engine/model/element/__init__.py  |  38 +
watcher/decision_engine/model/element/base.py      |  29 +
.../model/element/compute_resource.py              |  57 ++
watcher/decision_engine/model/element/disk_info.py |  59 ++
watcher/decision_engine/model/element/instance.py  |  54 ++
watcher/decision_engine/model/element/node.py      |  53 ++
watcher/decision_engine/model/element/resource.py  |  61 ++
watcher/decision_engine/model/hypervisor.py        |  51 --
watcher/decision_engine/model/hypervisor_state.py  |  24 -
watcher/decision_engine/model/mapping.py           | 118 ++-
watcher/decision_engine/model/model_root.py        | 151 +++-
.../decision_engine/model/notification/__init__.py |   0
watcher/decision_engine/model/notification/base.py |  47 ++
.../model/notification/filtering.py                |  88 ++
watcher/decision_engine/model/notification/nova.py | 334 ++++++++
watcher/decision_engine/model/power_state.py       |  31 -
watcher/decision_engine/model/resource.py          |  58 --
watcher/decision_engine/model/sla/__init__.py      |   0
watcher/decision_engine/model/vm.py                |  31 -
watcher/decision_engine/model/vm_state.py          |  34 -
watcher/decision_engine/planner/default.py         |  18 +-
watcher/decision_engine/rpcapi.py                  |   2 +
watcher/decision_engine/scheduling.py              |  92 +++
watcher/decision_engine/scoring/__init__.py        |   0
watcher/decision_engine/scoring/base.py            | 126 +++
watcher/decision_engine/scoring/dummy_scorer.py    | 169 ++++
.../scoring/dummy_scoring_container.py             |  98 +++
watcher/decision_engine/scoring/scoring_factory.py | 106 +++
watcher/decision_engine/strategy/context/base.py   |  12 +-
.../decision_engine/strategy/context/default.py    |  22 +-
.../decision_engine/strategy/selection/default.py  |   2 -
.../strategy/strategies/__init__.py                |   5 +-
.../decision_engine/strategy/strategies/base.py    |  23 +-
.../strategy/strategies/basic_consolidation.py     | 300 ++++---
.../strategy/strategies/dummy_strategy.py          |   8 +-
.../strategy/strategies/dummy_with_scorer.py       | 166 ++++
.../strategy/strategies/outlet_temp_control.py     | 143 ++--
.../strategy/strategies/uniform_airflow.py         | 309 +++----
.../strategies/vm_workload_consolidation.py        | 390 +++++----
.../strategy/strategies/workload_balance.py        | 247 +++---
.../strategy/strategies/workload_stabilization.py  | 233 +++---
watcher/decision_engine/sync.py                    | 258 +++++-
watcher/doc.py                                     |  69 --
watcher/hacking/__init__.py                        |   0
watcher/hacking/checks.py                          | 309 +++++++
watcher/locale/watcher.pot                         | 909 ---------------------
watcher/metrics_engine/__init__.py                 |   0
watcher/metrics_engine/cluster_history/__init__.py |   0
watcher/metrics_engine/cluster_history/base.py     |  80 --
.../metrics_engine/cluster_history/ceilometer.py   |  49 --
.../cluster_model_collector/__init__.py            |   0
.../metrics_engine/cluster_model_collector/base.py | 119 ---
.../cluster_model_collector/manager.py             |  34 -
.../metrics_engine/cluster_model_collector/nova.py |  81 --
watcher/objects/__init__.py                        |   4 +-
watcher/objects/action_plan.py                     |   3 +-
watcher/objects/audit.py                           |   4 +-
watcher/objects/base.py                            |  10 +-
watcher/objects/scoring_engine.py                  | 257 ++++++
watcher/opts.py                                    |   3 +
.../action_plan/test_default_action_handler.py     |  18 +-
.../actions/test_change_nova_service_state.py      |  26 +-
.../test_default_workflow_engine.py                |   4 +-
.../common/messaging/test_messaging_handler.py     |   1 +
.../decision_engine/audit/test_audit_handlers.py   |  42 +-
.../cluster/test_cluster_data_model_collector.py   |  54 ++
.../decision_engine/cluster/test_nova_cdmc.py      |  82 ++
.../loading/test_collector_loader.py               |  83 ++
.../loading/test_default_planner_loader.py         |  31 +
.../loading/test_default_strategy_loader.py        |  78 ++
.../decision_engine/loading/test_goal_loader.py    |  78 ++
.../messaging/test_audit_endpoint.py               |   2 +-
.../decision_engine/model/notification/__init__.py |   0
.../model/notification/data/instance-create.json   |  70 ++
.../notification/data/instance-delete-end.json     |  49 ++
.../model/notification/data/instance-update.json   |  65 ++
.../data/scenario3_instance-create.json            |  70 ++
.../data/scenario3_instance-delete-end.json        |  49 ++
.../data/scenario3_instance-update.json            |  65 ++
.../data/scenario3_legacy_instance-create-end.json |  62 ++
.../data/scenario3_legacy_instance-delete-end.json |  46 ++
.../data/scenario3_legacy_instance-update.json     |  52 ++
...enario3_legacy_livemigration-post-dest-end.json |  61 ++
.../data/scenario3_service-update.json             |  21 +
.../model/notification/data/service-update.json    |  21 +
.../model/notification/fake_managers.py            |  53 ++
.../model/notification/test_notifications.py       | 106 +++
.../model/notification/test_nova_notifications.py  | 450 ++++++++++
.../decision_engine/model/test_named_element.py    |  32 -
.../planner/test_default_planner.py                |  79 +-
.../planner/test_default_planner_loader.py         |  31 -
.../decision_engine/scoring/test_dummy_scorer.py   |  54 ++
.../scoring/test_dummy_scoring_container.py        |  51 ++
.../scoring/test_scoring_factory.py                |  53 ++
.../solution/test_default_solution.py              |  11 +-
.../strategy/context/test_strategy_context.py      |  28 +-
.../decision_engine/strategy/loading/__init__.py   |   0
.../loading/test_default_strategy_loader.py        |  78 --
.../strategy/selector/test_strategy_selector.py    |   3 -
.../strategies/faker_cluster_and_metrics.py        | 268 +++---
.../strategy/strategies/faker_cluster_state.py     | 431 +++++-----
.../strategy/strategies/faker_metrics_collector.py |  73 +-
.../strategies/test_basic_consolidation.py         | 120 +--
.../strategy/strategies/test_dummy_strategy.py     |   4 +-
.../strategy/strategies/test_dummy_with_scorer.py  |  61 ++
.../strategies/test_outlet_temp_control.py         |  62 +-
.../strategy/strategies/test_uniform_airflow.py    | 123 +--
.../strategies/test_vm_workload_consolidation.py   | 172 ++--
.../strategy/strategies/test_workload_balance.py   | 106 +--
.../strategies/test_workload_stabilization.py      |  29 +-
.../services/infra_optim/v1/json/client.py         |   8 +-
261 files changed, 10240 insertions(+), 4919 deletions(-)


Requirements updates
--------------------

diff --git a/requirements.txt b/requirements.txt
index 74cb8d3..774b848 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -8 +8 @@ jsonpatch>=1.1 # BSD
-keystoneauth1>=2.7.0 # Apache-2.0
+keystoneauth1>=2.10.0 # Apache-2.0
@@ -12,3 +12,3 @@ oslo.cache>=1.5.0 # Apache-2.0
-oslo.config>=3.10.0 # Apache-2.0
-oslo.context>=2.4.0 # Apache-2.0
-oslo.db>=4.1.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.context>=2.9.0 # Apache-2.0
+oslo.db>=4.10.0 # Apache-2.0
@@ -19,0 +20 @@ oslo.reports>=0.6.0 # Apache-2.0
+oslo.serialization>=1.10.0 # Apache-2.0
@@ -21 +22 @@ oslo.service>=1.10.0 # Apache-2.0
-oslo.utils>=3.14.0 # Apache-2.0
+oslo.utils>=3.16.0 # Apache-2.0
@@ -24 +25 @@ pbr>=1.6 # Apache-2.0
-pecan>=1.0.0 # BSD
+pecan!=1.0.2,!=1.0.3,!=1.0.4,>=1.0.0 # BSD
@@ -27 +28 @@ voluptuous>=0.8.9 # BSD License
-python-ceilometerclient>=2.2.1 # Apache-2.0
+python-ceilometerclient>=2.5.0 # Apache-2.0
@@ -29,3 +30,3 @@ python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0
-python-glanceclient>=2.0.0 # Apache-2.0
-python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # Apache-2.0
-python-neutronclient>=4.2.0 # Apache-2.0
+python-glanceclient!=2.4.0,>=2.3.0 # Apache-2.0
+python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
+python-neutronclient>=5.1.0 # Apache-2.0
@@ -36 +37 @@ SQLAlchemy<1.1.0,>=1.0.10 # MIT
-stevedore>=1.10.0 # Apache-2.0
+stevedore>=1.16.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 27e4a7f..0ccc117 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -6 +5,0 @@ coverage>=3.6 # Apache-2.0
-discover # BSD
@@ -27 +26 @@ reno>=1.8.0 # Apache2
-bandit>=1.0.1 # Apache-2.0
+bandit>=1.1.0 # Apache-2.0





More information about the OpenStack-announce mailing list