We jubilantly announce the release of: vitrage 1.6.0: The OpenStack RCA Service This release is part of the pike release series. The source is available from: http://git.openstack.org/cgit/openstack/vitrage Download the package from: https://tarballs.openstack.org/vitrage/ For more details, please see below. 1.6.0 ^^^^^ New Features * A new "SNMP notifier" was added, in order to send SNMP traps from Vitrage. SNMP traps will be sent to signed up targets, when Vitrage deduced alarm is raised. This notifier allows to listen to alarms raised by Vitrage. The new notifier is pluggable so anyone can add an implementation. * The "Vitrage ID feature" is a change in the way we create an entity Vitrage ID. Instead of The ID being created as a concatenation of different entity fields, it is a standard openstak UUID generation. This also allows future history support. * Support definition of entity equivalence. If the equivalence between A and B is configured, all scenarios linked to entity A will be expanded to equivalent scenarios, i.e. same conditions and actions with the referenced entity A replaced by entity B. One use case of equivalence is treating equivalent alarms from multiple data source as the same, but no need for duplicating scenario definition in templates. * The "Multi Tenancy" feature allows different tenants to see their own entities for each api command. The feature allows admin to add the "--all-tenants" parameter in order to see all the entities for that api command. * The "Not Operator" feature adds support of the "not" operator to the templates language in addition to "and" and "or" operators. The "not" operator allows scenarios such as - - Support of High Availability scenarios - Support of negative scenarios, for example - raise an alarm on host that has no "cpu_alarm" on it. Changes in vitrage 1.5.0..1.6.0 ------------------------------- de64417 Add release note about entity equivalence 9015824 multi tenancy release notes 6fd8d07 not operator release notes 7ada6b2 Vitrage ID summary 925baba snmp notifier releasnote c031eab RDO install guide for Vitrage. 103d3ac Remove deprecated oslo.messaging.get_transport 4f99d8b Updated from global requirements 587e57e Replace assertTrue(isinstance()) with assertIsInstance() 5468748 Set default directory of equivalence under templates directory 2cb1faf Add support for building equivalent scenario 5c981e4 Vitrage was enabled twice. 24669a5 Updated from global requirements 5910220 Faster scenario evaluator. Instead of calculating templates connected components, every time, keep results cached for later use. 3d1d39f Faster subgraph matching Removed redundant copy 6d6c457 Faster properties hashing c6e827b Add support for loading entity equivalences a550662 Updated from global requirements e78235e fix bugs: Merging actions by action_spec should be moved after action_tracker 8fa4d75 Updated from global requirements 504ce49 Fix mark-down after the latest changes in vitrage-id dd16678 Updated from global requirements 0e93ae6 performance improvement in the processor. Use set instead of using list. 09c4b8d gate fix - extract images list from glance instead of nova 8669130 Add document about implementation details of template loading 6c24d20 Remove excessive logs for topology show 22d0c7d 'nova_version' should be str type 4352a45 Updated from global requirements 3c8b5e1 fix bug undefined dest for snmp traps 391967f remove relationship validation from template validator c15d47f fix bugs in _find_edge_subgraph_matching: scenario_edge may be not found in subgraph, so stop finding matches in this case. c97a76e Adding verifcation step. 0775712 Adding introduction to Vitrage installation guide. cbf612a Remove out of date comments 37d4356 create default datasource info mapper for static entities 761e712 change all entities from static datasource to be is_placeholder=False 3be6a01 Updated the Vitrage architecture diagram 4e6228d Add 'rm -f .testrepository/times.dbm' command in testenv a7c7be7 add the resource related APIs' description 8e8b212 Fixed the name of group options in get_opts_lists ff4ef5f its a bad idea to use mutable default parameter 589ca87 Fix Doctor test 85b6300 few changes: 1. bug fixs scenarioEvaluator, scenarioRepository, subgraph matching. 2. Tests for HA scenario 2277858 Vitrage ID and vertex id will be standard openstack uuid from now on - fixed fc6f076 add resource show API b242bac Minor fixes in the static datasource documentation e982c7f implement the resource list API 313c482 Updated from global requirements d549b5d Optimize the link address 3c275d5 Pass option list instead of group name c2fedc2 Updated from global requirements e6a92f5 fix the type of `all_tenants` 1e4efa9 Replace six.iteritems() with .items() 4ba42d2 Updated from global requirements a997fc3 Updated from global requirements 54b7635 static transformer fix. 1d883ab bugs fixes: 1. Receive the nova.zone and nova.host data without the 'manager' and 'info' properties which are overweight. 2. Fix the delete_relationship to update the is_deleted of a relationship to True, instead of removing it from the graph. 3. Fix the evaluator not to travel over is_placeholder=true elements 4. Fix the topology API to not return the is_deleted=true edges. f967bd6 Add documentation for static datasource 7e26ce9 Remove unexpected line break 8f327e3 support for not operator in templates condition 961fc9d snmp notifier support f8ccaa4 Remove log translations 6ec312f make a supplementary of API document 93ee110 Updated from global requirements 6f318ce add some delay between post and get a165ea7 fix the command of test cover 2e43bf4 reformat the event time to return unicode string d5cd8a9 reformat the event time d48fcc6 Delete the unnecessary word in algorithm.py 87ea080 make test event work 4ed0be9 neutron port transformer fix - should always find the instance neighbor 316cc03 Updated from global requirements e025a0a fix for running nosetests locally on devstack 60cdf65 Updated from global requirements a330ef9 deatils is a dictionary don't do loads 177abe6 add debug info for an exception 07ad004 fix the vitrage id in test f2cdca8 fix the TestEvent test 977f7d4 Subtracted utc offset from update_timestamp cc14a1c fix the TestEvent test c6bc0a4 Updated from global requirements 4b551d8 [Fix gate]Update test requirement b2df002 define constants for datasource opts 011a187 fix local config due to devstack change in gate 5d287b9 Change parameter topic 2f4138a Cookiecutting install guide 2929a3d Remove unused logging import 213f603 Updated from global requirements ba953e5 fix heat test 0e0621b Added manual installation documentation 47b97c7 Doctor datasource - severity is not mandatory c30cd20 change datasource type to static for devstack installation 61f0f2f template not condition validation 0eabe8a fix the client attribute for alarms 77fc3bb fix the client attribute for alarms a7620db Enable coverage report in console output 8c5edcc Updated from global requirements e5950de decouple test events from its base class e9e6ee8 use 'true' and 'false' instead of magic num b8cd631 Typo fix: Remove redundant quote mark a26298a Remove support for py34 606f49b Update reno for stable/ocata 231c884 Using oslo_log instead of logging. f1e5d53 Delete the unnecessary word in algorithm.py ae8ea68 delete the 'nova' exchange of listener target Diffstat (except docs and test files) ------------------------------------- README.rst | 4 +- devstack/gate_hook.sh | 45 +- devstack/post_test_hook.sh | 2 +- devstack/settings | 2 +- etc/vitrage/datasources_values/default.yaml | 24 + etc/vitrage/datasources_values/static.yaml | 24 + .../datasources_values/static_physical.yaml | 2 + etc/vitrage/policy.json | 1 + install-guide/source/common_configure.rst | 10 + install-guide/source/conf.py | 301 ++++++ install-guide/source/get_started.rst | 15 + install-guide/source/index.rst | 16 + install-guide/source/install-rdo.rst | 195 ++++ install-guide/source/install.rst | 17 + install-guide/source/next-steps.rst | 10 + install-guide/source/verify.rst | 16 + .../notes/SNMP_notifier-5c46505b3043b58f.yaml | 5 + .../notes/Vitrage_ID-93737ff7b7ced8e9.yaml | 6 + .../notes/entity-equivalence-44c0da4cf3b5bc7e.yaml | 8 + .../multi_tenancy_support-feb3d2d4f8014b66.yaml | 6 + ...ot_operator_for_templates-f1e4c517236ca54e.yaml | 8 + releasenotes/source/index.rst | 1 + releasenotes/source/ocata.rst | 6 + requirements.txt | 21 +- setup.cfg | 2 +- setup.py | 2 +- test-requirements.txt | 28 +- tox.ini | 10 +- vitrage/api/app.py | 27 +- vitrage/api/controllers/rest.py | 3 - vitrage/api/controllers/v1/__init__.py | 18 - vitrage/api/controllers/v1/alarm.py | 17 +- vitrage/api/controllers/v1/event.py | 3 +- vitrage/api/controllers/v1/rca.py | 13 +- vitrage/api/controllers/v1/resource.py | 91 +- vitrage/api/controllers/v1/template.py | 7 +- vitrage/api/controllers/v1/topology.py | 14 +- vitrage/api_handler/apis/alarm.py | 2 +- vitrage/api_handler/apis/base.py | 8 + vitrage/api_handler/apis/event.py | 9 +- vitrage/api_handler/apis/rca.py | 2 +- vitrage/api_handler/apis/resource.py | 76 ++ vitrage/api_handler/apis/topology.py | 59 +- vitrage/api_handler/service.py | 4 +- vitrage/cmd/graph.py | 3 +- vitrage/common/constants.py | 11 +- vitrage/datasources/alarm_transformer_base.py | 4 - vitrage/datasources/aodh/__init__.py | 13 +- vitrage/datasources/aodh/transformer.py | 1 + vitrage/datasources/cinder/volume/__init__.py | 7 +- vitrage/datasources/cinder/volume/transformer.py | 5 - vitrage/datasources/collectd/__init__.py | 9 +- .../collectd/collectd_vitrage/vitrageplugin.py | 2 +- vitrage/datasources/collectd/driver.py | 5 +- vitrage/datasources/collectd/transformer.py | 7 +- vitrage/datasources/consistency/transformer.py | 9 +- vitrage/datasources/doctor/__init__.py | 7 +- vitrage/datasources/doctor/driver.py | 1 - vitrage/datasources/doctor/transformer.py | 3 + vitrage/datasources/heat/stack/__init__.py | 7 +- vitrage/datasources/launcher.py | 6 +- vitrage/datasources/listener_service.py | 2 +- vitrage/datasources/nagios/__init__.py | 15 +- vitrage/datasources/nagios/config.py | 3 +- vitrage/datasources/nagios/driver.py | 11 +- vitrage/datasources/nagios/transformer.py | 9 +- vitrage/datasources/neutron/network/__init__.py | 7 +- vitrage/datasources/neutron/port/__init__.py | 7 +- vitrage/datasources/neutron/port/transformer.py | 23 +- vitrage/datasources/nova/host/__init__.py | 7 +- vitrage/datasources/nova/instance/__init__.py | 7 +- vitrage/datasources/nova/instance/transformer.py | 14 +- vitrage/datasources/nova/zone/__init__.py | 7 +- vitrage/datasources/nova/zone/driver.py | 3 +- vitrage/datasources/rescheduler.py | 8 +- vitrage/datasources/resource_transformer_base.py | 5 - vitrage/datasources/services.py | 2 +- vitrage/datasources/static/__init__.py | 24 +- vitrage/datasources/static/driver.py | 24 +- vitrage/datasources/static/transformer.py | 35 +- vitrage/datasources/static_physical/__init__.py | 9 +- vitrage/datasources/static_physical/transformer.py | 4 - vitrage/datasources/transformer_base.py | 5 +- vitrage/datasources/zabbix/__init__.py | 13 +- .../datasources/zabbix/auxiliary/zabbix_vitrage.py | 2 +- vitrage/datasources/zabbix/driver.py | 5 +- vitrage/datasources/zabbix/transformer.py | 7 +- .../consistency/consistency_enforcer.py | 6 +- .../mappings/datasource_info_mapper.py | 12 +- vitrage/entity_graph/processor/notifier.py | 2 +- vitrage/entity_graph/processor/processor.py | 176 +++- vitrage/entity_graph/processor/processor_utils.py | 10 + vitrage/evaluator/__init__.py | 4 + .../actions/evaluator_event_transformer.py | 6 +- vitrage/evaluator/actions/priority_tools.py | 7 +- vitrage/evaluator/actions/recipes/mark_down.py | 1 + vitrage/evaluator/actions/recipes/set_state.py | 3 +- vitrage/evaluator/equivalence_data.py | 60 ++ vitrage/evaluator/equivalence_repository.py | 39 + vitrage/evaluator/scenario_evaluator.py | 244 +++-- vitrage/evaluator/scenario_repository.py | 80 +- vitrage/evaluator/template_data.py | 334 +++++-- vitrage/evaluator/template_fields.py | 4 +- .../template_validation/status_messages.py | 1 + .../template_content_validator.py | 113 ++- .../template_syntax_validator.py | 58 +- vitrage/graph/algo_driver/algorithm.py | 2 +- vitrage/graph/algo_driver/networkx_algorithm.py | 116 ++- vitrage/graph/algo_driver/sub_graph_matching.py | 102 +- vitrage/graph/driver/elements.py | 16 + vitrage/graph/driver/graph.py | 36 +- vitrage/graph/driver/networkx_graph.py | 49 +- vitrage/i18n.py | 10 - vitrage/messaging.py | 4 +- vitrage/notifier/__init__.py | 2 +- vitrage/notifier/plugins/snmp/__init__.py | 37 + vitrage/notifier/plugins/snmp/base.py | 28 + vitrage/notifier/plugins/snmp/snmp_notifier.py | 61 ++ vitrage/notifier/plugins/snmp/snmp_sender.py | 203 ++++ vitrage/opts.py | 2 + vitrage/os_clients.py | 21 +- vitrage/service.py | 4 +- .../datasources/cinder/test_cinder_volume.py | 2 +- .../functional/datasources/heat/test_heat_stack.py | 2 +- .../functional/datasources/nagios/test_nagios.py | 2 +- .../datasources/nova/test_nova_datasources.py | 2 +- .../static_physical/test_static_physical.py | 2 +- .../entity_graph/consistency/test_consistency.py | 12 +- .../entity_graph/processor/test_processor.py | 2 +- .../states/test_datasource_info_mapper.py | 10 +- .../functional/evaluator/test_action_executor.py | 68 +- .../evaluator/test_scenario_evaluator.py | 1032 ++++++++++++++++++- .../resources/datasources_values/default.yaml | 24 + .../erroneous_values/default.yaml | 24 + .../driver/driver_host_snapshot_dynamic.json | 3 + .../driver/driver_inst_snapshot_dynamic.json | 1 + .../driver/driver_zone_snapshot_dynamic.json | 3 + .../resources/snmp_notifier/alarm_oid_mapping.yaml | 3 + .../oid_tree_with_severity_mapping.yaml | 29 + .../oid_tree_without_severity_mapping.yaml | 22 + .../equivalences_dup/dup-in-multi-def/multi.yaml | 21 + .../equivalences_dup/dup-in-multi-file/first.yaml | 12 + .../equivalences_dup/dup-in-multi-file/second.yaml | 12 + .../equivalences_dup/dup-in-one-def/one.yaml | 12 + .../templates/equivalent_scenarios/basic.yaml | 40 + .../equivalent_scenarios/equivalences/basic.yaml | 21 + .../complex_and_or_operator_deduced_alarm.yaml | 53 + .../complex_not_operator_deduced_alarm.yaml | 59 ++ .../templates/evaluator/high_availability.yaml | 67 ++ .../simple_not_operator_deduced_alarm.yaml | 31 + .../simple_or_operator_deduced_alarm.yaml | 42 + .../templates/general/equivalences/basic.yaml | 20 + .../not_operator/basic_correct_not_condition.yaml | 30 + .../basic_incorrect_not_condition.yaml | 30 + .../complicated_correct_not_condition.yaml | 57 ++ .../complicated_incorrect_not_condition.yaml | 57 ++ .../unit/datasources/aodh/test_aodh_driver.py | 11 +- .../unit/datasources/aodh/test_aodh_transformer.py | 5 +- .../cinder/test_cinder_volume_transformer.py | 3 +- .../collectd/test_collectd_transformer.py | 6 +- .../consistency/test_consistency_transformer.py | 3 +- .../datasources/doctor/test_doctor_transformer.py | 6 +- .../heat/test_heat_stack_transformer.py | 3 +- .../unit/datasources/nagios/test_nagios_config.py | 12 +- .../unit/datasources/nagios/test_nagios_driver.py | 7 +- .../unit/datasources/nagios/test_nagios_parser.py | 3 - .../datasources/nagios/test_nagios_transformer.py | 12 +- .../datasources/nova/test_nova_host_transformer.py | 3 +- .../nova/test_nova_instance_transformer.py | 5 +- .../datasources/nova/test_nova_zone_transformer.py | 5 +- .../unit/datasources/static/test_static_driver.py | 22 +- .../datasources/static/test_static_transformer.py | 25 +- .../static_physical/test_static_physical_driver.py | 17 +- .../test_static_physical_transformer.py | 3 +- .../datasources/test_alarm_transformer_base.py | 3 - .../datasources/test_datasource_update_method.py | 40 +- .../unit/datasources/test_transformer_base.py | 4 - .../zabbix/test_zabbix_configuration.py | 13 +- .../unit/datasources/zabbix/test_zabbix_driver.py | 6 +- .../datasources/zabbix/test_zabbix_transformer.py | 3 +- .../unit/entity_graph/processor/test_processor.py | 24 +- .../states/test_datasource_info_mapper.py | 46 +- .../evaluator/recipes/test_set_state_recipe.py | 6 +- .../unit/evaluator/test_equivalence_repository.py | 35 + .../unit/evaluator/test_scenario_repository.py | 55 +- .../evaluator/test_template_content_validator.py | 83 +- .../evaluator/test_template_syntax_validator.py | 10 - .../notifier/snmp_notifier/test_snmp_notifier.py | 50 + .../test_snmp_sender_with_severity_map.py | 122 +++ .../test_snmp_sender_without_severity_map.py | 97 ++ vitrage/utils/__init__.py | 3 +- vitrage/utils/datetime.py | 8 + vitrage/utils/file.py | 4 + 237 files changed, 7839 insertions(+), 1181 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 2e276d7..4ffb546 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,2 +5,2 @@ -pbr>=1.8 # Apache-2.0 -Babel>=2.3.4 # BSD +pbr!=2.1.0,>=2.0.0 # Apache-2.0 +Babel!=2.4.0,>=2.3.4 # BSD @@ -9 +9 @@ python-ceilometerclient>=2.5.0 # Apache-2.0 -python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 +python-cinderclient>=2.1.0 # Apache-2.0 @@ -12,2 +12,2 @@ python-keystoneclient>=3.8.0 # Apache-2.0 -python-neutronclient>=5.1.0 # Apache-2.0 -python-novaclient!=7.0.0,>=6.0.0 # Apache-2.0 +python-neutronclient>=6.3.0 # Apache-2.0 +python-novaclient>=7.1.0 # Apache-2.0 @@ -17,3 +17,3 @@ networkx>=1.10 # BSD -oslo.config!=3.18.0,>=3.14.0 # Apache-2.0 -oslo.messaging>=5.14.0 # Apache-2.0 -oslo.log>=3.11.0 # Apache-2.0 +oslo.config>=4.0.0 # Apache-2.0 +oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 +oslo.log>=3.22.0 # Apache-2.0 @@ -22 +22 @@ oslo.service>=1.10.0 # Apache-2.0 -oslo.i18n>=2.1.0 # Apache-2.0 +oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 @@ -27 +27 @@ keystonemiddleware>=4.12.0 # Apache-2.0 -stevedore>=1.17.1 # Apache-2.0 +stevedore>=1.20.0 # Apache-2.0 @@ -29,0 +30 @@ sympy>=0.7.6 # BSD +pysnmp>=4.2.3 # BSD diff --git a/test-requirements.txt b/test-requirements.txt index 02fea7f..e80188a 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -5 +5 @@ -hacking<0.11,>=0.10.0 +hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 @@ -7 +7 @@ python-dateutil>=2.4.2 # BSD -coverage>=4.0 # Apache-2.0 +coverage!=4.4,>=4.0 # Apache-2.0 @@ -11,3 +11,3 @@ python-ceilometerclient>=2.5.0 # Apache-2.0 -python-cinderclient!=1.7.0,!=1.7.1,>=1.6.0 # Apache-2.0 -python-neutronclient>=5.1.0 # Apache-2.0 -python-novaclient!=7.0.0,>=6.0.0 # Apache-2.0 +python-cinderclient>=2.1.0 # Apache-2.0 +python-neutronclient>=6.3.0 # Apache-2.0 +python-novaclient>=7.1.0 # Apache-2.0 @@ -17,2 +17,2 @@ pyzabbix>=0.7.4 # LGPL -sphinx!=1.3b1,<1.4,>=1.2.1 # BSD -oslo.log>=3.11.0 # Apache-2.0 +sphinx!=1.6.1,>=1.5.1 # BSD +oslo.log>=3.22.0 # Apache-2.0 @@ -22,3 +22,3 @@ oslo.service>=1.10.0 # Apache-2.0 -oslo.config!=3.18.0,>=3.14.0 # Apache-2.0 -oslo.messaging>=5.14.0 # Apache-2.0 -oslo.i18n>=2.1.0 # Apache-2.0 +oslo.config>=4.0.0 # Apache-2.0 +oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 +oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 @@ -31 +31 @@ testtools>=1.4.0 # MIT -stevedore>=1.17.1 # Apache-2.0 +stevedore>=1.20.0 # Apache-2.0 @@ -34 +34,5 @@ sympy>=0.7.6 # BSD -reno>=1.8.0 # Apache-2.0 +reno!=2.3.1,>=1.8.0 # Apache-2.0 +pysnmp>=4.2.3 # BSD + +# Doc requirements +openstackdocstheme>=1.5.0 # Apache-2.0