We are jazzed to announce the release of: vitrage 1.7.0: The OpenStack RCA Service This release is part of the pike release series. The source is available from: http://git.openstack.org/cgit/openstack/vitrage Download the package from: https://tarballs.openstack.org/vitrage/ For more details, please see below. 1.7.0 ^^^^^ New Features * Added osprofiler support. "OSProfiler" is an OpenStack cross- project profiling library. It allows the user to generate a trace per request which is processed in multiple services, and then generate a tree of calls which is intuitive to understand. Changes in vitrage 1.6.0..1.7.0 ------------------------------- ca8ec97 Added OSProfiler release notes 5d40367 Doc Migration: move the developer guide documents under the "contributor" folder b977d86 Updated from global requirements 807b079 Add osprofiler support for RPC, api controllers and handlers 9b9e53d Add document about how configure and use profiler in vitrage 9d3c811 Create a folder for the contributor docs df289c2 Add template validations, to handle the case of actions that don't have an action_target 6714cc4 Synchronize notification listener and evaluator queue 6491f4c Fixed order of arguments in assertEqual e25ab6b Updated from global requirements 2a71751 Execute an external engine from Vitrage e2b9641 Separation of Evaluator and InitializationStatus from ConsistencyEnforcer 9c30206 Remove sample_timestamp from update events made by vitrage c0f845d supprt keycloak first stage e405bea Use openstackdocstheme bf615f2 Update URL home-page in documents according to document migration 1c49907 Support different resource types in collectd 3b67a40 add some logs ac059ea Enable some off-by-default checks 93609fd use assertIsNone ad67454 Replace six.iteritems() with .items() fc5cdd4 Replaces uuid.uuid4 with uuidutils.generate_uuid() 6882ea4 Enable osprofiler for api 5057c96 Introduce the execute-mistral action. This patch includes validating the action, and converting it to a general-purpose ExecuteExternal step. be693ab New Tool- To generate numerous mock notifications. Can create large graph at any size c9ea8ea Aligning install guide with the Pike doc structure e242793 messaging use blocking instead of threading collector notification topic name change 6d2c908 add healthcheck option 41f863b Mark edge as deleted instead of removing it 014d464 Move template_content_validator and test_template_content_validator to the correct packages. This should have been part of https://review.openstack.org/477572 but was removed due to git issues. 17ee70a support uwsgi in devstack cca78a3 log change 53df5e4 Refactor the Vitrage template content validators and their tests. b73387b Remove requeue of messages to rabbitmq because try and catch handle it 5756e2e Pass environment variables of proxy to tox b727352 remove deprecated auth type password-vitrage-legacy 64488a1 add no authentication for vitrage bdfa317 create collector service 0894e67 Delete the external-actions.rst file, it belongs in the vitrage-specs project a9a43fd event tempst test changes 83a41a4 Pushed to Vitrage instead of Vitrage-specs 89b2798 remove tox warnning cffda41 Describe the Vitrage HA and history vision. d02498c Support external actions in Vitrage templates. 24105fb Http Post specs 01b53d1 Replacing hardcoded values with constants 3b1de00 Vertex Properties refactoring b32382b Faster graph copy 8fe48f8 Updated from global requirements a0ce352 Add links to design documents c9b93cb for ML - add resource details: id and type to alarms 6ab76a5 Fix a broken link in the installation doc dfe9bf9 Remove usage of parameter enforce_type 73873ff Add Apache License Content in index.rst b1d889d Set access_policy for messaging's dispatcher Diffstat (except docs and test files) ------------------------------------- Dockerfile | 10 + devstack/plugin.sh | 78 +- devstack/settings | 11 +- .../contributor/host_high_memory_consumption.yaml | 72 + .../contributor/images/add_aodh_alarm_flow.png | Bin 0 -> 205286 bytes .../contributor/images/add_aodh_alarm_graph.png | Bin 0 -> 13261 bytes .../contributor/images/add_nova_instance_flow.png | Bin 0 -> 221898 bytes .../contributor/images/add_nova_instance_graph.png | Bin 0 -> 13562 bytes .../contributor/images/complex_rca_graph.png | Bin 0 -> 26304 bytes .../images/nagios_causes_deduced_flow.png | Bin 0 -> 216665 bytes .../images/nagios_causes_deduced_graph.png | Bin 0 -> 20911 bytes .../contributor/images/vitrage-ha-vision.png | Bin 0 -> 99836 bytes .../images/vitrage_graph_architecture.png | Bin 0 -> 269090 bytes .../contributor/nagios-devstack-installation.rst | 96 ++ .../template_validation_status_code.rst | 111 ++ .../contributor/vitrage-ha-and-history-vision.rst | 230 ++++ etc/vitrage/api-paste.ini | 61 +- etc/vitrage/vitrage-config-generator.conf | 5 +- install-guide/source/common_configure.rst | 10 - install-guide/source/conf.py | 301 ----- install-guide/source/get_started.rst | 15 - install-guide/source/index.rst | 16 - install-guide/source/install-rdo.rst | 195 --- install-guide/source/install.rst | 17 - install-guide/source/next-steps.rst | 10 - install-guide/source/verify.rst | 16 - .../notes/osprofiler-784098daab42a150.yaml | 6 + releasenotes/source/conf.py | 10 +- releasenotes/source/index.rst | 13 + requirements.txt | 11 +- setup.cfg | 8 +- test-requirements.txt | 14 +- tools/__init__.py | 15 + tools/load_generator/__init__.py | 15 + tools/load_generator/computes.yaml | 515 +++++++ tools/load_generator/load_generator.py | 146 ++ tools/load_generator/notification_info.py | 157 +++ tools/load_generator/templates/port_0.yaml | 84 ++ tools/load_generator/templates/port_1.yaml | 84 ++ tools/load_generator/templates/port_2.yaml | 84 ++ tools/load_generator/templates/port_3.yaml | 84 ++ tools/load_generator/templates/port_4.yaml | 84 ++ tools/load_generator/templates/vm_0.yaml | 84 ++ tools/load_generator/templates/vm_1.yaml | 84 ++ tools/load_generator/templates/vm_2.yaml | 84 ++ tools/load_generator/templates/vm_3.yaml | 84 ++ tools/load_generator/templates/vm_4.yaml | 84 ++ tox.ini | 9 +- vitrage/api/__init__.py | 23 +- vitrage/api/app.py | 61 +- vitrage/api/controllers/root.py | 4 +- vitrage/api/controllers/v1/alarm.py | 3 + vitrage/api/controllers/v1/event.py | 3 + vitrage/api/controllers/v1/rca.py | 3 + vitrage/api/controllers/v1/resource.py | 3 + vitrage/api/controllers/v1/template.py | 3 + vitrage/api/controllers/v1/topology.py | 3 + vitrage/api_handler/apis/alarm.py | 13 +- vitrage/api_handler/apis/base.py | 80 +- vitrage/api_handler/apis/event.py | 3 + vitrage/api_handler/apis/rca.py | 4 +- vitrage/api_handler/apis/resource.py | 5 +- vitrage/api_handler/apis/template.py | 7 +- vitrage/api_handler/apis/topology.py | 14 +- vitrage/cmd/api.py | 5 + vitrage/cmd/collector.py | 44 + vitrage/cmd/graph.py | 32 +- vitrage/common/constants.py | 29 +- vitrage/datasources/__init__.py | 6 +- vitrage/datasources/alarm_driver_base.py | 8 +- vitrage/datasources/aodh/driver.py | 2 +- vitrage/datasources/aodh/transformer.py | 24 +- vitrage/datasources/cinder/volume/driver.py | 6 +- vitrage/datasources/cinder/volume/transformer.py | 10 +- .../collectd/collectd_vitrage/vitrageplugin.py | 9 +- vitrage/datasources/collectd/driver.py | 2 +- vitrage/datasources/collectd/transformer.py | 42 +- vitrage/datasources/collector_notifier.py | 53 + vitrage/datasources/consistency/transformer.py | 4 +- vitrage/datasources/doctor/driver.py | 2 +- vitrage/datasources/doctor/properties.py | 3 +- vitrage/datasources/doctor/transformer.py | 8 +- vitrage/datasources/driver_base.py | 5 + vitrage/datasources/heat/stack/driver.py | 43 +- vitrage/datasources/heat/stack/transformer.py | 14 +- vitrage/datasources/launcher.py | 37 +- vitrage/datasources/listener_service.py | 12 +- vitrage/datasources/nagios/config.py | 2 +- vitrage/datasources/nagios/driver.py | 2 +- vitrage/datasources/nagios/transformer.py | 15 +- vitrage/datasources/neutron/network/transformer.py | 10 +- vitrage/datasources/neutron/port/transformer.py | 10 +- vitrage/datasources/nova/host/driver.py | 6 +- vitrage/datasources/nova/host/transformer.py | 15 +- vitrage/datasources/nova/instance/driver.py | 8 +- vitrage/datasources/nova/instance/transformer.py | 12 +- vitrage/datasources/nova/zone/driver.py | 7 +- vitrage/datasources/nova/zone/transformer.py | 19 +- vitrage/datasources/static/driver.py | 13 +- vitrage/datasources/static/transformer.py | 26 +- vitrage/datasources/static_physical/driver.py | 6 +- vitrage/datasources/static_physical/transformer.py | 21 +- vitrage/datasources/transformer_base.py | 48 +- vitrage/datasources/zabbix/driver.py | 2 +- vitrage/datasources/zabbix/transformer.py | 15 +- .../consistency/consistency_enforcer.py | 94 +- vitrage/entity_graph/consistency/service.py | 31 +- vitrage/entity_graph/initialization_status.py | 29 - vitrage/entity_graph/mappings/alarm_handler.py | 4 +- .../mappings/datasource_info_mapper.py | 48 +- vitrage/entity_graph/mappings/resource_handler.py | 9 +- vitrage/entity_graph/processor/notifier.py | 12 +- vitrage/entity_graph/processor/processor.py | 67 +- vitrage/entity_graph/processor/processor_utils.py | 44 +- vitrage/entity_graph/service.py | 60 +- vitrage/entity_graph/transformer_manager.py | 5 +- vitrage/entity_graph/utils.py | 35 + vitrage/entity_graph/vitrage_init.py | 115 ++ vitrage/evaluator/actions/action_executor.py | 50 +- vitrage/evaluator/actions/base.py | 4 +- .../actions/evaluator_event_transformer.py | 27 +- vitrage/evaluator/actions/notifier.py | 75 ++ vitrage/evaluator/actions/priority_tools.py | 6 +- vitrage/evaluator/actions/recipes/action_steps.py | 3 + .../evaluator/actions/recipes/execute_mistral.py | 59 + vitrage/evaluator/actions/recipes/raise_alarm.py | 3 +- vitrage/evaluator/condition.py | 189 +++ vitrage/evaluator/equivalence_data.py | 5 +- vitrage/evaluator/scenario_evaluator.py | 29 +- vitrage/evaluator/scenario_repository.py | 4 +- vitrage/evaluator/template_data.py | 177 ++- .../template_validation/content/__init__.py | 15 + .../content/add_causal_relationship_validator.py | 73 + .../evaluator/template_validation/content/base.py | 60 + .../content/execute_mistral_validator.py | 41 + .../content/mark_down_validator.py | 44 + .../content/raise_alarm_validator.py | 54 + .../content/set_state_validator.py | 50 + .../content/template_content_validator.py | 289 ++++ .../template_validation/status_messages.py | 11 +- .../template_content_validator.py | 328 ----- .../template_syntax_validator.py | 2 +- vitrage/graph/algo_driver/algorithm.py | 6 + vitrage/graph/algo_driver/networkx_algorithm.py | 3 +- vitrage/graph/driver/networkx_graph.py | 10 +- vitrage/graph/query.py | 8 +- vitrage/graph/utils.py | 48 +- vitrage/keystone_client.py | 61 - vitrage/messaging.py | 2 +- vitrage/middleware/__init__.py | 15 + vitrage/middleware/keycloak.py | 98 ++ vitrage/notifier/plugins/aodh/aodh_notifier.py | 3 +- vitrage/notifier/plugins/base.py | 9 + vitrage/notifier/plugins/nova/nova_notifier.py | 2 +- vitrage/notifier/plugins/snmp/snmp_sender.py | 2 +- vitrage/notifier/service.py | 38 +- vitrage/rpc.py | 57 +- vitrage/service.py | 13 +- .../datasources/cinder/test_cinder_volume.py | 6 +- .../functional/datasources/collectd/__init__.py | 15 + .../datasources/collectd/test_collectd.py | 200 +++ .../functional/datasources/heat/test_heat_stack.py | 12 +- .../functional/datasources/nagios/test_nagios.py | 9 +- .../static_physical/test_static_physical.py | 6 +- .../entity_graph/consistency/test_consistency.py | 61 +- .../states/test_datasource_info_mapper.py | 16 +- .../functional/evaluator/test_action_executor.py | 110 +- .../evaluator/test_scenario_evaluator.py | 770 ++++++----- .../driver/driver_host_snapshot_dynamic.json | 2 +- .../driver/driver_inst_snapshot_dynamic.json | 2 +- .../driver/driver_static_snapshot_dynamic.json | 2 +- .../driver/driver_zone_snapshot_dynamic.json | 2 +- .../transformer_aodh_snapshot_dynamic.json | 14 +- .../transformer_aodh_update_dynamic.json | 14 +- .../oid_tree_with_severity_mapping.yaml | 4 +- .../oid_tree_without_severity_mapping.yaml | 4 +- .../templates/evaluator/conditions/complex1.yaml | 57 + .../templates/evaluator/conditions/complex2.yaml | 55 + .../evaluator/conditions/complex_not.yaml | 57 + .../conditions/complex_not_unsupported.yaml | 57 + .../evaluator/conditions/not_edge_unsupported.yaml | 35 + .../evaluator/conditions/not_or_unsupported.yaml | 46 + .../evaluator/conditions/not_or_unsupported2.yaml | 46 + .../templates/evaluator/conditions/one_edge.yaml | 35 + .../templates/evaluator/conditions/one_vertex.yaml | 24 + .../templates/evaluator/conditions/simple_and.yaml | 44 + .../evaluator/conditions/simple_and2.yaml | 53 + .../templates/evaluator/conditions/simple_or.yaml | 46 + .../templates/evaluator/conditions/simple_or2.yaml | 57 + .../templates/evaluator/conditions/simple_or3.yaml | 46 + .../conditions/simple_or_unsupported.yaml | 55 + .../simple_not_operator_deduced_alarm.yaml | 2 +- .../not_operator/basic_correct_not_condition.yaml | 8 +- .../complicated_correct_not_condition.yaml | 2 +- .../datasources/aodh/aodh_transformer_base_test.py | 23 +- .../unit/datasources/aodh/test_aodh_driver.py | 5 +- .../cinder/test_cinder_volume_transformer.py | 47 +- .../datasources/collectd/test_collectd_driver.py | 3 +- .../collectd/test_collectd_transformer.py | 10 +- .../consistency/test_consistency_transformer.py | 5 +- .../unit/datasources/doctor/test_doctor_driver.py | 3 +- .../heat/test_heat_stack_transformer.py | 47 +- .../unit/datasources/nagios/test_nagios_config.py | 3 +- .../unit/datasources/nagios/test_nagios_driver.py | 100 +- .../unit/datasources/nagios/test_nagios_parser.py | 7 +- .../datasources/nagios/test_nagios_transformer.py | 23 +- .../datasources/nova/test_nova_host_transformer.py | 46 +- .../nova/test_nova_instance_transformer.py | 50 +- .../datasources/nova/test_nova_zone_transformer.py | 61 +- .../datasources/static/test_static_transformer.py | 42 +- .../static_physical/test_static_physical_driver.py | 36 +- .../test_static_physical_transformer.py | 45 +- .../datasources/test_alarm_transformer_base.py | 12 +- .../datasources/test_datasource_update_method.py | 15 +- .../unit/datasources/test_rescheduler_functions.py | 4 +- .../unit/datasources/test_transformer_base.py | 4 +- .../zabbix/test_zabbix_configuration.py | 3 +- .../unit/datasources/zabbix/test_zabbix_driver.py | 3 +- .../datasources/zabbix/test_zabbix_transformer.py | 21 +- .../entity_graph/processor/test_entity_graph.py | 49 +- .../unit/entity_graph/processor/test_processor.py | 98 +- .../states/test_datasource_info_mapper.py | 143 +- .../unit/evaluator/recipes/test_execute_mistral.py | 70 + .../unit/evaluator/recipes/test_raise_alarm.py | 23 +- .../evaluator/recipes/test_set_state_recipe.py | 4 +- .../unit/evaluator/template_validation/__init__.py | 15 + .../template_validation/content/__init__.py | 15 + .../evaluator/template_validation/content/base.py | 114 ++ .../test_add_causal_relationship_validator.py | 111 ++ .../content/test_execute_mistral_validator.py | 99 ++ .../content/test_mark_down_validator.py | 61 + .../content/test_raise_alarm_validator.py | 95 ++ .../content/test_set_state_validator.py | 83 ++ .../content/test_template_content_validator.py | 246 ++++ .../test_template_syntax_validator.py | 243 ++++ .../unit/evaluator/test_scenario_repository.py | 15 +- .../evaluator/test_template_content_validator.py | 515 ------- .../evaluator/test_template_syntax_validator.py | 247 ---- .../notifier/snmp_notifier/test_snmp_notifier.py | 31 +- .../test_snmp_sender_with_severity_map.py | 20 +- .../test_snmp_sender_without_severity_map.py | 16 +- 347 files changed, 13113 insertions(+), 8596 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 4ffb546..879e0fc 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9 +9 @@ python-ceilometerclient>=2.5.0 # Apache-2.0 -python-cinderclient>=2.1.0 # Apache-2.0 +python-cinderclient>=3.0.0 # Apache-2.0 @@ -13 +13 @@ python-neutronclient>=6.3.0 # Apache-2.0 -python-novaclient>=7.1.0 # Apache-2.0 +python-novaclient>=9.0.0 # Apache-2.0 @@ -17 +17 @@ networkx>=1.10 # BSD -oslo.config>=4.0.0 # Apache-2.0 +oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 @@ -20 +20 @@ oslo.log>=3.22.0 # Apache-2.0 -oslo.policy>=1.17.0 # Apache-2.0 +oslo.policy>=1.23.0 # Apache-2.0 @@ -30,0 +31,3 @@ pysnmp>=4.2.3 # BSD +PyJWT>=1.0.1 # MIT +osprofiler>=1.4.0 # Apache-2.0 + diff --git a/test-requirements.txt b/test-requirements.txt index e80188a..5afd487 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -11 +11 @@ python-ceilometerclient>=2.5.0 # Apache-2.0 -python-cinderclient>=2.1.0 # Apache-2.0 +python-cinderclient>=3.0.0 # Apache-2.0 @@ -13 +13 @@ python-neutronclient>=6.3.0 # Apache-2.0 -python-novaclient>=7.1.0 # Apache-2.0 +python-novaclient>=9.0.0 # Apache-2.0 @@ -17 +16,0 @@ pyzabbix>=0.7.4 # LGPL -sphinx!=1.6.1,>=1.5.1 # BSD @@ -19 +17,0 @@ oslo.log>=3.22.0 # Apache-2.0 -oslosphinx>=4.7.0 # Apache-2.0 @@ -22 +20 @@ oslo.service>=1.10.0 # Apache-2.0 -oslo.config>=4.0.0 # Apache-2.0 +oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 @@ -25 +23 @@ oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 -oslo.policy>=1.17.0 # Apache-2.0 +oslo.policy>=1.23.0 # Apache-2.0 @@ -35,0 +34 @@ pysnmp>=4.2.3 # BSD +osprofiler>=1.4.0 # Apache-2.0 @@ -38 +37,2 @@ pysnmp>=4.2.3 # BSD -openstackdocstheme>=1.5.0 # Apache-2.0 +openstackdocstheme>=1.11.0 # Apache-2.0 +sphinx>=1.6.2 # BSD