We are amped to announce the release of: mistral 11.0.0: Mistral Project This release is part of the victoria release series. The source is available from: https://opendev.org/openstack/mistral Download the package from: https://pypi.org/project/mistral Please report issues through: https://bugs.launchpad.net/mistral/+bugs For more details, please see below. 11.0.0 ^^^^^^ New Features ************ * There has been a big change in the Mistral action management. All Mistral subsystems don't access the database directly if they need to work with action definitions. Instead, they work action providers registered in the new entry point "mistral.action.providers". All action providers need to implement the base class ActionProvider declared in "mistral-lib" starting with the version 2.3.0. Action providers are responsible for delivering so-called action descriptors that carry all the most important information about particular actions like "name", "description", names of input parameters and so on. The entire system has now been refactored with action providers. Using this new mechanism it's now possible to deliver actions into the system dynamically w/o having to reboot Mistral. We just need to come up with an action provider implementation that can do that and register it in the entry point from any Python project installed on the same Python environment. This approach also means that actions don't have to be stored in the database anymore. It fully depends on a particular action provider how to store action descriptors and how to perform lookup. It is possible to create action providers fetching information about actions over HTTP, AMQP and potentially any other protocol. Additionally, switching to action providers allowed to make engine code much cleaner and more encapsulated. For example, ad-hoc actions are no longer a concern of the Mistral engine. Instead of implementing all ad-hoc action logic in the engine there's now a special action provider fully responsible for ad-hoc actions. The detailed documentation on using action providers will be added soon. Upgrade Notes ************* * As part of the transition to action providers the compatibility of the /actions REST API endpoint has been broken to some extent. For ad-hoc actions it remained almost the same. We still can do all CRUD operations upon them. However, all the standard actions (prefixed with "std.") are not stored in the DB anymore. For that reason this type of actions doesn't have IDs anymore, and generally actions are not identified by IDs anymore, only by name. This change needs to be taken into account when updating to this version of Mistral. Changes in mistral 10.0.0..11.0.0 --------------------------------- 068e5cc0 Add release notes for the action provider transition 06a0f334 Refactor Mistral with Action Providers 4bc61625 Remove six library da5ac254 Remove six.moves aba14934 Remove usage of six.add_metaclass ac84e6d8 Remove translation sections from setup.cfg b55cc013 [goal] Migrate testing to ubuntu focal 63225d41 Remove TripleO gate job 4733e084 Change python-nose package name in devstack 2e600ce6 Generalize caught exceptions on workflow delete f2392bae Use assertCountEqual instead of assertItemsEqual d0d86506 Fix Python 3 compatibility issue in V8EvalEvaluator 4f617856 Add a protection for list commands from parallel deletes a2fda405 Add a migration that deletes redundant delayed calls 104ab8e9 Update Zuul job configuration d0d75be7 drop mock from lower-constraints 2ad7bb82 Moving task type constants into Mistral task specifications module 2b7a2bba Remove one more self.notify() call from class Task 90d1f1ba Refactor workflow notifications b55dbdea Refactor task notifications 986277d6 Stop to use the __future__ module. a620dabb Simplify setting task "started_at" and "finished_at" 7dec19ae Fix calculating task execution result for "with-items" ddf95777 Refactor task policies c4c11c9b Cap jsonschema 3.2.0 as the minimal version 4b52d4d4 Switch to newer openstackdocstheme and reno versions 6c0f3f47 Delete an obsolete namespace from oslo-config-generator config c6503b2c Fix hacking min version to 3.0.1 84041d67 Monkey patch original current_thread _active 1f9e1c1f added a new realcase scenario to rally and clean leftovers c36cbe0a added statistics-only in report Api and a new field in statistics 3c8dafcc Add a Rally scenario for a very big mistral workflow 99cf8cba Bump default tox env from py37 to py38 3eef103b Add py38 package metadata 4ddd3c63 Add Python3 victoria unit tests b56576ee Update master for stable/ussuri d33313f8 typo of guide 'How to write a Custom Action in mistral' Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 56 +- api-ref/source/conf.py | 18 +- devstack/plugin.sh | 10 +- .../extensions/creating_custom_action.rst | 2 +- lower-constraints.txt | 17 +- mistral/actions/action_factory.py | 1 + mistral/actions/adhoc.py | 260 + mistral/actions/legacy.py | 181 + mistral/actions/std_actions.py | 19 +- mistral/actions/test.py | 66 + mistral/api/controllers/v2/action.py | 159 +- mistral/api/controllers/v2/execution.py | 2 +- mistral/api/controllers/v2/execution_report.py | 108 +- mistral/api/controllers/v2/resources.py | 20 +- mistral/api/controllers/v2/service.py | 3 +- mistral/api/controllers/v2/types.py | 3 +- mistral/auth/__init__.py | 4 +- mistral/auth/keycloak.py | 4 +- mistral/cmd/launch.py | 12 +- mistral/config.py | 34 + .../sqlalchemy/migration/alembic_migrations/env.py | 2 - .../038_delete_delayed_calls_with_empty_key.py | 47 + .../versions/039_add_workbook_name_column.py | 43 + mistral/db/sqlalchemy/migration/cli.py | 3 +- mistral/db/utils.py | 7 +- mistral/db/v2/api.py | 35 +- mistral/db/v2/sqlalchemy/api.py | 2 +- mistral/db/v2/sqlalchemy/models.py | 9 +- mistral/engine/action_handler.py | 43 +- mistral/engine/actions.py | 431 +- mistral/engine/base.py | 44 +- mistral/engine/default_engine.py | 25 +- mistral/engine/policies.py | 227 +- mistral/engine/task_handler.py | 34 +- mistral/engine/tasks.py | 320 +- mistral/engine/utils.py | 2 + mistral/engine/workflow_handler.py | 9 +- mistral/engine/workflows.py | 90 +- mistral/event_engine/base.py | 4 +- mistral/event_engine/default_event_engine.py | 8 +- mistral/executors/base.py | 23 +- mistral/executors/default_executor.py | 93 +- mistral/executors/executor_server.py | 45 +- mistral/expressions/__init__.py | 9 +- mistral/expressions/jinja_expression.py | 5 +- mistral/expressions/std_functions.py | 14 +- mistral/expressions/yaql_expression.py | 3 +- mistral/hacking/checks.py | 5 +- mistral/lang/base.py | 13 +- mistral/lang/parser.py | 4 +- mistral/lang/v2/actions.py | 4 +- mistral/lang/v2/on_clause.py | 4 +- mistral/lang/v2/retry_policy.py | 4 +- mistral/lang/v2/task_defaults.py | 6 +- mistral/lang/v2/tasks.py | 21 +- mistral/lang/v2/workflows.py | 3 +- mistral/messaging.py | 4 +- mistral/notifiers/base.py | 7 +- mistral/notifiers/notification_events.py | 44 +- mistral/notifiers/publishers/webhook.py | 2 +- mistral/rpc/clients.py | 28 +- mistral/rpc/kombu/kombu_client.py | 7 +- mistral/rpc/kombu/kombu_hosts.py | 3 +- mistral/rpc/kombu/kombu_listener.py | 8 +- mistral/scheduler/base.py | 4 +- mistral/service/coordination.py | 17 +- mistral/services/action_manager.py | 119 +- mistral/services/actions.py | 159 +- mistral/services/adhoc_actions.py | 145 + mistral/services/triggers.py | 3 +- mistral/services/workbooks.py | 10 +- .../unit/actions/test_adhoc_action_provider.py | 50 + .../unit/actions/test_legacy_action_provider.py | 202 + .../unit/engine/test_action_heartbeat_checker.py | 2 + .../unit/engine/test_disabled_yaql_conversion.py | 7 +- .../test_execution_fields_size_limitation.py | 3 +- .../unit/engine/test_task_started_finished_at.py | 2 +- .../unit/expressions/test_jinja_expression.py | 2 +- ...ion_service.py => test_adhoc_action_service.py} | 44 +- mistral/utils/__init__.py | 28 +- mistral/utils/filter_utils.py | 80 +- mistral/utils/javascript.py | 2 +- mistral/utils/rest_utils.py | 30 +- mistral/utils/ssh_utils.py | 4 +- mistral/utils/wf_trace.py | 1 + mistral/workflow/data_flow.py | 35 +- .../extra/scenarios/big_wf/deploy_params.json | 60027 ++++++++++++ rally-jobs/extra/scenarios/big_wf/deploy_wf.yaml | 92176 +++++++++++++++++++ .../extra/scenarios/big_wf/dummy_actions.yaml | 46 + .../scenarios/big_wf/dummy_actions_nuage.yaml | 37 + rally-jobs/extra/scenarios/big_wf/sub_wfs.yaml | 2340 + .../extra/scenarios/big_wf/terminate_params.json | 60045 ++++++++++++ .../extra/scenarios/big_wf/terminate_wf.yaml | 6671 ++ rally-jobs/extra/scenarios/big_wf/wf_input.json | 3 + rally-jobs/plugins/mistral_huge_wf_scenario.py | 128 + rally-jobs/task-mistral.yaml | 32 +- .../notes/action_providers-a1b65db20bd01071.yaml | 38 + releasenotes/source/conf.py | 14 +- releasenotes/source/index.rst | 1 + releasenotes/source/ussuri.rst | 6 + requirements.txt | 11 +- setup.cfg | 5 + test-requirements.txt | 2 +- tools/config/config-generator.mistral.conf | 1 - tools/install_venv.py | 2 - tools/install_venv_common.py | 2 - tools/test-setup.sh | 4 +- tox.ini | 4 +- 147 files changed, 224710 insertions(+), 2163 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index e2c1a283..9620816b 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6 +5,0 @@ alembic>=0.9.6 # MIT -Babel!=2.4.0,>=2.3.4 # BSD @@ -10 +9 @@ dogpile.cache>=0.6.2 # BSD -eventlet!=0.20.1,!=0.21.0,!=0.23.0,!=0.25.0,>=0.20.0 # MIT +eventlet>=0.26.0 # MIT @@ -12 +11 @@ Jinja2>=2.10 # BSD License (3 clause) -jsonschema>=2.6.0 # MIT +jsonschema>=3.2.0 # MIT @@ -15,3 +14,2 @@ kombu!=4.0.2,>=4.6.1 # BSD -mistral-lib>=1.4.0 # Apache-2.0 -networkx<2.3,>=1.10;python_version<'3.0' # BSD -networkx>=2.3;python_version>='3.4' # BSD +mistral-lib>=2.3.0 # Apache-2.0 +networkx>=2.3 # BSD @@ -38 +35,0 @@ tenacity>=5.0.1 # Apache-2.0 -six>=1.10.0 # MIT diff --git a/test-requirements.txt b/test-requirements.txt index f7d71d4e..16c40073 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -4 +4 @@ -hacking>=3.0,<3.1.0 # Apache-2.0 +hacking>=3.0.1,<3.1.0 # Apache-2.0