We exuberantly announce the release of: mistral 8.0.0: Mistral Project This release is part of the stein release series. The source is available from: https://git.openstack.org/cgit/openstack/mistral Download the package from: https://pypi.org/project/mistral Please report issues through: https://bugs.launchpad.net/mistral/+bugs For more details, please see below. 8.0.0 ^^^^^ Bug Fixes * For an ad-hoc action, preparing input for its base action was done more than once. It happened during the validation phase and the scheduling phase. However, input preparation may be expensive in case of heavy expressions and data contexts. This has now been fixed by caching a prepared input within an AdHocAction instance. * WorkflowExecution database model had only "root_execution_id" to reference a root workflow execution, i.e. the most parent workflow execution in the execution tree. So if we needed to get an entity itself we'd always make a direct query to the database, in fact, w/o using an entity cache in the SQLAlchemy session. It's now been fixed by adding a normal mapped entity for root workflow execution. In other words, WorkflowExecution class now has the property "root_execution". It slightly improves performance in case this property is accessed more than once per the database session. Changes in mistral 7.0.0..8.0.0 ------------------------------- 1f146bfb Set the lower version of sphinxcontrib-pecanwsme to 0.10.0 65102f03 Replace openstack.org git:// URLs with https:// 4c9e35f1 Add "root_execution" mapped property to WorkflowExecution model a8133ef3 Optimize action scheduling 65c33ca7 Temporarily disable TripleO jobs in stable/stein 228090dc Add release notes for engine optimizations 3efb5378 Update UPPER_CONSTRAINTS_FILE for stable/stein 99ebc1b5 Retries shouldn't execute if join task failed because of child task d660d2b1 add python 3.7 unit test job 43f80eaa standalone/undercloud do not cover full mistral workflow 0085d08b Stop using deprecated keystone_authtoken/auth_uri aa30e18c Update dogpile.cache to match global requirements 1d16d2ff Revert "Fix how Mistral prepares data for evaluating a YAQL expression" ed7c351d Release note for fixing event-engines HA a39db2d3 Fix how Mistral prepares data for evaluating a YAQL expression ae2c5fdb Add a workflow execution report endpoint 87200f6a Sending TASK_FAILED event in case of MistralException 81af1b48 Process all task batches in wf output evaluation 2f4db0a6 Replace tripleo-scenario003-multinode with scenario003-standalone c7509a9c Fix the misspelling of "default" efca6f2e fix typo mistakes 7301d8cb Amend the spelling error of a word 98589e12 Update mailinglist from dev to discuss 278fc331 Remove those copy words occured twice times in wf_lang_v2.rst 44454117 Gate fix for failing at openstack-tox-docs 0fb96bbd Add loging for sending an action to executor 99be4227 Clarify REST field valiation error 12a2fe87 Remove tripleo newton and ocata jobs 3153f1ca Set admin security context in the action execution checker thread 09ad2eeb Fix the tests for workflow "started_at" and "finished_at" times e0522a33 Add Python 3.6 classifier to setup.cfg 38cc50bc Eliminating datetime.now() f9b355f6 Add missing ws separator between words ea7fa0e4 Add started_at and finished_at to task execution. c9e08a88 Fix "join" when the last indirect inbound task failed 90ddf442 Clone cached action definitions 05ce6f89 Fix race condition in refreshing "join" task state 0744042d Omit the twice occured words in index.rst a82c4918 Remove setup.py check from pep8 job 43a8bddc Fix how action result is assigned to task 'state_info' field c712e369 Divide yaml input to save it into definitions separately. 2d74e6eb Refactor action execution checker without using scheduler 3b4136ff Add batch size for integrity checker 80a1bed6 Simplify workflow and join completion logic b413aa08 Allow None for 'params' when starting a workflow execution b1dd0613 Update min tox version to 2.0 3d7acd39 Improve workflow completion logic by removing periodic jobs ec3d1411 Fix senlin fake client creation c39842b8 Fix usage of cachetools in lookup_utils 1a4c599a Improve join by removing periodic jobs 0aa73edb Mistral install guide defff087 [Event-engine] Allow event_engine to work in HA 0b38cd80 Reduce the concurrency in the 500 wb join Rally task 041f3bd3 An execution hangs in the RUNNING state after rerun 991734a2 Add sqlalchemy.exc.OperationalError to the retry decorator ae23de73 make user_info_endpoint_url independent of auth_url b902b963 Increment versioning with pbr instruction e98614cd Update OnClauseSPec task name criteria 08ba20c8 Fix next link in get resource list rest API 9be7e928 Remove remaining references to the rpc_backend 488b4083 Update version.version_string to actually be a string 64622cff Don't quote {posargs} in tox.ini f0b49a82 Add a release note for Ic98e2db02abd8483591756d73e06784cc2e9cbe3 c802ad28 Make task execution logging more readable and informative 5c005a79 Cleanup transport along RPC clients 5e3cdec9 Add entry point to allow for oslo.policy CLI usage dfdff783 Fix how Mistral calculates workflow output f85e57da Fix SSHAction under python3 c93b45a6 Remove extra information from std.ssh action 5a70238e Add release note for auth_context bugfix 219ffa60 Minor improvement of the NoopPublisher bd198eac Minor bug of _assert_multiple_items function 8f40b01c Performance: remove unnecessary workflow execution update fdca8f89 add python 3.6 unit test job 4a9ceaf4 switch documentation job to new PTI 1f8ae7fc Increase delayed_calls_v2.auth_context aa60f7c1 Add py36 to tox and default to python3 for pep8 and venv 5c9aad65 Use mock to patch lookup_utils._ACTION_DEF_CACHE ea40339d Fix some format errors in installation guide. b16a4ce4 Explicitly convert X-Target-Insecure to a boolean 1ea5cf82 Remove -u root as mysql is executed with root user 1c249f02 import zuul job settings from project-config 72f5f77a Add tripleo-ci-centos-7-undercloud-containers job 7b71f096 New experimental scheduler: the first working version 6b4cc9a4 Removes non needed parameter passed in magnum client creation 7b5bffe7 Update reno for stable/rocky 93d4a7f8 Fix error workbook example Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 46 ++- CONTRIBUTING.rst | 2 +- devstack/plugin.sh | 2 +- lower-constraints.txt | 4 +- mistral/actions/openstack/actions.py | 6 +- mistral/actions/std_actions.py | 5 +- mistral/api/access_control.py | 23 ++ mistral/api/controllers/resource.py | 11 +- mistral/api/controllers/v2/execution.py | 4 +- mistral/api/controllers/v2/execution_report.py | 164 +++++++++ mistral/api/controllers/v2/resources.py | 158 ++++++++ mistral/api/controllers/v2/task.py | 11 +- mistral/auth/keycloak.py | 12 +- mistral/cmd/launch.py | 2 +- mistral/config.py | 54 ++- mistral/context.py | 15 +- .../030_increase_delayed_calls_v2_auth_context.py | 36 ++ ...started_at_and_finished_at_to_task_execution.py | 40 ++ mistral/db/utils.py | 11 +- mistral/db/v2/api.py | 46 ++- mistral/db/v2/sqlalchemy/api.py | 144 +++++++- mistral/db/v2/sqlalchemy/models.py | 52 ++- mistral/engine/action_queue.py | 133 ------- mistral/engine/actions.py | 79 ++-- mistral/engine/default_engine.py | 32 +- mistral/engine/dispatcher.py | 15 +- mistral/engine/engine_server.py | 5 +- mistral/engine/policies.py | 36 +- mistral/engine/post_tx_queue.py | 131 +++++++ mistral/engine/task_handler.py | 215 +++++++---- mistral/engine/tasks.py | 101 ++++- mistral/engine/workflow_handler.py | 190 +++++----- mistral/engine/workflows.py | 95 +++-- mistral/event_engine/default_event_engine.py | 2 - mistral/executors/executor_server.py | 13 +- mistral/lang/v2/on_clause.py | 2 +- mistral/notifiers/publishers/noop.py | 3 +- mistral/rpc/base.py | 2 +- mistral/rpc/clients.py | 20 +- mistral/rpc/kombu/base.py | 5 +- mistral/rpc/kombu/kombu_client.py | 2 +- mistral/rpc/kombu/kombu_hosts.py | 21 +- mistral/rpc/kombu/kombu_listener.py | 2 +- mistral/rpc/oslo/oslo_client.py | 10 +- mistral/scheduler/__init__.py | 0 mistral/scheduler/base.py | 78 ++++ mistral/scheduler/default_scheduler.py | 292 +++++++++++++++ mistral/scheduler/scheduler_server.py | 68 ++++ mistral/services/action_execution_checker.py | 127 +++++-- mistral/services/scheduler.py | 5 +- mistral/services/triggers.py | 2 + mistral/services/workflows.py | 38 +- .../unit/engine/test_direct_workflow_rerun.py | 71 ++++ .../engine/test_direct_workflow_with_cycles.py | 43 ++- .../unit/engine/test_subworkflows_pause_resume.py | 30 -- .../unit/engine/test_task_started_finished_at.py | 188 ++++++++++ mistral/utils/__init__.py | 4 + mistral/utils/openstack/keystone.py | 8 +- mistral/utils/rest_utils.py | 4 +- mistral/utils/ssh_utils.py | 7 +- mistral/version.py | 2 +- mistral/workflow/base.py | 18 +- mistral/workflow/commands.py | 4 +- mistral/workflow/data_flow.py | 5 +- mistral/workflow/direct_workflow.py | 62 +++- mistral/workflow/lookup_utils.py | 39 +- mistral/workflow/reverse_workflow.py | 3 + playbooks/legacy/mistral-ha/run.yaml | 2 +- rally-jobs/task-mistral.yaml | 8 +- ...gging_for_sending_actions-c2ddd97027803ecd.yaml | 7 + ...workflow_execution_params-f25b752e207d51d7.yaml | 6 + ...nup-rpc-cleints-transport-eaa90fef070b81fd.yaml | 8 + ...cached_action_definitions-e8b6005b467f35f2.yaml | 5 + ...rkflows-within-namespaces-e4fba869a889f55f.yaml | 4 +- ...-context-with-big-catalog-7647a07d616e653f.yaml | 12 + .../fix-event-engines-ha-cc78f341095cdabf.yaml | 5 + .../fix-next-url-formatting-2cc0d8a27625c73a.yaml | 9 + ...t_finished_indirect_error-b0e5adf99cde9a58.yaml | 5 + ...ask_state_info_assignment-e25481ce8c3193ba.yaml | 9 + .../fix_workflow_output-cee5df431679de6b.yaml | 13 + ...checker_work_with_batches-56c1cd94200d4c38.yaml | 13 + ..._adhoc_actions_scheduling-e324f66f962ae409.yaml | 8 + ...rtbeats_without_scheduler-9c3500d6a2b25a4d.yaml | 12 + .../remove_polling_from_join-3a7921c4af741822.yaml | 9 + ...workflow_execution_update-bdc9526bd39539c4.yaml | 10 + ..._action_execution_checker-eee7fb697fb213d1.yaml | 9 + ...and_join_completion_check-77a47c5d8953096d.yaml | 10 + ...entity_for_root_execution-1af6af12ee437282.yaml | 11 + .../x-target-insecure-values-4b2bdbfd42526abc.yaml | 8 + releasenotes/source/index.rst | 1 + releasenotes/source/rocky.rst | 6 + requirements.txt | 3 +- setup.cfg | 8 +- test-requirements.txt | 2 +- tox.ini | 15 +- 140 files changed, 4325 insertions(+), 821 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 7fbac3fe..f21d2750 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9,0 +10 @@ cachetools>=2.0.0 # MIT License +dogpile.cache>=0.6.2 # BSD @@ -14 +15 @@ jsonschema<3.0.0,>=2.6.0 # MIT -keystonemiddleware>=4.17.0 # Apache-2.0 +keystonemiddleware>=4.18.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index b8f5540f..c2ce4d6c 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -10 +10 @@ fixtures>=3.0.0 # Apache-2.0/BSD -keystonemiddleware>=4.17.0 # Apache-2.0 +keystonemiddleware>=4.18.0 # Apache-2.0