We enthusiastically announce the release of: mistral 10.0.0: Mistral Project This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/mistral Download the package from: https://pypi.org/project/mistral Please report issues through: https://bugs.launchpad.net/mistral/+bugs For more details, please see below. Changes in mistral 9.0.0.0rc1..10.0.0 ------------------------------------- 545a34c7 Use unittest.mock instead of third party mock c90472d3 [ussuri][goal] Add project specific contributor docs a7da00d7 Use constraints for docs installs 6fb17d82 Add YAQL sanitizing for iterators 75d01bfb Added two new Rally scenarios 2c99ddc3 Update hacking for Python3 0dbab33c Fix serialization of structures that might contain YAQL types fe71a0d3 Bump oslo.serialization version to 2.21.1 a594a252 Keep openstack mapping_path option 019cffb3 Fix ContextView JSON serialization de633d5d Fix incorrect in-depth search of affected tasks cc825ebc Add an article about profiling into the docs 77b450d7 Add a unit test for @tx_cached and fix a bug in it a6839840 Removing duplicated descriptions b7624085 Fix readme 756bd556 Add caching for YAQL/Jinja expression functions b705666f Move preinstalled workflows to mistral-extra 5aa1c704 Fix requirements 8bdf341a Remove OpenStack actions from mistral 592981f4 Refactor expressions 753f1bc0 Fix YAQL engine initialization 95d9f899 Extend task and workflow notification data 253fd978 Release note for "convert_output_data" config option 6dc0c05f Fix adhoc actions 0a3880c2 Make tripleO CI job non-voting f23d3c07 Move registration of CLI options to launch.py 8d757843 Add "convert_output_data" config property for YAQL 829e8225 Init profiler in for a new thread in post_tx_queue.py 5b5576dd Set the delayed call "key" field to the right value 511be4f9 Initialize profiler for scheduler threads eaf0916e Fix fake clients in actions 906291d7 Set tempest configuration to support service api 0ec5e480 Adjust images in the docs and other small fixes 6928615f Move "Long Running Business Process" article from Wiki to docs 9dfdd205 Move the article about testing from Wiki to the built-in docs 12b2eceb Move FAQ from Wiki to the built-in docs 06e776e6 Added a new API to fetch sub-execution of an execution or a task fb3db8dd Add json param to HTTPAction 5c05636e Move "Cloud Cron" cookbook from Wiki to the built-in docs 9fb213c3 The first iteration of restructuring Mistral doc c1c21d7c Disable key sorting in workflow definition 4a630968 wrapped the value of parameters in inputs(in wf get API) with "". 947a986a Gnocchi: fix client options 7f92cc8f Fix typo 504243bc Add 'interface' field to Workflow resource 55f181f8 Fix requirements remove py2 c888a46c Fix keycloak authentication e4d54ec8 Remove mistral-devstack-base 63ab3d40 Add the 1st version of Mistral coding guidelines into the docs 20c34086 Add namespaces to Ad-Hoc actions 8991a20e Fixed a bug regarding workbooks namespace in postgresql d21cf4a7 Designate uses only v2 bb94ca1f Use 406 error instead of 500 error when Service API is not supported 6c677195 Fix duplicated words issue like "from from parsed token" a0663305 Update hacking and fix warnings 852ec7e2 Remove unnecessary comma in help message in conf 1981e63b Don't use eventlet.monkey_patch under wsgi d838607b Add coordination support for devstack e03fa281 Use MISTRAL_SERVICE_HOST as the host ip for standalone 8165fb4c [train][goal] Run 'mistral-devstack-tempest-ipv6-only' job in gate 6948e50d Allow the prefixes like "eq:" and "neq:" in the custom REST UUID type eac23d9e Disable the use of anchors when parsing yaml d1b8c8bc Add a release note for graceful scale-in feature f61929a3 Implement engine graceful shutdown e596ee2e Enlarge tags support 7b58b9c2 Fix log messages in the action heartbeats mechanism 822ea0ec Remove the TripleO standalone job c49212e1 Make it possible to set None to REST API filters fa2042a8 [ussuri][goal] Drop python 2.7 support and testing a25c8fab Mask sensitive data when logging action results 7ec4f267 Make action heartbeats work for all executor types 80ff5228 Make sure minimum amqp is 2.5.2 0e758e16 Refactor action execution reporter 6b862e62 Extend capabilities to clean up old executions 59bf2509 Refactor rerun of joins c7a54d22 Task publish is ignored silently 0224b1ee Add a script that generates a test workflow with needed paralellism fd24972b Fix task expression context a68136d1 Evaluate input expression should check the in_context 50e9d31a Make test_action_definition_cache_ttl more robust 6e892181 tox: Keeping going with docs f01d1772 Prohibit creation workflows with spaces in name aaf68794 Cap psycopg2 to 2.8.3 7d9c9ad7 Using std.ssh without private_key_filename causes TypeError 3bedead7 Add pdf build support 714ba028 Switch to Ussuri jobs f6ec559c adjust doc string to correct key 510713f2 Add Source links to readme 948ceb15 Add release note for fix error of cron trigger run 6284c9dc Add missing :param statement in doc string 10fe69e1 Adjust doc string to correct param 8f02e8a7 Change the action error message format ab4d4a58 Fix the global publish for task c3434b48 New alembic migration to support namespaces in postgresql c0857a7a Log the original exception in is_sync 7a6aac0f Fix "root_execution" lazy loading issue and refactor execution.py 9103b543 Add the explicit dependency from kombu e4f62dab Update master for stable/train 0b8dd84b Add a cookiecutter template to generate custom stuff bdbfb823 Reformat rerun logic for tasks with join Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 65 +- CONTRIBUTING.rst | 69 +- README.rst | 18 +- devstack/plugin.sh | 25 + devstack/settings | 4 + .../{ => admin}/configuration/config-guide.rst | 0 .../{ => admin}/configuration/policy-guide.rst | 0 .../{ => admin}/configuration/samples/index.rst | 0 .../configuration/samples/policy-yaml.rst | 2 +- .../{ => admin}/img/mistral_architecture.png | Bin .../{ => admin}/install/installation_guide.rst | 7 +- .../{ => admin}/install/mistralclient_guide.rst | 84 +- .../developer/contributor/coding_guidelines.rst | 543 +++++++ .../contributor/debugging_and_testing.rst | 144 ++ .../{ => developer}/contributor/devstack.rst | 0 .../contributor}/img/Pycharm_run_config_menu.png | Bin .../contributor/img/dashboard_debug_config.png} | Bin .../contributor/img/dashboard_django_settings.png} | Bin .../img/dashboard_environment_variables.png} | Bin .../contributor/troubleshooting.rst | 16 +- .../extensions}/creating_custom_action.rst | 0 .../extensions}/extending_yaql.rst | 10 +- .../{contributor => user}/asynchronous_actions.rst | 2 +- .../img/cloud_cron_updating_multiple_servers.png | Bin 0 -> 31591 bytes .../{ => user}/terminology/cron_triggers.rst | 5 +- .../terminology/img/actions.png} | Bin .../terminology/img/cron_trigger.png} | Bin .../terminology/img/direct_workflow.png} | Bin .../terminology/img/reverse_workflow.png} | Bin .../terminology/img/workbook_namespacing.png} | Bin .../img/long_running_business_process.png | Bin 0 -> 39543 bytes .../use_cases/long_running_business_process.rst | 73 + lower-constraints.txt | 138 +- mistral/actions/action_generator.py | 31 - mistral/actions/base.py | 91 -- mistral/actions/generator_factory.py | 43 - mistral/actions/openstack/action_generator/base.py | 172 --- mistral/actions/openstack/actions.py | 1094 -------------- mistral/actions/openstack/base.py | 136 -- mistral/actions/openstack/mapping.json | 1518 -------------------- mistral/actions/std_actions.py | 25 +- mistral/api/controllers/v2/action.py | 56 +- mistral/api/controllers/v2/action_execution.py | 5 +- mistral/api/controllers/v2/execution.py | 87 +- mistral/api/controllers/v2/resources.py | 58 +- mistral/api/controllers/v2/service.py | 14 +- mistral/api/controllers/v2/sub_execution.py | 134 ++ mistral/api/controllers/v2/task.py | 78 +- mistral/api/controllers/v2/types.py | 8 +- mistral/auth/keycloak.py | 173 ++- mistral/cmd/launch.py | 36 +- mistral/config.py | 92 +- mistral/context.py | 7 + mistral/db/sqlalchemy/base.py | 16 +- .../versions/035_namespace_support_postgresql.py | 43 + ...6_namespace_support_for_workbooks_postgresql.py | 44 + ...7_add_namespace_column_to_action_definitions.py | 67 + mistral/db/sqlalchemy/migration/cli.py | 18 +- mistral/db/sqlalchemy/model_base.py | 22 + mistral/db/sqlalchemy/types.py | 11 +- mistral/db/utils.py | 90 ++ mistral/db/v2/api.py | 26 +- mistral/db/v2/sqlalchemy/api.py | 64 +- mistral/db/v2/sqlalchemy/models.py | 40 +- mistral/engine/action_handler.py | 15 +- mistral/engine/actions.py | 52 +- mistral/engine/base.py | 8 +- mistral/engine/default_engine.py | 26 +- mistral/engine/engine_server.py | 76 +- mistral/engine/policies.py | 4 +- mistral/engine/post_tx_queue.py | 6 + mistral/engine/task_handler.py | 9 +- mistral/engine/tasks.py | 66 +- mistral/engine/workflows.py | 17 +- mistral/event_engine/default_event_engine.py | 6 +- mistral/event_engine/event_engine_server.py | 2 +- mistral/exceptions.py | 4 + mistral/executors/default_executor.py | 32 +- mistral/executors/executor_server.py | 53 +- .../expressions/{base_expression.py => base.py} | 21 + mistral/expressions/jinja_expression.py | 44 +- .../std_functions.py} | 145 +- mistral/expressions/yaql_expression.py | 135 +- mistral/hacking/checks.py | 22 +- mistral/lang/parser.py | 4 +- mistral/lang/v2/publish.py | 10 + mistral/lang/v2/tasks.py | 39 +- mistral/lang/v2/workflows.py | 6 +- mistral/resources/workflows/create_instance.yaml | 88 -- mistral/resources/workflows/delete_instance.yaml | 25 - mistral/rpc/clients.py | 10 +- mistral/rpc/oslo/oslo_server.py | 1 + mistral/scheduler/default_scheduler.py | 7 + mistral/service/coordination.py | 40 +- mistral/services/action_execution_reporter.py | 93 -- ...tion_checker.py => action_heartbeat_checker.py} | 8 +- mistral/services/action_heartbeat_sender.py | 117 ++ mistral/services/action_manager.py | 69 +- mistral/services/actions.py | 37 +- mistral/services/expiration_policy.py | 13 + mistral/services/legacy_scheduler.py | 10 +- mistral/services/workbooks.py | 15 +- mistral/services/workflows.py | 46 +- .../resources/openstack/action_collection_wb.yaml | 53 - .../actions/openstack/test_openstack_actions.py | 411 ------ .../unit/actions/test_std_mistral_http_action.py | 3 +- .../unit/actions/test_std_test_dict_action.py | 2 +- ...artbeat.py => test_action_heartbeat_checker.py} | 6 +- .../unit/engine/test_action_heartbeat_sender.py | 148 ++ .../unit/engine/test_direct_workflow_rerun.py | 68 +- .../engine/test_direct_workflow_rerun_cancelled.py | 2 +- .../unit/engine/test_disabled_yaql_conversion.py | 243 ++++ .../unit/engine/test_reverse_workflow_rerun.py | 2 +- .../test_reverse_workflow_rerun_cancelled.py | 2 +- .../unit/expressions/test_jinja_expression.py | 19 +- .../expressions/test_yaql_json_serialization.py | 89 ++ mistral/utils/__init__.py | 51 + mistral/utils/filter_utils.py | 66 +- mistral/utils/inspect_utils.py | 94 -- mistral/utils/javascript.py | 37 +- mistral/utils/openstack/keystone.py | 221 +-- mistral/utils/safe_yaml.py | 62 + mistral/utils/ssh_utils.py | 30 +- mistral/workflow/base.py | 9 +- mistral/workflow/data_flow.py | 30 +- mistral/workflow/direct_workflow.py | 31 +- mistral/workflow/states.py | 2 + playbooks/legacy/mistral-ha/run.yaml | 1 + rally-jobs/plugins/mistral_expressions_scenario.py | 142 ++ rally-jobs/task-mistral.yaml | 34 + ...output_data_config_option-4a0fa926a736de7e.yaml | 14 + .../notes/drop-py-2-7-d6ce46d3dc571c01.yaml | 6 + ...ken_when_run_cron_trigger-7beffc06b75294fb.yaml | 5 + ...use_of_action_error_first-4a730a7cbc36f375.yaml | 13 + .../mistral-engine-scale-in-bd348f9237f32481.yaml | 15 + ..._mistral_to_mistral_extra-b3f7bc71ffd72c6e.yaml | 3 + .../notes/namespace_for_adhoc_actions.yaml | 14 + releasenotes/notes/sub_execution_api.yaml | 4 + releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 + requirements.txt | 42 +- setup.cfg | 28 +- test-requirements.txt | 18 +- tools/cookiecutter-mistral-custom/README.rst | 37 + .../cookiecutter-mistral-custom/cookiecutter.json | 8 + .../run_cookiecutter.sh | 12 + .../cookiecutter-mistral-custom/update_actions.sh | 4 + .../{{cookiecutter.project_name}}/LICENSE | 202 +++ .../{{cookiecutter.project_name}}/README.rst | 24 + .../{{cookiecutter.project_name}}/requirements.txt | 6 + .../{{cookiecutter.project_name}}/setup.cfg | 44 + .../{{cookiecutter.project_name}}/setup.py | 19 + .../{{cookiecutter.project_name}}/tox.ini | 29 + .../{{cookiecutter.pkg_name}}}/__init__.py | 0 .../{{cookiecutter.pkg_name}}/actions.py | 25 + .../expression_functions.py | 28 + tools/get_action_list.py | 356 ----- tools/sync_db.py | 2 - tools/wf_generators/generate_parallel_wf.py | 58 + tox.ini | 50 +- 294 files changed, 7584 insertions(+), 6681 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 167cc70f..e2c1a283 100644 --- a/requirements.txt +++ b/requirements.txt @@ -5,2 +5 @@ -alembic>=0.8.10 # MIT -aodhclient>=0.9.0 # Apache-2.0 +alembic>=0.9.6 # MIT @@ -12 +10,0 @@ eventlet!=0.20.1,!=0.21.0,!=0.23.0,!=0.25.0,>=0.20.0 # MIT -gnocchiclient>=3.3.1 # Apache-2.0 @@ -14 +12 @@ Jinja2>=2.10 # BSD License (3 clause) -#jsonschema>=2.6.0 # MIT +jsonschema>=2.6.0 # MIT @@ -16 +14,2 @@ keystonemiddleware>=4.18.0 # Apache-2.0 -mistral-lib>=1.2.0 # Apache-2.0 +kombu!=4.0.2,>=4.6.1 # BSD +mistral-lib>=1.4.0 # Apache-2.0 @@ -22 +21 @@ oslo.context>=2.20.0 # Apache-2.0 -oslo.db>=4.27.0 # Apache-2.0 +oslo.db>=4.40.0 # Apache-2.0 @@ -29 +28 @@ oslo.log>=3.36.0 # Apache-2.0 -oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 +oslo.serialization>=2.21.1 # Apache-2.0 @@ -32 +31 @@ osprofiler>=1.4.0 # Apache-2.0 -paramiko>=2.0.0 # LGPLv2.1+ +paramiko>=2.4.1 # LGPLv2.1+ @@ -35,25 +34,2 @@ pecan>=1.2.1 # BSD -python-barbicanclient>=4.5.2 # Apache-2.0 -python-cinderclient!=4.0.0,>=3.3.0 # Apache-2.0 -python-zaqarclient>=1.0.0 # Apache-2.0 -python-designateclient>=2.7.0 # Apache-2.0 -python-glanceclient>=2.8.0 # Apache-2.0 -python-glareclient>=0.3.0 # Apache-2.0 -python-heatclient>=1.10.0 # Apache-2.0 -python-keystoneclient>=3.8.0 # Apache-2.0 -python-mistralclient!=3.2.0,>=3.1.0 # Apache-2.0 -python-manilaclient>=1.23.0 # Apache-2.0 -python-magnumclient>=2.1.0 # Apache-2.0 -python-muranoclient>=0.8.2 # Apache-2.0 -python-neutronclient>=6.7.0 # Apache-2.0 -python-novaclient>=9.1.0 # Apache-2.0 -python-senlinclient>=1.1.0 # Apache-2.0 -python-swiftclient>=3.2.0 # Apache-2.0 -python-tackerclient>=0.8.0 # Apache-2.0 -python-troveclient>=2.2.0 # Apache-2.0 -python-ironicclient!=2.7.1,!=3.0.0,>=2.7.0 # Apache-2.0 -python-ironic-inspector-client>=1.5.0 # Apache-2.0 -python-vitrageclient>=2.0.0 # Apache-2.0 -python-zunclient>=3.4.0 # Apache-2.0 -python-qinlingclient>=1.0.0 # Apache-2.0 -PyJWT>=1.0.1 # MIT -PyYAML>=3.12 # MIT +PyJWT>=1.5 # MIT +PyYAML>=5.1 # MIT diff --git a/test-requirements.txt b/test-requirements.txt index bb2db372..f7d71d4e 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -4 +4 @@ -hacking>=1.1.0,<1.2.0 # Apache-2.0 +hacking>=3.0,<3.1.0 # Apache-2.0 @@ -7 +6,0 @@ coverage!=4.4,>=4.0 # Apache-2.0 -croniter>=0.3.4 # MIT License @@ -8,0 +8 @@ doc8>=0.6.0 # Apache-2.0 +Pygments>=2.2.0 # BSD license @@ -10,5 +9,0 @@ fixtures>=3.0.0 # Apache-2.0/BSD -keystonemiddleware>=4.18.0 # Apache-2.0 -mistral-lib>=1.2.0 # Apache-2.0 -mock>=2.0.0 # BSD -networkx<2.3,>=1.10;python_version<'3.0' # BSD -networkx>=2.3;python_version>='3.4' # BSD @@ -17,6 +11,0 @@ oslotest>=3.2.0 # Apache-2.0 -oslo.db>=4.27.0 # Apache-2.0 -oslo.messaging>=5.29.0 # Apache-2.0 -oslo.policy>=1.30.0 # Apache-2.0 -osprofiler>=1.4.0 # Apache-2.0 -os-api-ref>=1.4.0 # Apache-2.0 -oauthlib>=0.6.2 # BSD @@ -24 +12,0 @@ requests-mock>=1.2.0 # Apache-2.0 -tooz>=1.58.0 # Apache-2.0 @@ -29 +17 @@ unittest2>=1.1.0 # BSD -WSME>=0.8.0 # MIT +