We are chuffed to announce the release of:

mistral 6.0.0: Mistral Project

This release is part of the queens release series.

Download the package from:

https://pypi.python.org/pypi/mistral

For more details, please see below.

Changes in mistral 5.0.0..6.0.0 -------------------------------

24be746 Really make the cron trigger execution interval configurable 5691ddf Consider size of output_on_error 7c807d3 Tags in workflows were not being properly checked a1ab5d8 Make the cron trigger execution interval configurable 760fb91 Adding Keycloak authorization support. 7fac88c Fix how a cron trigger starts a workflow 8973644 Fixes mistral-server --version command 79028f2 More tests for running workflows based on existing. 0b28e30 Remove achieved goals from the lis of annual goals e0951c1 Fixing grammar mistake c55eaba Using oslo_log instead of logging b79f91e Propagated a task timeout to a action execution 49bdc31 modify the import order 57e6936 Fix docs to better reflect Jinja and YAQL usage 540f8d6 Remove the invalid toctree 78c78ce Add claim_messages and delete_messages zaqar actions 4ac3c7d Fix some reST field lists in docstrings 956a0bb Updated from global requirements 2ddb656 Remove addition of a new task execution to task_executions collection 0aaa43b Disable the wsme Sphinx extension from the API ref docs c3686e0 Fix the 'params' field of the workflow execution REST resource 0562dad Running new workflow based on an existing execution. e9693c5 the word arguements should be arguments 1b5d92c Updated from global requirements dfc03ae Migrate the jobs to native Zuul v3 format c1a07cb TrivialFix: remove redundant import alias b03e641 Remove any old client actions that no longer exist 2ffbc41 Fix break_on calculation in before_task_start cbb7179 Fix std.http action doc b448f88 task name can not be reserved keyword cf0484a Fixed integration of the unit tests with PosrgeSQL 5eaba36 Remove the redundant word 05252da Added session.flush() before update_on_match() 763a131 Added the limit on selection of delayed calls 68825bd Modify error spelling word 7139302 change import order 670150f fix syntax error the 'that' can not be ignore 631144b Updated from global requirements c4f53ea Allow ssh utils to use an absolute path 96f4d3f Updated from global requirements 609f8dc Added the missing options (SCHEDULER_GROUP and CRON_TRIGGER_GROUP) to a generating config. 94ef4e2 Fix the error url 16de7e0 Remove ceilometer actions from mistral e67838b Remove call to sys.exc_clear() in Python 3 467f60a Make workflow execution creation idempotent 6b81707 Add missing user/project name in action context 7184596 Gracefully handle DB disconnected connect errors d0a9d69 Readonly db transactions for testing 4f57c90 Remove intree mistral tempest plugin 93d0b80 Minor cosmetic changes c6e086d Updated from global requirements 9d2f12b Actually add the yaml_dump expression ad733d3 Add executions yaql filter 33db0ad Disable unstable tempest test_run_ssh_proxied_action test 9b356b6 Updated from global requirements 287ba76 Use mock for HTTP calls in unit tests f46825a Updated from global requirements 9a1dd29 Change log level for RestControllers fa5988e Remove the __init__ method from the test action 4283998 Fix inconsistencies when setting policy values f9457b8 Use the new action context in MistralHTTPAction dd4a4bd Pass the new ActionContext to mistral-lib fc3fdc1 Use the latest policy-json-file reference d5d6d1f Clear error info b291e50 Re-work the direct action call tempest test b084b6e Make more CI jobs voting e7b2224 Fix race condition between task completion and child task processing bc686f3 Updated from global requirements 07fcb70 Log a warning log message if the task isn't found 43e453d Fix swift endpoint d53c1a9 Disable unstable tempest test_create_action_execution_sync test cd37173 Disable unstable tempest multi_vim_authentication test d018cd7 Avoid tox_install.sh for constraints support edafce3 Add id field to db query if no sorting order is provided f73fa54 Use a session for keystone auth c7089bf Add new tempest tests for swift and zaqar client actions 20aed8d Updated from global requirements 28130bb Allow filtering executions by their root_execution_id 60d1627 Implement policy in code - docs and reno (end) 4469cac Implement policy in code - event trigger (11) c73fb88 Implement policy in code - workflow (10) 0f6241b Implement policy in code - workbook (9) 643da24 Implement policy in code - service and task (8) 3c999d0 Implement policy in code - member (7) d2274af Implement policy in code - execution (6) 518066b Implement policy in code - environment (5) 22ff1ad Implement policy in code - cron trigger (4) e32fa68 Implement policy in code - action (3) 33407d6 Implement policy in code - action execution (2) 49ed570 Implement policy in code (1) a944cdb Don't use oslo context get_logging_values f254c39 Wrong handling of is_system flag at workbooks causes DB error with MySQL 5.7 bec878e Switch zaqarclient and swiftclient to use a session 76eba6c Stop passing auth_token to ironic-inspector-client 068aa53 Modify log infomation to achieve the same format 5ea1865 zuul: update tripleo zuul v3 jobs f1e32f1 Remove setting of version/release from releasenotes f5bef4a Remove _get_task_executions function 97d4eec Updated from global requirements 6c8f0fa Delete rows directly 4bf3e1e Updated from global requirements 2cd1e6a Fix yaql / json_pp deprecation warning 4ff4eca Remove _get_event_trigger function 0d2ebb7 Add a periodic job to check workflow execution integrity e188e5c Fix wf_trace info adding useless space at some conditions e951513 Remove _get_db_object_by_name_or_id function 1764e25 Use mock for HTTP calls in unit tests 42d7963 Updated from global requirements be618c3 Fix sporadically overwriting of finished workflow execution state bff3ebf Add retries to read-only db operations b5812e0 Remove _get_wf_object_by_name_and_namespace function 9a82e31 Get rid of ensure_* functions from db api 6fc541a Add a json_dump expression function bb0f9f9 Re-raise DB errors when evaluating expressions b26ec5e Updated from global requirements cc548f2 Do not parse updated_at for task if it was not updated 7f469d3 [API] Support get/delete cron triggers in any projects for admin ba15171 [API] Support project_id filter in cron_triggers API 1b6856d Normalize sorting b205e21 'all' parameter breaks task context 100d2c6 Zuul: add file extension to playbook path b199e02 Fix launcher tests 330ebfe Drop pyflakes from the test requirements 6efe094 Add a config option to disable cron triggers c9ddb88 Fix named locks implementation 49e2a53 Remove wrapping of database exceptions in _get_collection() 2c1aa09 Replace or_ with in_ function for searching queries 75e6611 Invoke AuthHook before ContextHook 397a562 Fix deletion of delayed calls 8a0733b Add a yaml_dump expression bf0542c Redundant alias in import statement 826760f Add the Ironic wait_for_provision_state action eeed073 Revert "Enable eventlet monkey patching for MySQLdb driver" 8a81089 Optimize mistral queries for 'get_task_executions' e0eeca6 [Event-engine] Make listener pool name configurable 987930c Updated from global requirements 6bd96d7 Add yaml and json parsing functions 9f87190 Decoupling of Mistral tempest test from Mistral code base f6b6f1d Make scheduler delay configurable 14c8d80 Optimize sending result to parent workflow 2871812 Added created_at and updated_at fields to functions task() and exection() 91ce898 Allow mistral actions to run when authentication is not configured. 68a44fd Mistral fails on RabbitMQ restart 89f98b7 Enable eventlet monkey patching for MySQLdb driver c3eb6d6 remove all common jobs 9e5d57d Add actions for the ironic virtual network interface commands f84952a Add get cron-trigger by id support 3ce3467 Dynamic action name evaluation. 098845e Migrate Mistral jobs to Zuul v3 c07f281 Updated from global requirements 4f8ca81 TrivialFix: Add doc/build directory in .gitignore fbee074 Update README with Keystone authtoken config 6ad421b Replace @loopingcall.RetryDecorator with @tenacity.retry e1d881c Updated from global requirements 5c0b720 Removed NOT IN query from expiration policy. a995916 Use @db_utils.retry_on_deadlock to retry scheduler transactions d2fb361 Updated from global requirements 2018962 Add project_id to API resources 47fae66 Add README.mistral.conf doc in etc directory 76bafad TrivialFix: pretty format the json code block 053308e Add root_execution_id to sub-workflow executions 0fafe6d Use get_rpc_transport instead of get_transport 0b03640 Updated from global requirements 4dbacfe Add mistral/tests/unit/expressions/__init__.py af7a915 Updated from global requirements e416188 Cleanup test_std_http_action 10b27fc Fixes issue rendering strings containing multiple jinja expressions 1783694 Handle case with None encoding during std.http action execution 433a5b4 Clean up screen and tail_log references d4676bf Using current pike stable release for devstack 2daa3c0 Fix Kombu RPC threading and use within multiprocess environment 7b45a50 Fix "with-items" locking d4ae172 Fix to use . to source script files af3d451 Updated from global requirements 8a4fe7f Fix services launcher to handle shutdown properly 8b2f63c Catch DBEntityNotFoundError exceptions for invalid AdHoc Actions 0266c85 Add "API server started." print statement for the API wsgi service bf3c423 Adding doc8 to test-requirements 387633b Updated from global requirements 1c98030 Add ssl support for keycloak auth middleware 1cc8b24 Process input defaults and output transforms for nested AdHoc Actions 517f164 Remove build files before run tox doc builder e379caf Updated from global requirements ff78d7f Dynamic workflow name evaluation. d08479f Fix cron keystone calls when token is available 869e752 Fix test for decoding utf8 6980b30 Update URL and indentations e1ee92d import fails in python3 3f62afd support py3 when doing db_sync 18bab73 Update reno for stable/pike

Diffstat (except docs and test files) -------------------------------------

.gitignore | 2 + .zuul.yaml | 117 ++ README.rst | 61 +- api-ref/source/conf.py | 1 - api-ref/source/index.rst | 4 - devstack/README.rst | 2 +- devstack/plugin.sh | 12 +- etc/README.mistral.conf | 5 + etc/policy.json | 69 +- mistral/actions/generator_factory.py | 2 +- mistral/actions/openstack/actions.py | 232 +-- mistral/actions/openstack/mapping.json | 148 +- mistral/actions/std_actions.py | 57 +- mistral/api/access_control.py | 16 +- mistral/api/app.py | 6 +- mistral/api/controllers/v2/action.py | 18 +- mistral/api/controllers/v2/action_execution.py | 23 +- mistral/api/controllers/v2/cron_trigger.py | 33 +- mistral/api/controllers/v2/environment.py | 21 +- mistral/api/controllers/v2/event_trigger.py | 17 +- mistral/api/controllers/v2/execution.py | 118 +- mistral/api/controllers/v2/member.py | 15 +- mistral/api/controllers/v2/resources.py | 62 +- mistral/api/controllers/v2/service.py | 2 +- mistral/api/controllers/v2/task.py | 30 +- mistral/api/controllers/v2/workbook.py | 18 +- mistral/api/controllers/v2/workflow.py | 25 +- mistral/api/hooks/content_type.py | 2 +- mistral/api/service.py | 12 + mistral/auth/keycloak.py | 34 +- mistral/cmd/launch.py | 80 +- mistral/config.py | 114 +- mistral/context.py | 45 +- .../versions/023_add_root_execution_id.py | 36 + ...d_composite_index_workflow_execution_id_name.py | 34 + mistral/db/sqlalchemy/migration/cli.py | 2 +- mistral/db/utils.py | 43 +- mistral/db/v2/api.py | 75 +- mistral/db/v2/sqlalchemy/api.py | 477 +++---- mistral/db/v2/sqlalchemy/models.py | 21 +- mistral/engine/action_handler.py | 3 +- mistral/engine/action_queue.py | 74 +- mistral/engine/actions.py | 149 +- mistral/engine/base.py | 8 +- mistral/engine/default_engine.py | 53 +- mistral/engine/engine_server.py | 48 +- mistral/engine/policies.py | 72 +- mistral/engine/task_handler.py | 4 + mistral/engine/tasks.py | 199 ++- mistral/engine/workflow_handler.py | 91 +- mistral/engine/workflows.py | 133 +- mistral/exceptions.py | 4 - mistral/executors/base.py | 8 +- mistral/executors/default_executor.py | 31 +- mistral/executors/executor_server.py | 17 +- mistral/executors/remote_executor.py | 4 +- mistral/expressions/jinja_expression.py | 25 +- mistral/expressions/yaql_expression.py | 19 + mistral/lang/base.py | 12 +- mistral/lang/v2/tasks.py | 18 +- mistral/lang/v2/workflows.py | 3 +- mistral/messaging.py | 6 +- mistral/policies/__init__.py | 45 + mistral/policies/action.py | 80 ++ mistral/policies/action_executions.py | 80 ++ mistral/policies/base.py | 33 + mistral/policies/cron_trigger.py | 80 ++ mistral/policies/environment.py | 80 ++ mistral/policies/event_trigger.py | 104 ++ mistral/policies/execution.py | 91 ++ mistral/policies/member.py | 83 ++ mistral/policies/service.py | 36 + mistral/policies/task.py | 58 + mistral/policies/workbook.py | 80 ++ mistral/policies/workflow.py | 91 ++ mistral/rpc/base.py | 2 +- mistral/rpc/clients.py | 76 +- mistral/rpc/kombu/base.py | 5 +- mistral/rpc/kombu/kombu_client.py | 74 +- mistral/rpc/kombu/kombu_listener.py | 25 +- mistral/rpc/kombu/kombu_server.py | 41 +- mistral/services/action_manager.py | 56 - mistral/services/expiration_policy.py | 26 +- mistral/services/periodic.py | 118 +- mistral/services/scheduler.py | 216 ++- mistral/services/triggers.py | 9 +- mistral/services/workbooks.py | 6 +- mistral/services/workflows.py | 6 +- .../actions/openstack/test_openstack_actions.py | 44 +- .../unit/engine/test_direct_workflow_rerun.py | 26 +- .../engine/test_direct_workflow_rerun_cancelled.py | 70 +- .../engine/test_direct_workflow_with_cycles.py | 4 +- .../test_execution_fields_size_limitation.py | 22 +- .../unit/engine/test_reverse_workflow_rerun.py | 6 +- .../test_reverse_workflow_rerun_cancelled.py | 9 +- .../unit/engine/test_subworkflows_pause_resume.py | 33 +- .../unit/expressions/test_jinja_expression.py | 181 ++- mistral/utils/__init__.py | 14 +- mistral/utils/expression_utils.py | 91 +- mistral/utils/openstack/keystone.py | 8 +- mistral/utils/rest_utils.py | 81 +- mistral/utils/ssh_utils.py | 5 +- mistral/utils/wf_trace.py | 3 +- mistral/workflow/data_flow.py | 4 +- mistral/workflow/lookup_utils.py | 3 +- .../services/v2/mistral_client.py | 227 --- .../actions/v2/test_multi_vim_authentication.py | 144 -- .../engine/actions/v2/test_openstack_actions.py | 92 -- .../scenario/engine/actions/v2/test_ssh_actions.py | 294 ---- playbooks/docker-buildimage/post.yaml | 21 + playbooks/docker-buildimage/run.yaml | 6 + playbooks/legacy/mistral-ha/run.yaml | 55 + playbooks/rally/run.yaml | 12 + rally-jobs/task-mistral.yaml | 21 +- ...on-dump-deprecate-json-pp-252c6c495fd2dea1.yaml | 10 + .../drop-ceilometerclient-b33330a28906759e.yaml | 5 + .../policy-and-doc-in-code-9f1737c474998991.yaml | 14 + ...manage-cron-trigger-by-id-ab544e8068b84967.yaml | 3 + .../notes/yaml-json-parse-53217627a647dc1d.yaml | 5 + releasenotes/source/conf.py | 14 +- releasenotes/source/index.rst | 1 + releasenotes/source/pike.rst | 6 + requirements.txt | 63 +- setup.cfg | 17 +- test-requirements.txt | 47 +- tools/config/policy-generator.mistral.conf | 2 + tools/get_action_list.py | 13 - tools/install_venv.py | 2 +- tools/tox_install.sh | 57 - tox.ini | 22 +- 245 files changed, 8911 insertions(+), 7738 deletions(-)

Requirements updates --------------------

diff --git a/requirements.txt b/requirements.txt index c81f27a..7beb073 100644 --- a/requirements.txt +++ b/requirements.txt @@ -6 +6 @@ alembic>=0.8.10 # MIT -aodhclient>=0.7.0 # Apache-2.0 +aodhclient>=0.9.0 # Apache-2.0 @@ -9 +9 @@ croniter>=0.3.4 # MIT License -cachetools>=1.1.0 # MIT License +cachetools>=2.0.0 # MIT License @@ -11 +11 @@ eventlet!=0.18.3,!=0.20.1,<0.21.0,>=0.18.2 # MIT -gnocchiclient>=2.7.0 # Apache-2.0 +gnocchiclient>=3.3.1 # Apache-2.0 @@ -13,16 +13,16 @@ Jinja2!=2.9.0,!=2.9.1,!=2.9.2,!=2.9.3,!=2.9.4,>=2.8 # BSD License (3 clause) -jsonschema!=2.5.0,<3.0.0,>=2.0.0 # MIT -keystonemiddleware>=4.12.0 # Apache-2.0 -mistral-lib>=0.2.0 # Apache-2.0 -networkx>=1.10 # BSD -oslo.concurrency>=3.8.0 # Apache-2.0 -oslo.config!=4.3.0,!=4.4.0,>=4.0.0 # Apache-2.0 -oslo.context>=2.14.0 # Apache-2.0 -oslo.db>=4.24.0 # Apache-2.0 -oslo.i18n!=3.15.2,>=2.1.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 -oslo.middleware>=3.27.0 # Apache-2.0 -oslo.policy>=1.23.0 # Apache-2.0 -oslo.utils>=3.20.0 # Apache-2.0 -oslo.log>=3.22.0 # Apache-2.0 -oslo.serialization!=2.19.1,>=1.10.0 # Apache-2.0 -oslo.service>=1.10.0 # Apache-2.0 +jsonschema<3.0.0,>=2.6.0 # MIT +keystonemiddleware>=4.17.0 # Apache-2.0 +mistral-lib>=0.3.0 # Apache-2.0 +networkx<2.0,>=1.10 # BSD +oslo.concurrency>=3.25.0 # Apache-2.0 +oslo.config>=5.1.0 # Apache-2.0 +oslo.context>=2.19.2 # Apache-2.0 +oslo.db>=4.27.0 # Apache-2.0 +oslo.i18n>=3.15.3 # Apache-2.0 +oslo.messaging>=5.29.0 # Apache-2.0 +oslo.middleware>=3.31.0 # Apache-2.0 +oslo.policy>=1.30.0 # Apache-2.0 +oslo.utils>=3.33.0 # Apache-2.0 +oslo.log>=3.36.0 # Apache-2.0 +oslo.serialization!=2.19.1,>=2.18.0 # Apache-2.0 +oslo.service!=1.28.1,>=1.24.0 # Apache-2.0 @@ -30 +30 @@ osprofiler>=1.4.0 # Apache-2.0 -paramiko>=2.0 # LGPLv2.1+ +paramiko>=2.0.0 # LGPLv2.1+ @@ -34,3 +34,2 @@ python-barbicanclient!=4.5.0,!=4.5.1,>=4.0.0 # Apache-2.0 -python-ceilometerclient>=2.5.0 # Apache-2.0 -python-cinderclient>=3.1.0 # Apache-2.0 -python-designateclient>=1.5.0 # Apache-2.0 +python-cinderclient>=3.3.0 # Apache-2.0 +python-designateclient>=2.7.0 # Apache-2.0 @@ -39 +38 @@ python-glareclient>=0.3.0 # Apache-2.0 -python-heatclient>=1.6.1 # Apache-2.0 +python-heatclient>=1.10.0 # Apache-2.0 @@ -42 +41 @@ python-mistralclient>=3.1.0 # Apache-2.0 -python-magnumclient>=2.0.0 # Apache-2.0 +python-magnumclient>=2.1.0 # Apache-2.0 @@ -45 +44 @@ python-neutronclient>=6.3.0 # Apache-2.0 -python-novaclient>=9.0.0 # Apache-2.0 +python-novaclient>=9.1.0 # Apache-2.0 @@ -50 +49 @@ python-troveclient>=2.2.0 # Apache-2.0 -python-ironicclient>=1.14.0 # Apache-2.0 +python-ironicclient>=2.2.0 # Apache-2.0 @@ -54 +53 @@ PyJWT>=1.0.1 # MIT -PyYAML>=3.10.0 # MIT +PyYAML>=3.10 # MIT @@ -58 +57 @@ setuptools!=24.0.0,!=34.0.0,!=34.0.1,!=34.0.2,!=34.0.3,!=34.1.0,!=34.1.1,!=34.2. -six>=1.9.0 # MIT +six>=1.10.0 # MIT @@ -61,3 +60,3 @@ stevedore>=1.20.0 # Apache-2.0 -WSME>=0.8 # MIT -yaql>=1.1.0 # Apache 2.0 License -tooz>=1.47.0 # Apache-2.0 +WSME>=0.8.0 # MIT +yaql>=1.1.3 # Apache 2.0 License +tooz>=1.58.0 # Apache-2.0 diff --git a/test-requirements.txt b/test-requirements.txt index 90c9a87..4b44a70 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -3,0 +4,2 @@ +hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 + @@ -5,0 +8 @@ croniter>=0.3.4 # MIT License +doc8>=0.6.0 # Apache-2.0 @@ -8,10 +11,9 @@ fixtures>=3.0.0 # Apache-2.0/BSD -hacking!=0.13.0,<0.14,>=0.12.0 # Apache-2.0 -keystonemiddleware>=4.12.0 # Apache-2.0 -mistral-lib>=0.2.0 # Apache-2.0 -mock>=2.0 # BSD -networkx>=1.10 # BSD -nose # LGPL -oslotest>=1.10.0 # Apache-2.0 -oslo.db>=4.24.0 # Apache-2.0 -oslo.messaging!=5.25.0,>=5.24.2 # Apache-2.0 -oslo.policy>=1.23.0 # Apache-2.0 +keystonemiddleware>=4.17.0 # Apache-2.0 +mistral-lib>=0.3.0 # Apache-2.0 +mock>=2.0.0 # BSD +networkx<2.0,>=1.10 # BSD +nose>=1.3.7 # LGPL +oslotest>=3.2.0 # Apache-2.0 +oslo.db>=4.27.0 # Apache-2.0 +oslo.messaging>=5.29.0 # Apache-2.0 +oslo.policy>=1.30.0 # Apache-2.0 @@ -19 +21 @@ osprofiler>=1.4.0 # Apache-2.0 -os-api-ref>=1.0.0 # Apache-2.0 +os-api-ref>=1.4.0 # Apache-2.0 @@ -21,9 +23,8 @@ pecan!=1.0.2,!=1.0.3,!=1.0.4,!=1.2,>=1.0.0 # BSD -pyflakes==0.8.1 # MIT -reno!=2.3.1,>=1.8.0 # Apache-2.0 -requests-mock>=1.1 # Apache-2.0 -sphinx>=1.6.2 # BSD -sphinxcontrib-httpdomain # BSD -sphinxcontrib-pecanwsme>=0.8 # Apache-2.0 -openstackdocstheme>=1.16.0 # Apache-2.0 -tooz>=1.47.0 # Apache-2.0 -tempest>=16.1.0 # Apache-2.0 +reno>=2.5.0 # Apache-2.0 +requests-mock>=1.1.0 # Apache-2.0 +sphinx!=1.6.6,>=1.6.2 # BSD +sphinxcontrib-httpdomain>=1.3.0 # BSD +sphinxcontrib-pecanwsme>=0.8.0 # Apache-2.0 +openstackdocstheme>=1.18.1 # Apache-2.0 +tooz>=1.58.0 # Apache-2.0 +tempest>=17.1.0 # Apache-2.0 @@ -31,3 +32,3 @@ testrepository>=0.0.18 # Apache-2.0/BSD -testtools>=1.4.0 # MIT -unittest2 # BSD -WSME>=0.8 # MIT +testtools>=2.2.0 # MIT +unittest2>=1.1.0 # BSD +WSME>=0.8.0 # MIT