We are excited to announce the release of: mistral 1.0.1: Mistral Project This release is part of the liberty stable release series. With package available at: https://pypi.python.org/pypi/mistral For more details, please see below. 1.0.1 ^^^^^ Bug Fixes * [bug 1518012 (https://bugs.launchpad.net/keystone/+bug/1518012)] [bug 1513456 (https://bugs.launchpad.net/keystone/+bug/1513456)] Fix concurrency issues by using READ_COMMITTED This release note describes bugs: * #1513456 - task stuck in RUNNING state when all action executions are finished regarding the problem and the fix. * #1518012- WF execution stays in RUNNING although task and action executions are in SUCCESS. This fix does not require any action from Mistral users and does not have any implications other than the bug fix. The state of a workflow execution was not updated even when all task executions were completed if some tasks finished at the same time as other tasks. Because we were using our connections with transaction isolation level = REPEATABLE_READ - Each process was using a snapshot of the DB created at the first read statement in that transaction. When a task finished and evaluated the state of all the other tasks it did not see the up-to-date state of those tasks - and so, because not all tasks were completed - the task did not change the workflow execution state. Similar behavior happened with multiple action executions under same task. On completion, each action execution checked the status of the other action executions and did not see the up-to-date state of these action execution - causing task execution to stay in RUNNING state. The solution is to change DB transaction isolation level from REPEATABLE_READ to READ_COMMITTED so process A can see changes committed in other transactions even if process A is in the middle of a transaction. A short explaination regarding the different isolation levels: * REPEATABLE_READ - while in transaction, the first read operation to the DB creates a snapshot of the entire DB so you are guarantee that all the data in the DB will remain the same until the end of the transaction. REPEATABLE_READ example: * ConnectionA selects from tableA in a transaction. * ConnectionB deletes all rows from tableB in a transaction. * ConnectionB commits. * ConnectionA loops over the rows of tableA and fetches from tableB using the tableA_tableB_FK - ConnectionA will get rows from tableB. * READ_COMMITTED - while in a transaction, every query to the DB will get the committed data. READ_COMMITTED example: * ConnectionA starts a transaction. * ConnectionB starts a transaction. * ConnectionA insert row to tableA and commits. * ConnectionB insert row to tableA. * ConnectionB selects tableA and gets two rows. * ConnectionB commits / rollback. Two good articles about isolation levels are: * Differences between READ-COMMITTED and REPEATABLE-READ transaction isolation levels (https://www.percona.com/blog/2015/01/14/mysql-performance- implications-of-innodb-isolation-modes/). * MySQL performance implications of InnoDB isolation modes (https://www.percona.com/blog/2012/08/28/differences-between- read-committed-and-repeatable-read-transaction-isolation- levels/). Changes in mistral 1.0.0..1.0.1 ------------------------------- aed10de Revert "Add ceilometer action support" ff2c51c Revert "Add Trove to mistral actions" 835d819 Add Trove to mistral actions 097e5cd Updated from global requirements 09f206f Fix concurrency issues by using READ_COMMITTED eeb72e2 Fixing execution saved in wrong tenant 5895441 Updated from global requirements 8c1b5ef Add database indices to improve query performance 50f76b7 Force releasenotes warnings to be treated as errors 543171d Adding releasenotes management to Mistral 5bee630 Send mail to mutli to_addrs failed. b2aa981 Ensure only one WF execution for every CT cycle 33fb462 Avoid comparision between "None" type and "int" type. 501c912 Fixed some unit test issue. 9775b39 Adding callback url to action context 9581f2e Fixing SSH actions to use names of private keys f7cb207 Do not use len() in log_exec decorator be796de Fix all H405 pep8 errors 909ce29 Remove iso8601 dependency 84243f5 Remove kombu as a dependency for Mistral 73d783e Removing wait() when initializing notification listener 84c46be Fix state change on exception during task state change ef886d1 Updated from global requirements 15e42a5 Fixing reference of floating_ips_client in tests 55468eb Pass creds into the clients.Manager() in functional tests 6986c59 Increased size of "state_info" column to 64kb. e09ec65 Wrap sync_db operations in transactions 99a804d Fixing workflow execution state calculation b8e1f95 Fixing wf execution creation at initial stage b56b6a5 Add ceilometer action support bd8927d Updated from global requirements 0d80177 Updated from global requirements 7d7d3da Fix mistral dsvm gate 3c5fbf5 Updated from global requirements bf9266d Updated from global requirements 4e4921d Switch stable/liberty to post-versioning Diffstat (except docs and test files) ------------------------------------- .gitignore | 3 + mistral/actions/std_actions.py | 19 +- mistral/api/controllers/v2/environment.py | 1 + mistral/api/hooks/content_type.py | 4 +- mistral/cmd/launch.py | 29 +- mistral/config.py | 1 - mistral/db/sqlalchemy/base.py | 32 ++- .../008_increase_size_of_state_info_column.py | 34 +++ .../versions/009_add_database_indices.py | 210 ++++++++++++++ mistral/db/sqlalchemy/model_base.py | 4 +- mistral/db/v2/api.py | 6 +- mistral/db/v2/sqlalchemy/api.py | 56 +++- mistral/db/v2/sqlalchemy/models.py | 58 +++- mistral/engine/default_engine.py | 86 +++--- mistral/engine/rpc.py | 6 +- mistral/services/action_manager.py | 51 ++-- mistral/services/actions.py | 37 ++- mistral/services/expiration_policy.py | 2 +- mistral/services/periodic.py | 102 +++++-- mistral/services/scheduler.py | 21 +- mistral/services/triggers.py | 2 +- mistral/services/workflows.py | 45 ++- .../functional/api/v2/test_mistral_basic_v2.py | 42 ++- .../engine/actions/v2/test_ssh_actions.py | 31 ++- .../unit/engine/test_direct_workflow_rerun.py | 2 + .../test_execution_fields_size_limitation.py | 18 +- mistral/utils/__init__.py | 20 +- mistral/utils/javascript.py | 3 +- mistral/utils/rest_utils.py | 8 +- mistral/utils/ssh_utils.py | 29 +- mistral/workflow/direct_workflow.py | 12 +- mistral/workflow/reverse_workflow.py | 6 +- mistral/workflow/with_items.py | 13 +- releasenotes/notes/.placeholder | 0 ...n-level-to-read-committed-7080833ad284b901.yaml | 66 +++++ releasenotes/source/_static/.placeholder | 0 releasenotes/source/_templates/.placeholder | 0 releasenotes/source/conf.py | 272 ++++++++++++++++++ releasenotes/source/index.rst | 9 + releasenotes/source/liberty.rst | 6 + releasenotes/source/unreleased.rst | 5 + requirements.txt | 14 +- setup.cfg | 1 - test-requirements.txt | 1 + tox.ini | 4 +- 70 files changed, 1743 insertions(+), 345 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index f6f445a..42b5e9f 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9 +8,0 @@ eventlet>=0.17.4 -iso8601>=0.1.9 @@ -11,2 +10 @@ jsonschema!=2.5.0,<3.0.0,>=2.0.0 -keystonemiddleware>=2.0.0 -kombu>=3.0.7 +keystonemiddleware!=2.4.0,>=2.0.0 @@ -18,2 +16,2 @@ oslo.db>=2.4.1 # Apache-2.0 -oslo.messaging!=1.17.0,!=1.17.1,>=1.16.0 # Apache-2.0 -oslo.utils>=2.0.0 # Apache-2.0 +oslo.messaging!=1.17.0,!=1.17.1,!=2.6.0,!=2.6.1,!=2.7.0,!=2.8.0,!=2.8.1,!=2.9.0,!=3.1.0,>=1.16.0 # Apache-2.0 +oslo.utils!=2.6.0,>=2.0.0 # Apache-2.0 @@ -29 +27 @@ python-heatclient>=0.3.0 -python-keystoneclient>=1.6.0 +python-keystoneclient!=1.8.0,>=1.6.0 @@ -31 +29 @@ python-neutronclient>=2.6.0 -python-novaclient>=2.28.1 +python-novaclient!=2.33.0,>=2.28.1 @@ -33 +31 @@ PyYAML>=3.1.0 -requests!=2.8.0,>=2.5.2 +requests!=2.8.0,!=2.9.0,>=2.5.2 diff --git a/test-requirements.txt b/test-requirements.txt index 5c7d264..306262f 100644 --- a/test-requirements.txt +++ b/test-requirements.txt @@ -17,0 +18 @@ unittest2 +reno>=0.1.1 # Apache2