November 2019 - Release-announce - lists.openstack.org

mistral 7.1.0 (rocky)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are gleeful to announce the release of: mistral 7.1.0: Mistral Project This release is part of the rocky stable release series. The source is available from: https://opendev.org/openstack/mistral Download the package from: https://pypi.org/project/mistral Please report issues through: https://bugs.launchpad.net/mistral/+bugs For more details, please see below. 7.1.0 ^^^^^ New Features ************ * This makes getting a root_execution_id available to the jinja execution object. Before this it was only possible to get that through filtering and querying the executions search. * Added HTTPProxyToWSGI middleware in front of the Mistral API. The purpose of this middleware is to set up the request URL correctly in the case there is a proxy (for instance, a loadbalancer such as HAProxy) in front of the Mistral API. The HTTPProxyToWSGI is off by default and needs to be enabled via a configuration value. Fixes [bug 1590608 (https://bugs.launchpad.net/mistral/+bug/1590608) Fixes [bug 1816364 (https://bugs.launchpad.net/mistral/+bug/1816364) * It's now possible to add reply-to address when sending email. Bug Fixes ********* * Mistral doesn't log enough info about sending actions to executor and receiving them on the executor side. It makes it hard to debug situations when an action got stuck in RUNNING state. It has now been fixed by adding additional log statements. * Added the "convert_input_data" config property under the "yaql" group. By default it's set to True which preserves the current behavior so there's no risk with compatibility. If set to False, it disables the additional data conversion that was initially added to support some tricky cases like working with sets of dicts (although dict is not a hashable type and can't be put into a set). Disabling it give a significant performance boost in cases when data contexts are very large. * Fixed a backward compatibility issue: there was a change made in Rocky that disallowed the 'params' property of a workflow execution to be None when one wants to start a workflow. * [bug 1837468 (https://bugs.launchpad.net/keystone/+bug/1837468) Fixed unit.actions.openstack.test_generator.GeneratorTest.test_generator failure by avoid triggering version discovery as it needs to talk to Keystone server. Talking to a live Keystone server is not needed for this particular unit test. * Sometimes Mistral was raising DetachedInstanceError for action defintions coming from cache. It's now fixed by cloning objects before caching them. * [bug 1715848 (https://bugs.launchpad.net/mistral/+bug/1715848) Fixed a bug that prevents event-engines to work correctly in HA. * Fix error validate token when run cron trigger. The problem is that a trust client can't do validate token when run cron trigger. * Fixed the issue when "join" task remained in WAITING state forever if the last inbound task failed and it was not a direct predecessor. * If an action execution fails but returns a result as a list (error=[]) the result of this action is assigned to the task execution 'state_info' field which is a string according to the DB model. On Python 3 it this list magically converts to a string. On Python 2.7 it doesn't. The reason is probably in how SQLAlchemy works on different versions of Python. This has now been fixed with an explicit type coercion. * Workflow execution integrity checker mechanism was too agressive in case of big workflows that have many task executions in RUNNING state at the same time. The mechanism was selecting them all in one query and calling "on_action_complete" for each of them within a single DB transaction. That could lead to situations when this mechanism would totally block all normal workflow processing whereas it should only be a "last chance" aid in case of real infrastructure failures (e.g. MQ outage). This issue has been fixed by adding a configurable batch size, so that the checker can't select more than this number of task executions in RUNNING state at once. * For an ad-hoc action, preparing input for its base action was done more than once. It happened during the validation phase and the scheduling phase. However, input preparation may be expensive in case of heavy expressions and data contexts. This has now been fixed by caching a prepared input within an AdHocAction instance. * Action heartbeat checker was using scheduler to process expired action executions periodically. The side effect was that upon system reboot there may have been duplicating delayed calls in the database. So over time, the number of such calls could be significant and those jobs could even affect performance. This has now been fixed with regular threads without using scheduler at all. Additionally, the new configuration property "batch_size" has been added under the group "action_heartbeat" to control the maximum number of action executions processed during one iteration of the action execution heartbeat checker. * Action execution checker didn't set a security context before failing expired action executions. It caused ApplicationContextNotFoundException in case if corresponding workflow specification was not in the cache and Mistral had to load a DB object. The DB operation in turn was trying to access a security context which wasn't set. It's now fixed by setting an admin context in the action execution checker thread. * Workflow and join completion check logic is now simplified with using post transactional queue of operations which is a more generic version of action_queue module previously serving for scheduling action runs outside of the main DB transaction. Workflow completion check is now registered only once when a task completes which reduces clutter and it's registered only if the task may potentially lead to workflow completion. * WorkflowExecution database model had only "root_execution_id" to reference a root workflow execution, i.e. the most parent workflow execution in the execution tree. So if we needed to get an entity itself we'd always make a direct query to the database, in fact, w/o using an entity cache in the SQLAlchemy session. It's now been fixed by adding a normal mapped entity for root workflow execution. In other words, WorkflowExecution class now has the property "root_execution". It slightly improves performance in case this property is accessed more than once per the database session. Changes in mistral 7.0.4..7.1.0 ------------------------------- 4d3ef888 Mask sensitive data when logging action results 4dc6cda9 Make the mistral devstack jobs non-voting in Rocky eab767f3 Add release note for fix error of cron trigger run d1b8dd16 Fix error validate token when run cron trigger f2a0b569 disable triggering version discovery in test_generator unit tests c9e61b80 Add release notes to changes to wflanguage execution object ff1e5b6e Process all task batches in wf output evaluation 81daaf09 Adding root_execution_id to the jinja executor function 9e128262 OpenDev Migration Patch b1b888e7 Add release notes for reply-to feature 5be18b7e Add reply-to to std.email e23b5ced Add release note for I04ba85488b27cb05c3b81ad8c973c3cc3fe56d36 6e05155c Add http_proxy_to_wsgi middleware c8865a33 Add "convert_input_data" config property for YAQL expressions cddaeda9 Add release notes for engine optimizations 77a68312 Add "root_execution" mapped property to WorkflowExecution model 192af4ee Optimize action scheduling 35dc4eec Replace openstack.org git:// URLs with https:// fa0c4f12 Release note for fixing event-engines HA ec06ccf6 [Event-engine] Allow event_engine to work in HA ce3d8607 Add loging for sending an action to executor 5e5901bb Set admin security context in the action execution checker thread a3059ba7 Remove tripleo newton and ocata jobs (rocky branch) 30822d57 Fix "join" when the last indirect inbound task failed dc3a2a81 Fix race condition in refreshing "join" task state 9b302e2d Clone cached action definitions 8133811a Refactor action execution checker without using scheduler d8b19e93 Add batch size for integrity checker 6774a23b Simplify workflow and join completion logic aaebbe40 An execution hangs in the RUNNING state after rerun 4c911a76 Fix how action result is assigned to task 'state_info' field 5d128805 Allow None for 'params' when starting a workflow execution 21299860 Improve workflow completion logic by removing periodic jobs Diffstat (except docs and test files) ------------------------------------- .gitreview | 2 +- .zuul.yaml | 25 ++- bindep.txt | 8 + mistral/actions/std_actions.py | 18 +- mistral/api/app.py | 4 + mistral/api/controllers/root.py | 2 +- mistral/api/controllers/v2/execution.py | 2 +- mistral/api/controllers/v2/root.py | 3 +- mistral/config.py | 41 ++++- mistral/context.py | 2 +- mistral/db/v2/api.py | 12 +- mistral/db/v2/sqlalchemy/api.py | 19 ++- mistral/db/v2/sqlalchemy/models.py | 7 + mistral/engine/action_queue.py | 133 --------------- mistral/engine/actions.py | 85 ++++++---- mistral/engine/default_engine.py | 32 ++-- mistral/engine/engine_server.py | 5 +- mistral/engine/policies.py | 22 +-- mistral/engine/post_tx_queue.py | 131 +++++++++++++++ mistral/engine/task_handler.py | 55 +++++- mistral/engine/tasks.py | 59 ++++++- mistral/engine/workflow_handler.py | 185 +++++++++------------ mistral/engine/workflows.py | 85 ++++++---- mistral/event_engine/default_event_engine.py | 2 - mistral/executors/executor_server.py | 13 +- mistral/rpc/base.py | 2 +- mistral/rpc/clients.py | 18 +- mistral/rpc/kombu/kombu_client.py | 2 +- mistral/rpc/oslo/oslo_client.py | 10 +- mistral/services/action_execution_checker.py | 99 +++++++---- mistral/services/triggers.py | 2 + .../unit/engine/test_direct_workflow_rerun.py | 71 ++++++++ .../engine/test_direct_workflow_with_cycles.py | 43 +++-- .../unit/expressions/test_jinja_expression.py | 8 +- mistral/utils/expression_utils.py | 10 +- mistral/utils/openstack/keystone.py | 8 +- mistral/utils/rest_utils.py | 2 +- mistral/workflow/base.py | 5 + mistral/workflow/data_flow.py | 5 +- mistral/workflow/direct_workflow.py | 15 +- mistral/workflow/lookup_utils.py | 14 +- playbooks/legacy/mistral-ha/run.yaml | 2 +- ...gging_for_sending_actions-c2ddd97027803ecd.yaml | 7 + ...oot_execution_id_to_jinja-90b67c69a50370b5.yaml | 5 + ...nput_data_config_property-09822dee1f46eb8e.yaml | 11 ++ ...workflow_execution_params-f25b752e207d51d7.yaml | 6 + .../notes/bug-1837468-94a8a2e4326ab1e5.yaml | 8 + ...cached_action_definitions-e8b6005b467f35f2.yaml | 5 + .../fix-event-engines-ha-cc78f341095cdabf.yaml | 5 + ...ken_when_run_cron_trigger-7beffc06b75294fb.yaml | 5 + ...t_finished_indirect_error-b0e5adf99cde9a58.yaml | 5 + ...ask_state_info_assignment-e25481ce8c3193ba.yaml | 9 + ...y-to-wsgi-oslo-middleware-f66f1b9533ea1e8a.yaml | 11 ++ ...checker_work_with_batches-56c1cd94200d4c38.yaml | 13 ++ ..._adhoc_actions_scheduling-e324f66f962ae409.yaml | 8 + ...rtbeats_without_scheduler-9c3500d6a2b25a4d.yaml | 12 ++ ..._action_execution_checker-eee7fb697fb213d1.yaml | 9 + ...and_join_completion_check-77a47c5d8953096d.yaml | 10 ++ .../notes/std.email-reply-to-c283770c798db7d0.yaml | 3 + ...entity_for_root_execution-1af6af12ee437282.yaml | 11 ++ tools/config/config-generator.mistral.conf | 1 + 76 files changed, 1397 insertions(+), 480 deletions(-)

1 0

openstack-ansible 18.1.14 (rocky)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are glad to announce the release of: openstack-ansible 18.1.14: Ansible playbooks for deploying OpenStack This release is part of the rocky stable release series. The source is available from: https://opendev.org/openstack/openstack-ansible Download the package from: https://tarballs.openstack.org/null/ Please report issues through: https://bugs.launchpad.net/openstack-ansible/+bugs For more details, please see below. Changes in openstack-ansible 18.1.13..18.1.14 --------------------------------------------- ea3b8911a Bump SHAs for stable/rocky Diffstat (except docs and test files) ------------------------------------- ansible-role-requirements.yml | 10 +-- playbooks/defaults/repo_packages/gnocchi.yml | 2 +- .../defaults/repo_packages/openstack_services.yml | 76 +++++++++++----------- 3 files changed, 44 insertions(+), 44 deletions(-)

1 0

osprofiler 2.9.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are pumped to announce the release of: osprofiler 2.9.0: OpenStack Profiler Library This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/osprofiler Download the package from: https://pypi.org/project/osprofiler Please report issues through: https://bugs.launchpad.net/osprofiler/+bugs For more details, please see below. Changes in osprofiler 2.8.2..2.9.0 ---------------------------------- 29144ca Switch to Ussuri jobs cc868f5 Update master for stable/train 34ca7ab Handle driver initialization errors to avoid service crash Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 2 +- osprofiler/drivers/elasticsearch_driver.py | 6 ++--- osprofiler/drivers/jaeger.py | 4 ++-- osprofiler/drivers/mongodb.py | 6 ++--- osprofiler/drivers/redis_driver.py | 6 ++--- osprofiler/notifier.py | 35 +++++++++++++++++++++--------- releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 +++++ 9 files changed, 53 insertions(+), 22 deletions(-)

1 0

oslo.i18n 3.25.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are glad to announce the release of: oslo.i18n 3.25.0: Oslo i18n library This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.i18n Download the package from: https://pypi.org/project/oslo.i18n Please report issues through: https://bugs.launchpad.net/oslo.i18n/+bugs For more details, please see below. Changes in oslo.i18n 3.24.0..3.25.0 ----------------------------------- 0d76d2c tox: Keeping going with docs cee9c22 Update master for stable/train 2249925 Deprecate Message.translate in favor of Message.translation f8fa059 Allow Message.translate to handle unhashable inputs Diffstat (except docs and test files) ------------------------------------- lower-constraints.txt | 2 +- oslo_i18n/_message.py | 36 ++++++++++++- oslo_i18n/_translate.py | 2 +- releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 +++ tox.ini | 8 ++- 8 files changed, 122 insertions(+), 51 deletions(-)

1 0

oslo.cache 1.38.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We eagerly announce the release of: oslo.cache 1.38.0: Cache storage for OpenStack projects. This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.cache Download the package from: https://pypi.org/project/oslo.cache Please report issues through: https://bugs.launchpad.net/oslo.cache/+bugs For more details, please see below. Changes in oslo.cache 1.37.0..1.38.0 ------------------------------------ 29098ac tox: Keeping going with docs 23101c7 Update master for stable/train ca13430 Update memcached configuration help text Diffstat (except docs and test files) ------------------------------------- oslo_cache/_opts.py | 2 +- releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 ++++++ tox.ini | 7 +++++-- 5 files changed, 15 insertions(+), 5 deletions(-)

1 0

oslo.log 3.45.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are pleased to announce the release of: oslo.log 3.45.0: oslo.log library This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.log Download the package from: https://pypi.org/project/oslo.log Please report issues through: https://bugs.launchpad.net/oslo.log/+bugs For more details, please see below. Changes in oslo.log 3.44.1..3.45.0 ---------------------------------- 3d4070e tox: Keeping going with docs 7fcef4b Update master for stable/train 054df38 Add Ussuri release to versionutils Diffstat (except docs and test files) ------------------------------------- lower-constraints.txt | 2 +- oslo_log/versionutils.py | 2 ++ releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 ++++++ tox.ini | 10 ++++++++-- 6 files changed, 20 insertions(+), 5 deletions(-)

1 0

taskflow 3.8.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are happy to announce the release of: taskflow 3.8.0: Taskflow structured state management library. This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/taskflow Download the package from: https://pypi.org/project/taskflow Please report issues through: https://bugs.launchpad.net/taskflow/+bugs For more details, please see below. Changes in taskflow 3.7.1..3.8.0 -------------------------------- 471b37c3 Switch to Ussuri jobs dc6495cf Update TaskFlow for networkx 2.x 94220173 Update master for stable/train 43365993 Fix python3.8 hmac compatibility Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 2 +- lower-constraints.txt | 2 +- releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 + requirements.txt | 4 +- taskflow/engines/action_engine/compiler.py | 6 +- taskflow/engines/action_engine/deciders.py | 4 +- taskflow/engines/action_engine/engine.py | 2 +- taskflow/engines/action_engine/process_executor.py | 3 +- taskflow/engines/action_engine/runtime.py | 10 +- taskflow/engines/action_engine/scopes.py | 2 +- taskflow/engines/action_engine/traversal.py | 8 +- taskflow/formatters.py | 6 +- taskflow/patterns/graph_flow.py | 11 +- taskflow/patterns/linear_flow.py | 6 +- taskflow/patterns/unordered_flow.py | 4 +- taskflow/types/graph.py | 121 +++------------------ taskflow/utils/misc.py | 5 - 21 files changed, 86 insertions(+), 170 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index e0652e61..92daa08c 100644 --- a/requirements.txt +++ b/requirements.txt @@ -23,2 +23,2 @@ fasteners>=0.7.0 # Apache-2.0 -networkx>=1.10,<2.3;python_version<'3.0' # BSD -networkx>=1.10;python_version>='3.4' # BSD +networkx>=2.1.0,<2.3;python_version<'3.0' # BSD +networkx>=2.1.0;python_version>='3.4' # BSD

1 0

oslo.config 6.12.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are excited to announce the release of: oslo.config 6.12.0: Oslo Configuration API This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.config Download the package from: https://pypi.org/project/oslo.config Please report issues through: https://bugs.launchpad.net/oslo.config/+bugs For more details, please see below. Changes in oslo.config 6.11.1..6.12.0 ------------------------------------- f0c992f Stop configuring install_command in tox. 561db7a fix: fix float opt min and max value format errror 0f7244f tox: Keeping going with docs 064aaa0 Update the constraints url a6302e4 Update master for stable/train Diffstat (except docs and test files) ------------------------------------- lower-constraints.txt | 2 +- oslo_config/generator.py | 4 ++-- releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 ++++++ tox.ini | 8 ++++---- 6 files changed, 16 insertions(+), 9 deletions(-)

1 0

oslo.messaging 10.4.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We are stoked to announce the release of: oslo.messaging 10.4.0: Oslo Messaging API This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.messaging Download the package from: https://pypi.org/project/oslo.messaging Please report issues through: https://bugs.launchpad.net/oslo.messaging/+bugs For more details, please see below. Changes in oslo.messaging 10.3.0..10.4.0 ---------------------------------------- 37fabdd8 Migrate grenade jobs to py3 c8d6fed7 Make sure minimum amqp is 2.5.2 4529981d Switch to Ussuri jobs 1e7a0c6f tox: Keeping going with docs b104f254 Add RPC incoming and reply log 40e5be48 Modify some comments to make them clickable Diffstat (except docs and test files) ------------------------------------- .zuul.yaml | 2 +- lower-constraints.txt | 6 +++--- oslo_messaging/_drivers/impl_fake.py | 2 ++ oslo_messaging/_drivers/impl_rabbit.py | 6 +++--- oslo_messaging/rpc/server.py | 18 ++++++++++++++++++ .../oslo.messaging-src-grenade-dsvm-multinode/run.yaml | 1 + playbooks/oslo.messaging-src-grenade-dsvm/run.yaml | 1 + requirements.txt | 4 ++-- tox.ini | 10 ++++++++-- 11 files changed, 42 insertions(+), 14 deletions(-) Requirements updates -------------------- diff --git a/requirements.txt b/requirements.txt index 7ed73945..6de25a49 100644 --- a/requirements.txt +++ b/requirements.txt @@ -28,2 +28,2 @@ PyYAML>=3.12 # MIT -amqp>=2.4.1 # BSD -kombu!=4.0.2,>=4.6.1 # BSD +amqp>=2.5.2 # BSD +kombu>=4.6.6 # BSD

1 0

oslo.policy 2.4.0 (ussuri)
by no-reply＠openstack.org 25 Nov '19

25 Nov '19

We enthusiastically announce the release of: oslo.policy 2.4.0: Oslo Policy library This release is part of the ussuri release series. The source is available from: https://opendev.org/openstack/oslo.policy Download the package from: https://pypi.org/project/oslo.policy Please report issues through: https://bugs.launchpad.net/oslo.policy/+bugs For more details, please see below. 2.4.0 ^^^^^ Bug Fixes * Deprecated policy warnings are now suppressed in the "oslopolicy- list-redundant" tool so that they don't overwhelm the relevant output. Changes in oslo.policy 2.3.2..2.4.0 ----------------------------------- e6fe8b2 tox: Keeping going with docs 99daead Modernize policy checker 2ba4990 Update the constraints url a235ebf Update master for stable/train 5d3aeb5 Suppress deprecation warnings in oslopolicy-list-redundant 82a2c8d Fix reference cycle caused by deprecated sample override Diffstat (except docs and test files) ------------------------------------- lower-constraints.txt | 2 +- oslo_policy/generator.py | 5 + oslo_policy/policy.py | 8 +- oslo_policy/shell.py | 10 +- ...dant-deprecation-warnings-f84a06133efdaedd.yaml | 6 + releasenotes/source/index.rst | 1 + releasenotes/source/train.rst | 6 + sample_data/auth_v3_token_system_admin.json | 136 ++++++++++++++++++++ tox.ini | 10 +- 15 files changed, 436 insertions(+), 20 deletions(-)

1 0