Open Stack

After some online discussions with Renat, the following is a revision of
the proposal to address the following related blueprints.
*
https://blueprints.launchpad.net/mistral/+spec/mistral-execution-environment
* https://blueprints.launchpad.net/mistral/+spec/mistral-global-context
*
https://blueprints.launchpad.net/mistral/+spec/mistral-default-input-values
* https://blueprints.launchpad.net/mistral/+spec/mistral-runtime-context

Please refer to the following threads for backgrounds.
*
http://lists.openstack.org/pipermail/openstack-dev/2014-December/052643.html
*
http://lists.openstack.org/pipermail/openstack-dev/2014-December/052960.html
*
http://lists.openstack.org/pipermail/openstack-dev/2014-December/052824.html


*Workflow Context Scope*
1. context to workflow is passed to all its subflows and subtasks/actions
(aka children) only explicitly via inputs
2. context are passed by value (copy.deepcopy) to children
3. change to context is passed to parent only when it's explicitly
published at the end of the child execution
4. change to context at the parent (after a publish from a child) is passed
to subsequent children

*Environment Variables*
Solves the problem for quickly passing pre-defined inputs to a WF
execution.  In the WF spec, environment variables are referenced as
$.env.var1, $.env.var2, etc.  We should implement an API and DB model where
users can pre-defined different environments with their own set of
variables.  An environment can be passed either by name from the DB or
adhoc by dict in start_workflow.  On workflow execution, a copy of the
environment is saved with the execution object.  Action inputs are still
declared explicitly in the WF spec.  This does not solve the problem where
common inputs are specified over and over again.  So if there are multiple
SQL tasks in the WF, the WF author still needs to supply the conn_str
explicitly for each task.  In the example below, let's say we have a SQL
Query Action that takes a connection string and a query statement as
inputs.  The WF author can specify that the conn_str input is supplied from
the $.env.conn_str.

*Example:*

# Assume this SqlAction is registered as std.sql in Mistral's Action table.
class SqlAction(object):
    def __init__(self, conn_str, query):
        ...

...

version: "2.0"
workflows:
    demo:
        type: direct
        input:
            - query
        output:
            - records
        tasks:
            query:
                action: std.sql conn_str={$.env.conn_str} query={$.query}
                publish:
                    records: $

...

my_adhoc_env = {
    "conn_str": "mysql://admin:secrete <mysql://admin:secrete@>@localhost
/test"
}

...

# adhoc by dict
start_workflow(wf_name, wf_inputs, env=my_adhoc_env)

OR

# lookup by name from DB model
start_workflow(wf_name, wf_inputs, env="my_lab_env")


*Define Default Action Inputs as Environment Variables*
Solves the problem where we're specifying the same inputs to subflows and
subtasks/actions over and over again.  On command execution, if action
inputs are not explicitly supplied, then defaults will be lookup from the
environment.

*Example:*
Using the same example from above, the WF author can still supply both
conn_str and query inputs in the WF spec.  However, the author also has the
option to supply that as default action inputs.  An example environment
structure is below.  "__actions" should be reserved and immutable.  Users
can specific one or more default inputs for the sql action as nested dict
under "__actions".  Recursive YAQL eval should be supported in the env
variables.

version: "2.0"
workflows:
    demo:
        type: direct
        input:
            - query
        output:
            - records
        tasks:
            query:
                action: std.sql query={$.query}
                publish:
                    records: $

...

my_adhoc_env = {
    "sql_server": "localhost",
    "__actions": {
        "std.sql": {
            "conn_str": "mysql://admin:secrete@{$.env.sql_server}/test"
         }
    }
}


*Default Input Values Supplied Explicitly in WF Spec*
Please refer to this blueprint
<https://blueprints.launchpad.net/mistral/+spec/mistral-default-input-values>
for background.  This is a different use case.  To support, we just need to
set the correct order of precedence in applying values.
1. Input explicitly given to the sub flow/task in the WF spec
2. Default input supplied from env
3. Default input supplied at WF spec

*Putting this together...*
At runtime, the WF context would be similar to the following example.  This
will be use to recursively eval the inputs for subflow/tasks/actions.

ctx = {
   "var1": …,
   "var2": …,
   "my_server_ip": "10.1.23.250",
   "env": {
        "sql_server": "localhost",
        "__actions": {
            "std.sql": {
                "conn": "mysql://admin:secrete@{$.env.sql_server}/test"
            },
            "my.action": {
                "endpoint": "http://{$.my_server_ip}/v1/foo"
            }
        }
   }
}

*Runtime Context*
Please refer to this thread
<http://lists.openstack.org/pipermail/openstack-dev/2014-December/052824.html>
for
the background and discussion.  The only change here is that on run_action,
we will pass the runtime data as kwargs to all action invocation.  This
means all Action classes should have at least **kwargs added to the
__init__ method.

runtime = {
   "execution_id": ...,
   "task_id": ...,
   ...
}

...

action = SomeAction(..., runtime=runtime)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141223/27c9c889/attachment.html>