[openstack-dev] [Mistral] Crack at a "Real life" workflow

Dmitri Zimine dz at stackstorm.com
Fri Mar 7 06:36:46 UTC 2014


Folks, thanks for the input! 

@Joe: 

Hopefully Renat covered the differences.  Yet I am interested in how the same workflow can be expressed as Salt state(s) or Ansible playbooks. Can you (or someone else who knows them well) take a stub? 


@Joshua
I am still new to Mistral and learning, but I think it _is_ relevant to taskflow. Should we meet, and you help me catch up? Thanks! 

@Sandy:
Aaahr, I used the "D" word?!  :) I keep on arguing that YAML workflow representation doesn't make DSL. 

And YES to the object model first to define the workflow, with YAML/JSON/PythonDSL/what-else as a syntax to build it. We are having these discussions on another thread and reviews. 

> Basically, in order to make a grammar expressive enough to work across a
> web interface, we essentially end up writing a crappy language. Instead,
> we should focus on the callback hooks to something higher level to deal
> with these issues. Minstral should just say "I'm done this task, what
> should I do next?" and the callback service can make decisions on where
> in the graph to go next.

There must be some misunderstanding. Mistral _does follow AWS / BPEL engines approach, it is both doing "I'm done this task, what should I do next?" (executor) and "callback service" (engine that coordinates the flow and keeps the state). Like decider and activity workers in AWS Simple Workflow.

Engine maintains the state. Executors run tasks. Object model describes workflow as a graph of tasks with transitions, conditions, etc. YAML is one way to define a workflow. Nothing controversial :) 

@all:

Wether one writes Python code or uses yaml? Depends on the user. There are good arguments for YAML. But if it's crappy, it looses. We want to see how it feels to write it. To me, mixed feelings so far, but promising. What do you guys think?

Comments welcome here: 
https://github.com/dzimine/mistral-workflows/commit/d8c4a8c845e9ca49f6ea94362cef60489f2a46a3


DZ> 


On Mar 6, 2014, at 10:41 AM, Sandy Walsh <sandy.walsh at rackspace.com> wrote:

> 
> 
> On 03/06/2014 02:16 PM, Renat Akhmerov wrote:
>> IMO, it looks not bad (sorry, I’m biased too) even now. Keep in mind this is not the final version, we keep making it more expressive and concise.
>> 
>> As for killer object model it’s not 100% clear what you mean. As always, devil in the details. This is a web service with all the consequences. I assume what you call “object model” here is nothing else but a python binding for the web service which we’re also working on. Custom python logic you mentioned will also be possible to easily integrate. Like I said, it’s still a pilot stage of the project.
> 
> Yeah, the REST aspect is where the "tricky" part comes in :)
> 
> Basically, in order to make a grammar expressive enough to work across a
> web interface, we essentially end up writing a crappy language. Instead,
> we should focus on the callback hooks to something higher level to deal
> with these issues. Minstral should just say "I'm done this task, what
> should I do next?" and the callback service can make decisions on where
> in the graph to go next.
> 
> Likewise with things like sending emails from the backend. Minstral
> should just call a webhook and let the receiver deal with "active
> states" as they choose.
> 
> Which is why modelling this stuff in code is usually always better and
> why I'd lean towards the TaskFlow approach to the problem. They're
> tackling this from a library perspective first and then (possibly)
> turning it into a service. Just seems like a better fit. It's also the
> approach taken by Amazon Simple Workflow and many BPEL engines.
> 
> -S
> 
> 
>> Renat Akhmerov
>> @ Mirantis Inc.
>> 
>> 
>> 
>> On 06 Mar 2014, at 22:26, Joshua Harlow <harlowja at yahoo-inc.com> wrote:
>> 
>>> That sounds a little similar to what taskflow is trying to do (I am of course biased).
>>> 
>>> I agree with letting the native language implement the basics (expressions, assignment...) and then building the "domain" ontop of that. Just seems more natural IMHO, and is similar to what linq (in c#) has done.
>>> 
>>> My 3 cents.
>>> 
>>> Sent from my really tiny device...
>>> 
>>>> On Mar 6, 2014, at 5:33 AM, "Sandy Walsh" <sandy.walsh at RACKSPACE.COM> wrote:
>>>> 
>>>> DSL's are tricky beasts. On one hand I like giving a tool to
>>>> non-developers so they can do their jobs, but I always cringe when the
>>>> DSL reinvents the wheel for basic stuff (compound assignment
>>>> expressions, conditionals, etc).
>>>> 
>>>> YAML isn't really a DSL per se, in the sense that it has no language
>>>> constructs. As compared to a Ruby-based DSL (for example) where you
>>>> still have Ruby under the hood for the basic stuff and extensions to the
>>>> language for the domain-specific stuff.
>>>> 
>>>> Honestly, I'd like to see a killer object model for defining these
>>>> workflows as a first step. What would a python-based equivalent of that
>>>> real-world workflow look like? Then we can ask ourselves, does the DSL
>>>> make this better or worse? Would we need to expose things like email
>>>> handlers, or leave that to the general python libraries?
>>>> 
>>>> $0.02
>>>> 
>>>> -S
>>>> 
>>>> 
>>>> 
>>>>> On 03/05/2014 10:50 PM, Dmitri Zimine wrote:
>>>>> Folks, 
>>>>> 
>>>>> I took a crack at using our DSL to build a real-world workflow. 
>>>>> Just to see how it feels to write it. And how it compares with
>>>>> alternative tools. 
>>>>> 
>>>>> This one automates a page from OpenStack operation
>>>>> guide: http://docs.openstack.org/trunk/openstack-ops/content/maintenance.html#planned_maintenance_compute_node 
>>>>> 
>>>>> Here it is https://gist.github.com/dzimine/9380941
>>>>> or here http://paste.openstack.org/show/72741/
>>>>> 
>>>>> I have a bunch of comments, implicit assumptions, and questions which
>>>>> came to mind while writing it. Want your and other people's opinions on it. 
>>>>> 
>>>>> But gist and paste don't let annotate lines!!! :(
>>>>> 
>>>>> May be we can put it on the review board, even with no intention to
>>>>> check in,  to use for discussion? 
>>>>> 
>>>>> Any interest?
>>>>> 
>>>>> DZ> 
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140306/e61f013a/attachment.html>


More information about the OpenStack-dev mailing list