[openstack-dev] [Mistral] Crack at a "Real life" workflow

Joshua Harlow harlowja at yahoo-inc.com
Sat Mar 8 02:39:48 UTC 2014


@ Dmitri ;)

Sure lets catch up, most always I'm on IRC, feel free to ping me.

Or jump into #openstack-state-management (for more good times).

From: Dmitri Zimine <dz at stackstorm.com<mailto:dz at stackstorm.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Thursday, March 6, 2014 at 10:36 PM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Mistral] Crack at a "Real life" workflow

Folks, thanks for the input!

@Joe:

Hopefully Renat covered the differences.  Yet I am interested in how the same workflow can be expressed as Salt state(s) or Ansible playbooks. Can you (or someone else who knows them well) take a stub?


@Joshua
I am still new to Mistral and learning, but I think it _is_ relevant to taskflow. Should we meet, and you help me catch up? Thanks!

@Sandy:
Aaahr, I used the "D" word?!  :) I keep on arguing that YAML workflow representation doesn't make DSL.

And YES to the object model first to define the workflow, with YAML/JSON/PythonDSL/what-else as a syntax to build it. We are having these discussions on another thread and reviews.

Basically, in order to make a grammar expressive enough to work across a
web interface, we essentially end up writing a crappy language. Instead,
we should focus on the callback hooks to something higher level to deal
with these issues. Minstral should just say "I'm done this task, what
should I do next?" and the callback service can make decisions on where
in the graph to go next.

There must be some misunderstanding. Mistral _does follow AWS / BPEL engines approach, it is both doing "I'm done this task, what should I do next?" (executor) and "callback service" (engine that coordinates the flow and keeps the state). Like decider and activity workers in AWS Simple Workflow.

Engine maintains the state. Executors run tasks. Object model describes workflow as a graph of tasks with transitions, conditions, etc. YAML is one way to define a workflow. Nothing controversial :)

@all:

Wether one writes Python code or uses yaml? Depends on the user. There are good arguments for YAML. But if it's crappy, it looses. We want to see how it feels to write it. To me, mixed feelings so far, but promising. What do you guys think?

Comments welcome here:
https://github.com/dzimine/mistral-workflows/commit/d8c4a8c845e9ca49f6ea94362cef60489f2a46a3


DZ>


On Mar 6, 2014, at 10:41 AM, Sandy Walsh <sandy.walsh at rackspace.com<mailto:sandy.walsh at rackspace.com>> wrote:



On 03/06/2014 02:16 PM, Renat Akhmerov wrote:
IMO, it looks not bad (sorry, I’m biased too) even now. Keep in mind this is not the final version, we keep making it more expressive and concise.

As for killer object model it’s not 100% clear what you mean. As always, devil in the details. This is a web service with all the consequences. I assume what you call “object model” here is nothing else but a python binding for the web service which we’re also working on. Custom python logic you mentioned will also be possible to easily integrate. Like I said, it’s still a pilot stage of the project.

Yeah, the REST aspect is where the "tricky" part comes in :)

Basically, in order to make a grammar expressive enough to work across a
web interface, we essentially end up writing a crappy language. Instead,
we should focus on the callback hooks to something higher level to deal
with these issues. Minstral should just say "I'm done this task, what
should I do next?" and the callback service can make decisions on where
in the graph to go next.

Likewise with things like sending emails from the backend. Minstral
should just call a webhook and let the receiver deal with "active
states" as they choose.

Which is why modelling this stuff in code is usually always better and
why I'd lean towards the TaskFlow approach to the problem. They're
tackling this from a library perspective first and then (possibly)
turning it into a service. Just seems like a better fit. It's also the
approach taken by Amazon Simple Workflow and many BPEL engines.

-S


Renat Akhmerov
@ Mirantis Inc.



On 06 Mar 2014, at 22:26, Joshua Harlow <harlowja at yahoo-inc.com<mailto:harlowja at yahoo-inc.com>> wrote:

That sounds a little similar to what taskflow is trying to do (I am of course biased).

I agree with letting the native language implement the basics (expressions, assignment...) and then building the "domain" ontop of that. Just seems more natural IMHO, and is similar to what linq (in c#) has done.

My 3 cents.

Sent from my really tiny device...

On Mar 6, 2014, at 5:33 AM, "Sandy Walsh" <sandy.walsh at RACKSPACE.COM<mailto:sandy.walsh at RACKSPACE.COM>> wrote:

DSL's are tricky beasts. On one hand I like giving a tool to
non-developers so they can do their jobs, but I always cringe when the
DSL reinvents the wheel for basic stuff (compound assignment
expressions, conditionals, etc).

YAML isn't really a DSL per se, in the sense that it has no language
constructs. As compared to a Ruby-based DSL (for example) where you
still have Ruby under the hood for the basic stuff and extensions to the
language for the domain-specific stuff.

Honestly, I'd like to see a killer object model for defining these
workflows as a first step. What would a python-based equivalent of that
real-world workflow look like? Then we can ask ourselves, does the DSL
make this better or worse? Would we need to expose things like email
handlers, or leave that to the general python libraries?

$0.02

-S



On 03/05/2014 10:50 PM, Dmitri Zimine wrote:
Folks,

I took a crack at using our DSL to build a real-world workflow.
Just to see how it feels to write it. And how it compares with
alternative tools.

This one automates a page from OpenStack operation
guide: http://docs.openstack.org/trunk/openstack-ops/content/maintenance.html#planned_maintenance_compute_node

Here it is https://gist.github.com/dzimine/9380941
or here http://paste.openstack.org/show/72741/

I have a bunch of comments, implicit assumptions, and questions which
came to mind while writing it. Want your and other people's opinions on it.

But gist and paste don't let annotate lines!!! :(

May be we can put it on the review board, even with no intention to
check in,  to use for discussion?

Any interest?

DZ>


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140308/3806eb7e/attachment.html>


More information about the OpenStack-dev mailing list