[Openstack] [Nova-orchestration] Preliminary analysis of SpiffWorkflow

Sandy Walsh sandy.walsh at RACKSPACE.COM
Fri Apr 6 20:26:20 UTC 2012


That's great Ziad ... nice work!

Having written one of these libraries before I know the challenges are mostly conceptual, but not terribly technical (fortunately).

Generally the separation between WorkflowSpec and Workflow or TaskSpec and Task is the same as Class and Instance. You define the spec (class) and apply it to many running workflows (instances). 

Side note: this can get you into the problem of versioning too. You define a workflow, spawn 10 instances of it and then change the spec ... do the existing instances change or continue running the old spec? Fun stuff.

Look forward to seeing that larger project!

-S

________________________________________
From: Ziad Sawalha
Sent: Friday, April 06, 2012 4:53 PM
To: Sriram Subramanian; Dugger, Donald D; Sandy Walsh
Cc: nova-orchestration at lists.launchpad.net; openstack at lists.launchpad.net
Subject: Re: [Openstack] [Nova-orchestration] Preliminary analysis of SpiffWorkflow

Here's a link to my analysis so far:
http://wiki.openstack.org/NovaOrchestration/WorkflowEngines/SpiffWorkflow

It looks good, but I won't pass a final verdict until I have completed a
working project in it. I have one in progress and will let ya know when
it's done.

Z

On 4/3/12 4:56 PM, "Ziad Sawalha" <ziad.sawalha at rackspace.com> wrote:

>Just confirming what Sandy said; I am playing around with SpiffWorkflow.
>I'll post my findings when I'm done on the wiki under the Nova
>Orchestration page.
>
>So far I've found some of the documentation lacking and concepts
>confusing, which has resulted in a steep learning curve and made it
>difficult to integrate into something like RabbitMQ (for long-running
>tasks). But the thinking behind it (http://www.workflowpatterns.com/)
>seems sound and I will continue to investigate it.
>
>Z
>
>On 3/29/12 5:56 PM, "Sriram Subramanian" <sriram at computenext.com> wrote:
>
>>Guys,
>>
>>Sorry for missing the meeting today. Thanks for the detailed summary/
>>logs. I am cool with the action item : #action sriram to update the
>>Orchestration session proposal. This is my understanding the logs of
>>things to be updated in the blueprint:
>>
>>1) orchestration service provides state management with client side APIs
>>2) add API design and state storage as topics for the orchestration
>>session at the Summit
>>3) add implementation plan as session topic
>>
>>Please correct me if I missed anything.
>>
>>Just to bring everyone to same page, here are the new links
>>
>>Folsom BluePrint:
>>https://blueprints.launchpad.net/nova/+spec/nova-orchestration
>>Folsom Session proposal:
>>https://blueprints.launchpad.net/nova/+spec/nova-orchestration
>>Wiki: http://wiki.openstack.org/NovaOrchestration (I will clean this up
>>tonight)
>>
>>Maoy: Sandy's pointers are in this email thread (which n0ano meant to fwd
>>you)
>>Mikeyp: Moving the conversation to the main mailing list per your
>>suggestion
>>
>>Thanks,
>>_Sriram
>>
>>-----Original Message-----
>>From: Dugger, Donald D [mailto:donald.d.dugger at intel.com]
>>Sent: Thursday, March 29, 2012 12:52 PM
>>To: Sriram Subramanian; Sandy Walsh
>>Cc: Michael Pittaro (mikeyp at lahondaresearch.org)
>>Subject: RE: [Nova-orchestration] Thoughts on Orchestration (was Re:
>>Documentation on Caching)
>>
>>NP, I'll be on the IRC for whoever wants to talk.  Maybe we can try and
>>do the sync you want via email, that's always been my favorite way to
>>communicate (it allows you to focus thoughts and deals with timezones
>>nicely).
>>
>>--
>>Don Dugger
>>"Censeo Toto nos in Kansa esse decisse." - D. Gale
>>Ph: 303/443-3786
>>
>>
>>-----Original Message-----
>>From: Sriram Subramanian [mailto:sriram at computenext.com]
>>Sent: Thursday, March 29, 2012 1:45 PM
>>To: Sriram Subramanian; Sandy Walsh
>>Cc: Dugger, Donald D; Michael Pittaro (mikeyp at lahondaresearch.org)
>>Subject: RE: [Nova-orchestration] Thoughts on Orchestration (was Re:
>>Documentation on Caching)
>>
>>I will most likely be running little late from my 12 - 1 meeting which
>>doesn't seem to be ending anytime now :(
>>
>>I haven't gotten a chance to submit a branch yet. Hopefully by this week
>>end (at least a bare bones)
>>
>>If you are available for offline sync later this week - I would
>>appreciate that. Apologies for possibly missing the sync.
>>
>>Thanks,
>>-Sriram
>>
>>-----Original Message-----
>>From:
>>nova-orchestration-bounces+sriram=computenext.com at lists.launchpad.net
>>[mailto:nova-orchestration-bounces+sriram=computenext.com at lists.launchpad
>>.
>>net] On Behalf Of Sriram Subramanian
>>Sent: Wednesday, March 28, 2012 2:44 PM
>>To: Sandy Walsh
>>Cc: nova-orchestration at lists.launchpad.net
>>Subject: Re: [Nova-orchestration] Thoughts on Orchestration (was Re:
>>Documentation on Caching)
>>
>>Thanks for the pointers Sandy. I will try to spend some cycles on the
>>branch per your suggestion; we will also discuss more tomorrow.
>>
>>Yes, BP is not far off from last summit, and would like to flush out more
>>for this summit.
>>
>>Thanks,
>>-Sriram
>>
>>-----Original Message-----
>>From: Sandy Walsh [mailto:sandy.walsh at rackspace.com]
>>Sent: Wednesday, March 28, 2012 11:31 AM
>>To: Sriram Subramanian
>>Cc: Michael Pittaro; Dugger, Donald D (donald.d.dugger at intel.com);
>>nova-orchestration at lists.launchpad.net
>>Subject: Thoughts on Orchestration (was Re: Documentation on Caching)
>>
>>Ah, gotcha.
>>
>>I don't think the caching stuff will really affect the Orchestration
>>layer all that much. Certainly the Cells stuff that comstud is working on
>>should be considered.
>>
>>The BP isn't really too far off from what we discussed last summit.
>>Although I would give more consideration to the stuff Redhat is thinking
>>about and some of the efforts by HP and IBM with respect to scheduling
>>(mostly HPC stuff). Unifying and/or understanding those efforts would be
>>important.
>>
>>That said, as with all things OpenStack, code speaks louder than words.
>>The best way to solicit input on an idea is to submit a branch. That's
>>the approach I'd take now if I had the cycles to put back into Orch. I'd
>>likely build something on top of Amazon Workflow services (in such a way
>>as it could be ripped out later) http://aws.amazon.com/swf/ The
>>replacement could be a new OS Service with SWF as the api template.
>>
>>What I've been thinking about lately has been how to make a proof of
>>concept operate with trunk side-by-side without busting the existing
>>stuff. Tricky. Orchestration touches a lot of stuff. The error handling
>>is OS could be an issue and unifying the 3 Enum State Machine on Instance
>>could be nasty (Power, Virt & Task).
>>
>>I know Ziad is playing around with SpiffWorkflow so perhaps he might have
>>some fresh insights?
>>
>>Hope it helps! Nice to see an effort still going on here. I think it's an
>>important project!
>>
>>-S
>>
>>
>>On 03/28/2012 03:08 PM, Sriram Subramanian wrote:
>>> Hi Sandy,
>>>
>>>  We wanted to understand the scheduler improvements in Essex so that we
>>>can update Orchestration blueprint for Folsom. We also wanted to
>>>identify the gaps in the current bp so that we can address them.
>>>
>>> Towards the understanding, Mike and Don suggested during last
>>>Thursday's call that we understand the caching/ scheduler improvements
>>>and be prepared for this week's call.
>>>
>>> Any pointers towards getting the Orchestration proposal right is
>>>appreciated.
>>>
>>> Thanks,
>>> -Sriram
>>>
>>> -----Original Message-----
>>> From: Sandy Walsh [mailto:sandy.walsh at RACKSPACE.COM]
>>> Sent: Wednesday, March 28, 2012 4:40 AM
>>> To: Sriram Subramanian; Michael Pittaro; Dugger, Donald D
>>> (donald.d.dugger at intel.com)
>>> Cc: nova-orchestration at lists.launchpad.net
>>> Subject: RE: Documentation on Caching
>>>
>>> Hi Sriram,
>>>
>>> What specifically are you looking for on Caching (and how would it
>>>apply to Orchestration)?
>>>
>>> -S
>>>
>>> ________________________________________
>>> From: Sriram Subramanian [sriram at computenext.com]
>>> Sent: Tuesday, March 27, 2012 3:16 PM
>>> To: Michael Pittaro; Dugger, Donald D (donald.d.dugger at intel.com);
>>> Sandy Walsh
>>> Cc: nova-orchestration at lists.launchpad.net
>>> Subject: Documentation on Caching
>>>
>>> <Hope I am reaching the correct Michael>
>>>
>>> Michael - can you please point us to the updated documentation on
>>>cachig (as per our meeting last Thursday), so that we can come prepared
>>>this Thursday.
>>>
>>> Thanks,
>>> -Sriram
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>--
>>Mailing list: https://launchpad.net/~nova-orchestration
>>Post to     : nova-orchestration at lists.launchpad.net
>>Unsubscribe : https://launchpad.net/~nova-orchestration
>>More help   : https://help.launchpad.net/ListHelp
>>
>>
>>
>>
>>
>>_______________________________________________
>>Mailing list: https://launchpad.net/~openstack
>>Post to     : openstack at lists.launchpad.net
>>Unsubscribe : https://launchpad.net/~openstack
>>More help   : https://help.launchpad.net/ListHelp
>
>
>_______________________________________________
>Mailing list: https://launchpad.net/~openstack
>Post to     : openstack at lists.launchpad.net
>Unsubscribe : https://launchpad.net/~openstack
>More help   : https://help.launchpad.net/ListHelp





More information about the Openstack mailing list