Open Stack

We looked at SpiffWorkflow for nova a while back and we've been using it at Rackspace. It's not WFaaS, but it's a solid library if you want a simple workflow engine inside your application. 


Typoed on an autocorrecting mobile device...

On Apr 18, 2013, at 9:34 AM, "Joshua Harlow" <harlowja at yahoo-inc.com> wrote:

> Ok good to know! 
> 
> My major concern here though is that nova IMHO needs a solution right now and although it would be nice to wait for this common library and placement in oslo and so on. There might be paths that can be taken that don't require waiting for the optimal final solution to be perfected by heat. 
> 
> Any thoughts on how quickly said library can be carved out of heat (or created?) 
> 
> I would like to see how we fix nova now in all honesty, while designing said fixes so that they can be easily moved to this convection library when it's ready instead of waiting to fix nova until said convection library has matured.
> 
> Sent from my really tiny device...
> 
> On Apr 18, 2013, at 12:54 AM, "Adrian Otto" <adrian.otto at rackspace.com> wrote:
> 
>> Joshua,
>> 
>> On Apr 17, 2013, at 10:37 PM, Joshua Harlow <harlowja at yahoo-inc.com> wrote:
>> 
>>> I'd be very interested in having 'Scheduler & Workflow Service' part there
>>> be a library.
>>> 
>>> Pretty much every application is a workflow in some way, and using said
>>> library in nova for the orc work there would be very neat.
>> 
>> We had an unconference session today when we discussed this concept:
>> 
>> https://wiki.openstack.org/wiki/Convection
>> 
>> The idea is that this could start as a library in Heat, and then be moved to Oslo upon achievement of a reasonable level of maturity.
>> 
>>> As long as it doesn't change the use-case that heat desires of course (or
>>> overload that use-case and make everything complex when it doesn't need to
>>> be)…
>> 
>> It should actually simplify Heat.
>> 
>>> It would be very neat to use those 2 services as a library, where I can
>>> submit arbitrary code to (the state transitions that nova does) and have
>>> it handle calling those states, coordinating, rolling back (or at least
>>> calling a method to rollback, since rollback is usually very specific to
>>> what has occurred). Basically submitting jobs, but not via a DSL/CEP (or
>>> the like), not via a model interpreter, but directly via code. Is there
>>> any documentation on how heat handles task resumption (if 1 engine fails,
>>> another should be able to continue the work), how are said engines made HA
>>> and reliable…
>> 
>> At the very least, we should expect a solution that will allow you to submit a callback in some form (possibly a web hook, or a function) so that you can act upon state transitions in the workflow. There are ways to deal with rollback. Chances are that a rather simple early implementation would shift that to the client, but there are way to deal with it in the service too.
>> 
>>> Such a engine library would make sense as the core 'engine' for a lot of
>>> the openstack core projects imho.
>> 
>> Yes, if the vision is achieved, you would use Heat for Orchestration purposes, and Convection for simple task executions. The first use case for Convection would be a task that has a start, a sequence of tasks, followed by a termination. This could be very useful in Nova when you care about kicking off a workflow and be able to get a callback upon completion of the job/task.
>> 
>> For jobs that would benefit from parallel execution and coordination of multiple tasks, you would use Heat as an Orchestration instrument, which would in turn use multiple workflows.
>> 
>>> Thoughts?
>> 
>> The first implementation is likely to be library within the Heat project that the Heat engine uses. Next, an Oslo library, and then a standalone service with a REST API that uses the Oslo library for general purpose uses from applications outside Openstack, perhaps those deployed by Heat, etc. This gives us the opportunity to offer the service in an fault tolerant HA configuration for use cases that require reliable workflow execution. If you are interested in contributing, I urge you to connect with Keith Bray <keith.bray at racksopace.com> to coordinate efforts.
>> 
>> Cheers,
>> 
>> Adrian
>> 
>>> 
>>> On 4/17/13 2:47 PM, "Zane Bitter" <zbitter at redhat.com> wrote:
>>> 
>>>> Thanks Adrian!
>>>> 
>>>> We should probably make it clear for those who can't be at Summit that
>>>> this is not intended to be a record of the design summit discussions.
>>>> I'd describe this as a snapshot of Adrian's evolving vision for Heat,
>>>> taken after the design summit sessions. In particular, I think it
>>>> reflects a couple of misconceptions about the current architecture of
>>>> Heat, and it also contains some stuff that hasn't been discussed at all
>>>> at Summit (e.g. the CEP part... btw my initial reaction to this is that
>>>> it should probably talk to Ceilometer rather than the Autoscaling
>>>> service).
>>>> 
>>>> That said, there certainly *is* consensus that we want to evolve Heat
>>>> toward being able to orchestrate whole applications rather than just
>>>> services. We'll be doing further work throughout the week and
>>>> subsequently on the mailing list to refine some of these architectural
>>>> ideas, and we're looking forward to working with Adrian and his team at
>>>> Rackspace to make it happen :)
>>>> 
>>>> cheers,
>>>> Zane.
>>>> 
>>>> On 16/04/13 01:56, Adrian Otto wrote:
>>>>> Heaters,
>>>>> 
>>>>> I attended the various sessions at the Design Summit today in Portland,
>>>>> and assembled as many of the ideas for future planning as I could.  For
>>>>> the benefit of those who are not attending, or who were not in these
>>>>> sessions, I created this Wiki page to express what I think is an early
>>>>> consensus on where we could take things. Let's tweak this if it's not a
>>>>> good direction.
>>>>> 
>>>>> https://wiki.openstack.org/wiki/Heat/Vision
>>>>> 
>>>>> Keith will be doing an Unconference session on the Workflow Service
>>>>> ideaŠ I believe on Wednesday afternoon.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Adrian
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>> 
>>>> 
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>> 
>>> 
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> 
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev