[openstack-dev] [Heat] Future Vision for Heat
zbitter at redhat.com
Wed Apr 24 16:03:17 UTC 2013
On 18/04/13 07:37, Joshua Harlow wrote:
> I'd be very interested in having 'Scheduler & Workflow Service' part there
> be a library.
> Pretty much every application is a workflow in some way, and using said
> library in nova for the orc work there would be very neat.
> As long as it doesn't change the use-case that heat desires of course (or
> overload that use-case and make everything complex when it doesn't need to
That shouldn't be a big issue. Heat's use case and Nova's are similar,
except that we also need to run tasks in parallel. However, I'm in the
process of delivering that feature in Heat now and the 'library' part of
that code is very localised, so I don't think it would add much
complexity for anyone not using it.
> It would be very neat to use those 2 services as a library, where I can
> submit arbitrary code to (the state transitions that nova does) and have
> it handle calling those states, coordinating, rolling back (or at least
> calling a method to rollback, since rollback is usually very specific to
> what has occurred). Basically submitting jobs, but not via a DSL/CEP (or
> the like), not via a model interpreter, but directly via code.
Yes, I very much like the idea of just having a library that takes
Python code and runs it as a workflow, rather than having some sort of
DSL from which you have to try to generate/locate the code each time.
(That stuff would be the job of the proposed Convection service, which
ideally could also be built on top of this library.)
> Is there
> any documentation on how heat handles task resumption (if 1 engine fails,
> another should be able to continue the work), how are said engines made HA
> and reliable...
This is an issue we haven't yet tackled in Heat - there was a Design
Summit session about it, and it is on the agenda for Havana.
The easiest way forward, for both Heat and Nova, might be to mandate
that the tasks are written to be idempotent and rely on their existing
use of state stored in the database. Then we would need only one level
of locking (to determine whether the workflow is still being run, or has
died and we need to resume it or roll it back).
The more sophisticated approach is obviously to have it store the
current progress in the database and have some sort of event triggered
when a task completes that lets any other workflow engine process pick
it up and start the next step. I'm not convinced that's feasible for a
simple library though; a service with a DSL could do it, but I'd rather
not go down that path.
> Such a engine library would make sense as the core 'engine' for a lot of
> the openstack core projects imho.
More information about the OpenStack-dev