[openstack-dev] [Heat] HOT Software orchestration proposal for workflows

Lakshminaraya Renganarayana lrengan at us.ibm.com
Tue Oct 15 02:06:33 UTC 2013


Hi Angus,

Thanks for detailed reply. I have a few comments that I have written below
in the context.


Angus Salkeld <asalkeld at redhat.com> wrote on 10/13/2013 06:40:01 PM:

> >
> >- INPUTS: all the attributes that are consumed/used/read by that
resource
> >(currently, we have Ref, GetAttrs that can give this implicitly)
> >
> >- OUTPUTS: all the attributes that are produced/written by that resource
(I
> >do not know if this write-set is currently well-defined for a resource.
I
> >think some of them are implicitly defined by Heat on particular resource
> >types.)
> >
> >- Global name-space and data-space : all the values produced and
consumed
> >(INPUTS/OUTPUTS) are described using a names that are fully qualified
> >(XXX.stack_name.resource_name.property_name). The data values associated
> >with these names are stored in a global data-space.  Reads are blocking,
> >i.e., reading a value will block the execution resource/thread until the
> >value is available. Writes are non-blocking, i.e., any thread can write
a
> >value and the write will succeed immediately.
>
> I don't believe this would give us any new behaviour.

I believe that in today's Heat, wait-conditions and signals are the only
mechanism for synchronization during software configuration. The proposed
mechanism would provide a higher level synchronization based on
blocking-reads.
For example, if one is using Chef for software configuration, then the
recipes can use the proposed mechanism to wait for the all the node[][]
attributes
they require before starting the recipe execution. And, Heat can actually
analyze and reason about deadlock properties of such a synchronization. On
the other hand, if the recipe were using wait-conditions how would Heat
reason about deadlock properties of it?

>
> >
> >The ability to define resources at arbitrary levels of granularity
together
> >with the explicit specification of INPUTS/OUTPUTS allows us to reap the
> >benefits G1 and G2 outlined above. Note that the ability to reason about
> >the inputs/outputs of each resource and the induced dependencies will
also
> >allow Heat to detect dead-locks via dependence cycles (benefit G3). This
is
> >already done today in Heat for Refs, GetAttr on base-resources, but the
> >proposal is to extend the same to arbitrary attributes for any resource.
>
> How are TemplateResources and NestedStacks any different? To my
> knowledge this is aleady the case.
>
> >The blocking-read and non-blocking writes further structures the
> >specification to avoid deadlocks and race conditions (benefit G3).
>
> Have you experienced deadlocks with heat? I have never seen this...

Heat as it is today does not tackle the problem of synchronization during
software configuration and hence the problems I see cannot be attributed to
Heat and can only be attributed to the scripts / recipes that do the
software configuration. However, if we envision Heat to provide some
support for software configuration I can easily imagine cases where
it is impossible for Heat to analyze/reason with wait-conditions and hence
leading to deadlocks. Wait-conditions and signals are equal to
Timed-Semaphores in their power and expressivity and these are
known for their problems with deadlocks.

>
> To me what is missing to better support complex software configuration
> is :
> - better integrating with existing configuration tools (puppet, chef,
>    salt, ansible, etc). (resource types)

One question is, whether in this integration the synchronization is
completely
left to the configuration tools or Heat would be involved in it. If it is
left to configuration tools, say chef, then the question is how does the
iterative convergence style execution of chef interfere with the schedule
order that Heat determines for a template. On the other hand, if Heat
provides
the mechanism for synchronization, then the question is whether
wait-conditions
and signals are the right abstractions for them. What are your thoughts on
this?

Thanks,
LN
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20131014/c31e617b/attachment.html>


More information about the OpenStack-dev mailing list