[openstack-dev] [Heat] Future Vision for Heat

Thomas Spatzier thomas.spatzier at de.ibm.com
Tue Apr 23 18:46:05 UTC 2013


> From: Steven Hardy <shardy at redhat.com>
> To: OpenStack Development Mailing List
<openstack-dev at lists.openstack.org>,
> Date: 23.04.2013 20:21
> Subject: Re: [openstack-dev] [Heat] Future Vision for Heat
> On Tue, Apr 23, 2013 at 10:50:06AM +0200, Thomas Spatzier wrote:
> > Steven,
> >
> > > Excerpt from
> > > From: Steven Hardy <shardy at redhat.com>
> > > To: OpenStack Development Mailing List
> > <openstack-dev at lists.openstack.org>,
> > > Date: 23.04.2013 09:29
> > > Subject: Re: [openstack-dev] [Heat] Future Vision for Heat
> > >
> > > I agree, the model interpreter probably won't end up in the API (at
> > > in the short/medium term, because it will be necessarily coupled with
> > > model processor, aka "parser" in our previous discussions)
> >
> > I agree that the Model Interpreter shouldn't be part of the API layer.
> > Actually, before I read your mail, I already started to draw an
> > architecture diagram as base for discussion. I updated the wiki page,
so if
> > you refresh, you should see my version with some notes on the changes I
> > made.
> So your updated diagram is definitely getting closer to what I envisage,
> couple of comments:
> - I don't see us having to read monitoring data from the AMQP bus, my
>   expectation is that we define an alarm and associated metric(s) in
>   ceilometer, then simply wait for a callback (web-hook, request to our
>   API which triggers us of an alarm state change)

Ok, so I guess the way I have drawn it is due to my lack of detail
knowledge about how Heat work. Fine to change the figure.
So would you see the arrow going from Monitoring to the Heat REST API box?

> - The workflow service layer is not on our immediate roadmap (although I
>   support the idea), as I'm not sure who will implement it - I'll raise a
>   BP to track the idea, but I'm not sure this is realistic as a near-term
>   goal unless someone indicates they're interested in working on it

Yes, I got that point during our discussions last week. So maybe we gray
this box out as well to indicate this is out of scope for now.

> > > The way I see it working (short/medium term) is:
> > >
> > > - Stack template (either CFN or HOT) is passed to either the ReST or
> > >   API
> > > - Template is passed to the engine via RPC call
> > > - Template is processed by "model interpreter", which mangles e.g CFN
> > >   the internal format (HOT), or does nothing if it's already HOT
> > > - Processed template is passed to the "Model Processor", aka
> > >   which renders the template in our internal objects, ready to
> > orchestrate
> >
> > I am asking myself if there is a model interpreter at all in the Heat
> > Engine, or if the Engine just parses (processes) the model into its
> > internal objects. Since CFN is tightly built in at the moment and I
> > you said it would be hard to decouple it short term, I was assuming
> > the Model Processor for now would have to understand both Heat Template
> > CFN. I tried to mark this with the little CFN box in the diagram.
> Ok, so most of this is true, we have a thing called "parser" which
> transforms the data-structure representing the template into "Stack"
> object, which contains the various "Resource" objects and related data
> My aim is to avoid the engine parser/Model-Processor having to understand
> two formats (since I think this will be complex, error-prone, and also
> difficult to maintain)

That is what I had in mind as well, i.e. the core engine just having to
understand one language at the end and everythin else done in a translation

> So I see us doing the following
> - Add concepts identified as missing to out internal data model
> - Have a thin "interpreter" which performs a one-time
>   from the incoming template format to that supported by the parser.
>   Initially this will transform from DSL/HOT->HeatCFN+ExtraStuff, then
>   this works, we can flip the translation incrementally so the parser
>   eventually becomes tightly coupled to DSL/HOT and the translation
>   CFN->HOT (and at this point the translation could be done somewhere
>   than the heat engine)

Nice idea to have both components in place from the beginning and then
switch what they are doing :-)
Would you see this rendering/translation component as another thin green
box in the engine, or something on a higher layer?
I guess what this would turn into then later is a translator that
understands one format initially (CFN and a no-op for DSL/HOT), and could
be extended for pluggability for additional format as we go (more notes on
that below).

> > > Long term (when the internal HOT format is stable), it might make
> > to
> > > push the "model interpreter" up to the API level, probably via a
> > > template-interpreter-plugin model, but I see this as something we
> > shouldn't
> > > care about during the first-pass where the priority must be
> > > capturing/defining the new template language, and keeping our
> > > functionality working.
> >
> > That seems to make sense. Or I could also image to have the interpreter
> > layer (actually I see it more as a translation layer) as a layer below
> > API layer, basically by pulling the grayed out box (out of scope and
> > for now) in my diagram into the core.
> > That layer would do translation only and leave interpretation
completely to
> > the Heat Engine's Model Processor.
> This is sort of the translation plugin model I mentioned earlier, but as
> previously discussed, I think it's best we consider this out-of-scope for
> Heat until it becomes obvious there are people (ie new Heat core members
> regular contributors) willing to implement and maintain the translator
> components.

Got that point. So the additional formats are out of scope for now (grayed
out) and everything else depends on active contributors.
Going back to the translation layer mentioned above (the one that does
DSL->CFN initially and is then switched to CFN->DSL), when we got the
contributors we need (we are intending to get engaged, btw), we could think
about moving the gray box into the translation layer and make it pluggable.

I am attaching the pptx I used for my chart to this mail (it's small so
hope it is ok). Maybe you can put it somewhere and also make updates to
capture your thoughts.

> Steve
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
(See attached file: Heat Vision.pptx)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Heat Vision.pptx
Type: application/octet-stream
Size: 46372 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130423/14b6f545/attachment.obj>

More information about the OpenStack-dev mailing list