Open Stack

See in-line for comments on Clint's original list of options, and detail on each below.

On Apr 10, 2013, at 5:35 AM, "Doug Davis" <dug at us.ibm.com<mailto:dug at us.ibm.com>> wrote:

+1

I believe that the core of Heat should, for the most part, remain pretty agnostic
to the format of the deployment descriptions.

+1

There will be a bit of pain for users of Heat for a while until the dust settles and
the community gravitates towards the preferred format.  But I'd prefer to let the users
and the community decide the winner instead of just us doing it right now.

It will also allow Heat to have a longer life-span since it won't be tied to one
particular format and can more easily migrate in the future.  Formats will come and go.

Yes, this is part of the modularity vision I'm referring to. Heat should not be directly tied to a single format on the public side of the API. It should allow for whatever formats are needed by our Openstack user community.

thanks
-Doug
________________________________________________________
STSM |  Standards Architect  |  IBM Software Group
(919) 254-6905  |  IBM 444-6905  |  dug at us.ibm.com<mailto:dug at us.ibm.com>
The more I'm around some people, the more I like my dog.

Thomas Spatzier ---04/10/2013 04:24:59 AM---Hi Clint, you are raising a very valid point here. I agree that just adding support

From: Thomas Spatzier <thomas.spatzier at de.ibm.com<mailto:thomas.spatzier at de.ibm.com>>
To: OpenStack Development Mailing List <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>,
Date: 04/10/2013 04:24 AM
Subject: Re: [openstack-dev] [Heat] TOSCA, CAMP, CloudFormation, ???

Hi Clint,

you are raising a very valid point here. I agree that just adding support
for all of the format brought forward could be problematic, and I have a
few thoughts on that.

My experience is that there are many pattern engines (let me just use that
term here) out there - some done as open source, some in various products
of different vendors. I have seen very similar concepts in all of those
engines, but each has its own proprietary format, e.g. we have an internal
format in our product, partners in the OASIS TOSCA TC have their internal
format in their engines and so on. So what we did when adopting TOSCA was
to keep our internal format and add TOSCA as kind of front-end format
on-top. This works, if the internal format and TOSCA (or any other
standard) are well enough aligned.

Yes.

So the discussion should maybe not be the adoption of one or the other
format as the "native Heat metamodel", but the evolution of the internal
meta model to a state where it (1) fits the use cases Heat wants to
address, and (2) is aligned with the desired other formats as closely as
possible. Then we could have very thin pluggable layers that add support
for alternative "front-end formats".

Agreed.

Maybe we can have exactly this discussion next week. I have a session on
the TOSCA proposal at 11:50am on Monday, and I see your session is
scheduled for after lunch.
My plan was to give an overview of TOSCA in my session since maybe not all
are familiar with it, but not spend too much time on it, but use the
session for discussing the points above.

We have a lot to discuss in rather limited time, so we should keep this nice and crisp, getting across the conceptual difference between imperative and declarative models, and how TOSCA allows expression of each.

Does that make sense to you?

Regards,
Thomas
---------------------------------------------------------------------
Tivoli CTO Office - Cloud Orchestration, Cloud Standards


From: Clint Byrum <clint at fewbar.com<mailto:clint at fewbar.com>>
To: openstack-dev <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>,
Date: 10.04.2013 08:25
Subject: [openstack-dev] [Heat] TOSCA, CAMP, CloudFormation, ???

As we near the next OpenStack Summit, it has become clear that the way
forward for Heat users is.. well, not very clear.

Heat currently only supports one way to define applications. That is the
hybrid Heat-specific plus CloudFormation-compatible templating language.

We also have a proposal to add TOSCA support to Heat:

https://blueprints.launchpad.net/heat/+spec/tosca-support
http://lists.openstack.org/pipermail/openstack-dev/2013-April/007252.html

Adrian from RackSpace has stated that they are working on an "Open API &
DSL" and a "Declarative Model":

http://lists.openstack.org/pipermail/openstack-dev/2013-April/007126.html

I've also heard OASIS/CAMP thrown around as something that might be
implemented in Heat.

I count no less than 4 possible ways for users to express application
deployments. Succinctly:

* CloudFormation-ish

This is already supported to a reasonable extent, and probably makes sense to retain what's already done. Handling existing CloudFormation templates is a valid use case for Heat. I don't think we should lose sight of that.

* Open DSL w/ Declarative model(undefined yet)

See my comment below on CAMP, as these are actually not very different. I am writing a WIKI page that details what this looks like, and will link it (hopefully today/tomorrow) to a new BP that simply asks for an open and vendor independent API and DSL. I have a very clear vision for how this can be accomplished, and will detail it both at the summit and in written proposal form.

* TOSCA

We should be able to rather easily support TOSCA, at least the declarative modeling style, regardless if there is a default expression that aims to be more concise/simple.

* CAMP

CAMP focuses more on the interoperability aspects, and the API. It solves how you handle concurrent versions, extensibility, format discovery, type discovery, etc. The Open DSL with the declarative model that I'm referencing is synergistic here. Anish Karmarkar (Oracle) and I (Rackspace) will both be present at ODS and will attend the Heat sessions. We are co-editors on the CAMP specification, and can indicate what use of CAMP could be appropriate for the Heat API, while still enabling TOSCA.

Note that TOSCA and CAMP are also complimentary. This really is not an "either-or" choice, it's more of how we plan our development to iteratively advance toward the point where we can support arbitrary expressions/formats, and have a simple and effective modular approach to handling each.

Also, recognize I'm not suggesting that we try to support everything for everyone. We should start simple, and use an approach that we can easily build out as needed with minimal rework. I see no reason why we could not gradually support TOSCA, CAMP, and others all in Heat with a common system architecture that's actually not that complicated. It just seems that way because these standards feel like unknowns.

The right people will be present at the Summit to demystify this for the whole Heat dev team and interested OpenStack stakeholders. We should focus primarily on what the use cases are for Heat, and pick solutions that fit those use cases well.

We should create a WIKI page detailing the use cases so our community can begin to contribute to that list, and we can sort them by relative priority as a team.

http://xkcd.com/927/

Cute. To be clear, CAMP and TOSCA are not competing standards. They are complimentary with different areas of focus. What we do with Heat will certainly not create a competing standard. We will work to compliment and simplify what already exists.

I am concerned about the fragmentation that users may be presented
with if we do not take a step back and talk about the ramifications
of supporting all of them. Its great to be as broad and accepting as
possible when it makes sense, but not at the expense of user confusion
and/or incompatibility.

We should have an open template format that's simple and concise. A CAMP compliant REST API can ingest that. The simple format should be our default. TOSCA (or portions thereof) can be handled using a plug-in at the parser level, processed by the Heat engine. Remember that templates are a way to express an orchestration. The Heat engine handles what actually happens to carry out those instructions. Modularity can allow us to extend the engine as needed.

So, I am calling on those interested in the various formats to at least
raise your hands and let us, the heat users and developers, know what
you expect from Heat as a project, and what makes any of these standards
worth implementing. I think this is worth having a break-out session at
the summit next week to discuss, and would also encourage everyone to
start (or continue) the discussion right here on the mailing list.

Absolutely.

Clint Byrum
HP Cloud Services

Thanks,

Adrian Otto
--
Rackspace
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130410/735f2360/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: graycol.gif
Type: image/gif
Size: 105 bytes
Desc: graycol.gif
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130410/735f2360/attachment.gif>