Open Stack

There's been quite a bit of discussion recently about Heat's cfn-tools, 
which currently comprise a minimalist reimplementation by the Heat 
developers of Amazon's aws-cfn-bootstrap tools.

It was also brought up on IRC last night that there has been a formal 
request from Amazon (not yet acted upon) for Ubuntu to include 
aws-cfn-bootstrap in the distribution.[1]

At the same time, Fedora's Cloud SIG is discussing ways to make Fedora 
Cloud images "more awesome".[2] Although no specifics have been 
mentioned, packaging the aws-cfn-bootstrap tools would seem like an 
obvious possible improvement. The Cloud SIG also has plans for Fedora 19 
to introduce builds that target clouds other than EC2, including 
OpenStack.[3]

Steve Hardy has done a lot of work on Heat to make the data that gets 
passed to the instance as similar as possible to what gets passed by 
CloudFormation (e.g. a reference to a WaitConditionHandle now resolves 
to a pre-signed URL that can be used for signalling). Still, Heat is not 
able to use the aws-cfn-bootstrap tools directly because they are 
hard-coded with the CloudFormation endpoints. (There may be other issues 
too, and more investigation is required.)

However: The aws-cfn-bootstrap tools are open source, under the Apache 2 
license. I see little point in continuing to reimplement features 
piecemeal as we become aware of the need for them.[4] I propose that we 
instead take the following steps:

* Fork the aws-cfn-bootstrap tools with the goal of creating a 
cloud-agnostic upstream community (unaffiliated with OpenStack) around them.
* Modify the tools to read endpoints from a configuration file (perhaps 
defaulting to the CloudFormation endpoints).
* Make any further modifications to Heat (and/or cfn-bootstrap) required 
to implement full interoperability.
* Package this configurable version of aws-cfn-bootstrap for 
Debian/Ubuntu/Fedora.
* Work with Ubuntu and the Fedora Cloud SIG to get this configurable 
version included in cloud server builds.

This would not only save us a lot of work, and likely prevent a lot of 
bugs, but it would also offer a huge advantage to users: a single 
prebuilt cloud image available from the distro of their choice could run 
on both AWS and OpenStack without modification.

There is one known issue that anybody wanting to package 
aws-cfn-bootstrap in a distro would have to circumvent: the current 
tools are installed (appropriately) in /opt/aws/bin. Amazon, through 
their example templates, have encouraged people to use the 
fully-qualified path (e.g. /opt/aws/bin/cfn-init) when calling them, 
instead of just adding /opt/aws/bin to $PATH. The latter would have made 
the transition to including the tools in the distro (where packaging in 
/opt is *not* appropriate) transparent, but the former most certainly 
does not. I have several ideas for working around this, none of them good ;)

I'd like to apologise at this point for being the guy who shows up with 
a lot of talk and no code ;). I do think an initial proof-of-concept for 
Heat integration should be fairly easy to put together, and I intend to 
start working on it after the g-3 release is out of the way, provided 
that the suggestion proves sufficiently uncontroversial.

cheers,
Zane.


[1]: 
https://blueprints.launchpad.net/ubuntu/+spec/servercloud-p-cloud-formations-images
[2]: http://lists.fedoraproject.org/pipermail/cloud/2013-January/002219.html
[3]: https://fedoraproject.org/wiki/Features/FirstClassCloudImages
[4]: https://blueprints.launchpad.net/heat/+spec/aws-cloudformation-init