Open Stack

Hi Zane, 

Thanks for putting this together, overall looks great, a definite D++ from
me :)

Some comments inline:

On Thu, Apr 25, 2013 at 09:35:51PM +0200, Zane Bitter wrote:
> Greetings Heaters,
> I'm hearing a lot about (and struggling to keep track of) multiple
> people working on competing proposals for how a Heat template
> language should look if we designed it from scratch and, while
> that's valuable in helping to figure out the primitives we need, I'd
> also like to approach it from the other direction and start figuring
> out what path we need to get on to bring the new feature direction
> that our users want to fruition. We all agreed at the Summit that we
> need to pursue these new features in an incremental manner, and we
> should not forget *why*:
> 
>    A complex system that works is invariably found to have evolved
>    from a simple system that worked.
>                      -- http://en.wikipedia.org/wiki/Gall's_law
> 
> What follows is a concrete proposal for how we could implement one
> of the requested features, starting from use cases, through the
> predicted code changes required and looping back around to user
> benefits. It certainly does not purport to implement every feature
> requested by users. It is likely wrong in the particulars, certainly
> in ways that will not be discovered at least until we have
> implemented it. But it is a concrete proposal. It begins with an
> excellent summary of the use case from Adrian...
> 
> 
> On 13/04/13 00:36, Adrian Otto wrote:
> > Goal: Deploy my app on my chosen OpenStack based public cloud,
> using Heat for Dev/Test. For Production, deploy to my private
> OpenStack cloud.
> >
> > Scenario: My app depends on a MySQL database. My public cloud
> provider has a hosted "mysql" service that is offered through a
> Provider plug-in. It's there automatically because my cloud hosting
> company put it there.  I deploy, and finish my testing on the public
> cloud. I want to go to production now.
> >
> > Solution: The Provider gives you a way to abstract the different
> cloud implementations. I establish an equivalent Provider on my
> private OpenStack cloud using RedDwarf. I set up a Provider that
> offers "mysql" in my private cloud. Now the same setup works on both
> clouds, even though the API for my local "mysql" service may
> actually differ from the database provisioning API in the public
> cloud. Now I deploy on my "production" Environment in my private
> cloud, and it works!
> 
> So, the first and most important thing to note is that this is
> exactly how Heat works _now_.
> 
> There is a resource type called AWS::RDS::DBInstance and the
> operator of your cloud can choose a plugin to implement that
> interface. The plugin shipping with Heat actually just spins up a
> somewhat hacky Heat stack running MySQL... in future we hope to ship
> a RedDwarf Lite plugin, and of course any cloud provider with their
> own DBaaS could easily write a plugin to interface with that. (Note
> that when we do see a RedDwarf Lite plugin we'll probably also see a
> variant called something like OS::RedDwarf::DBInstance with more
> OpenStack-specific properties &c.)
> 
> So that's how Heat works today. How can we make this better? Well,
> one thing obviously sucks: your cloud operator, and not you, gets to
> decide which plugin is used. That sorta makes sense when it's an
> interface to an XaaS thing in their cloud, but if it's just a Nova
> instance running MySQL and you don't like the version your operator
> has gone with, you are SOL. You can try running your own Heat
> engine, but you're probably going to have to really hack at it first
> because whenever anything in-guest has to talk back to you, the
> endpoint is obtained from the same Keystone catalog that you're
> using to talk to the other services. And no cloud operator in the
> world - not even your friendly local IT department - is going to let
> users upload Python code to run in-memory in their orchestration
> engine along with all of the other users' code.
> 
> If only there were some sort of language for defining OpenStack
> services that could be safely executed by users...
> 
> Of course that's exactly what we're all about on this project :). So
> my proposal is to allow users to define their own resource types
> using a Heat template. Heat would know to use this template instead
> of a built-in type from a "Provider" member in the Resource
> definition that contains a URL to the template. (I'm appropriating
> the name "Provider" from Rackspace's DSL proposal for now because
> inventing new names for things that already exist is a sucker's
> game.)
> 
> These are the tasks that come to mind that would be required to
> implement this (each of these bullet points could become a
> blueprint):
> 
> * Create a Custom resource type that is based on a nested stack but,
> unlike the AWS::CloudFormation::Stack type, has properties and
> attributes inferred from the parameters and outputs (respectively)
> of the template provided.
> * Allow JSON values for parameters.
> * Introduce the concept of _stack_ Metadata, and provide a way to
> access it in a template (pseudo-parameter?).
> * Modify resource instantiation to create a Custom resource whenever
> a resource has a non-empty "Provider" attribute.
> * Introduce a schema for attributes (i.e. allowed arguments to
> Fn::GetAttr) [desirable anyway for autogenerating documentation]
> * Add an API to get a generic template version of any built-in
> resource (with all properties/outputs defined) that can be easily
> customised to make a new provider template.

These all sound like sensible, incremental and easily definable new features,
great job clarifying the problem and breaking it down :)

Seems we have enough support for these ideas that we can probably just go
ahead and raise the BPs so we can get people assigned and looking into the
details.

> A possible avenue for increasing flexibility:
> * Add support for more template functions that manipulate the
> template directly:
>   - Array/dictionary (uh, object) lookup
>   - more string functions?
>   - maybe conditionals?
>   - definitely NOT loops/map.
> 
> What might all of this give us?
>  + Users are no longer dependent on the operator to provide the
> resource type they need (perhaps for cross-cloud compatibility), but
> can supply their own.
>  + Users can effectively subclass built-in types. For example, you
> could create a Puppet Instance template that configures an instance
> as a Puppet slave, then selectively use that in place of a regular
> instance and just pass the metadata to specialise it.

I'm also not sure about the subclass idea, sounds like it could get
complex, but it does sound like a potentially powerful concept.

>  + Users can share their provider templates: as soon as a second
> person is trying to configure a puppet slave in a template we're
> wasting an opportunity - this will enable work like that to be
> shared.
>  + This is infinitely flexible at the platform layer - anybody
> (Puppet, Chef, OpenShift, &c.) can publish an Instance provider
> template
> 
> How else might we improve this? Well, having to load a template from
> a URL is definitely a limitation - albeit not a new one (the
> AWS::CloudFormation::Stack resource type already has this
> limitation). Perhaps we should let the user post multiple, named,
> implementations of a resource type and reference them by name
> (instead of URL) in the "Provider" field.
>
> * Modify the resource instantiation to search among multiple named
> definitions (either Custom or Plugin) for a resource type, according
> to the "Provider" name.
> * Add an API for posting multiple named implementations of a resource type.
> 
>  + The user can modify the default Provider for a resource if they
> so desire (but this is tenant-wide... or perhaps that's where
> Environments come in).
>  + Provider templates can be uploaded directly to Heat instead of
> e.g. to Swift.

I like Angus's suggestion of heat publish-template, where users can upload
templates either scoped to just the user, or if a tenant admin, optionally
upload tenant-scoped public templates for their project/organisation.

So adding a couple of ideas to what Angus already suggested:

- Enable user to define scope when publishing a template
- Enable user to define if the template is the default (ie always override
  the builtin or any other existing uploaded templates), this avoids the
  awkward "--use" thing unless you want to use the non-default template
- Allow the uploaded templates to be named, not just referred to by the
  resource type, e.g:

heat template-publish --private --type AWS::RDS::DBInstance --template-url
http://foo/mydb.template mydb

heat template-list --type AWS::RDS::DBInstance

id  name     type                  default  scope
1   builtin  AWS::RDS::DBInstance  true     public
2   mydb     AWS::RDS::DBInstance  false    private


heat template-publish --public --default --type AWS::RDS::DBInstance --template-file
~/mydefaultdb.template defdb


heat template-list --type AWS::RDS::DBInstance

id  name     type                  default  scope
1   builtin  AWS::RDS::DBInstance  false    public
2   mydb     AWS::RDS::DBInstance  false    private
2   defdb    AWS::RDS::DBInstance  true     private

This effectively provides a vairation on the environment idea, but one
which can be controlled either at provide (builtin), tenant (admin users),
or user scope

You could still allow the --use option to override the default "profile"
defined above.

>  + Operators can reuse the mechanism to provide versioned Plugins
> and/or multiple implementations of Resource types.

So maybe we could extend the idea above to allow a cloud-provider config
file which pre-populates public resource implementations which are visible
to all tenants (in addition to the resource plugins facility, which would
define the "builtin"

I'm not sure we should care about versioning internally, that could just be
handled via naming using the above strategy, e.g mydb_v1, mydb_v2 etc

> So, in summary, this plan appears to provide real, identified value
> to users; relies for the most part on existing technology in Heat
> (nested stacks are actually pretty cool, if I may say so); includes
> a series of changes required to implement the feature; has zero
> impact on existing users; does not IMO decrease maintainability of
> the Heat code base; and is entirely achievable before the Havana
> feature freeze, which is only ~4 months from now.

+100 - I think I've said the word "incremental" too many times now, but
this does provide us with a clear, incremental and low-risk plan to
delivering a really nice new feature, great job! :)

Steve