Open Stack

On 14/04/14 07:01, Steven Hardy wrote:
> Hi Rabi,
>
> On Mon, Apr 14, 2014 at 06:44:44AM -0400, Rabi Mishra wrote:
>> Hi All,
>>
>> Recently, I've come across some requirements for external integrations/resources that can be managed like stack resources (create,update,delete) from the stack.
>>
>> 1. Adding/Removing DNS records for instances created as part of a stack.
>> 2. Integration with IPAM solutions for allocate/release of IPs (IP allocation pool for provider network)
>> 3. Other custom integration for dynamic parameters to stacks.
>>
>> IMHO, it would probably make sense to create a custom resource like 'AWS CFN Custom Resource'[1] that can be used for these kind of use cases. I have created a blueprint[2] for this.
>
> Heat already has a couple of ways for custom resources to be defined.
>
> The one which probably matches your requirements best is the "provider
> resource" interface, which allows template defined resources to be mapped
> to user-definable resource types, via an environment file:
>
> http://hardysteven.blogspot.co.uk/2013/10/heat-providersenvironments-101-ive.html
> http://docs.openstack.org/developer/heat/template_guide/environment.html
>
> Provider resources can be defined by both users, and deployers (who can use
> templates to e.g wrap an existing resource with something like DNS
> registration logic, and expose the type transparently to the end-user)

That's great for composing existing Things, but when you want to either 
deal with a Thing that isn't yet represented by a resource (lack of 
DNSaaS in OpenStack is really hurting us IMO) or to insert Actions into 
the workflow, that doesn't really help.

> For deployer requirements not satisfied by provider resources (for example
> integration with third-party services), Heat also provides a python plugin
> API, which enables deployers to create their own resource plugins as
> needed:
>
> http://docs.openstack.org/developer/heat/pluginguide.html
>
> Personally, I think these two models provide sufficient flexibility that we
> should be able to avoid the burden of maintaining a CFN compatible custom
> resource plugin API.  I've not looked at it in detail, but the CFN model
> you refer to has always seemed pretty complex to me, and seems like
> something we don't necessarily want to replicate.

It really doesn't seem complicated at all to me. On every action 
(create/delete/update) you post a message to a Marconi queue with some 
data and a URL, then wait for a response to that URL (i.e. same as a 
WaitCondition, effectively). It sounds like about 50 lines of code, and 
would be completely self-contained except for the part that allows 
resource types starting with "Custom::".

It's actually my stated opinion that rather than just reimplementing the 
AWS CustomResource, we should add these kinds of hooks to _every_ 
resource type (i.e. the functionality would be built in to Heat). That 
would be useful for a wide variety of other purposes, such as 
"breakpoints" for debugging a template, hooks for Murano to customise 
the workflow, confirming resizes/rebuilds of servers, and a number of 
others.

The main obstacle to that is that Marconi will not be part of the 
integrated release for Juno, and is targeting graduation in time to be 
fully integrated in the K release. However, implementing the 
CustomResource would require some of the same code (so it could give us 
a head-start), would be helpful anyway for compatibility with existing 
CloudFormation templates, and seems entirely reasonable given that 
Marconi is an incubated project.

> If there are gaps where things are not yet possible via the provider
> resource interface, I'd rather discuss incremental improvements to that
> instead of wholesale reimplementation of something compatible with AWS.

We should do that as well obviously, but we can't hope to anticipate 
every possible need and provide a resource for it, since resources are 
pretty much reserved for things that the OpenStack API provides. 
Operators can, of course, create plugins for their own non-OpenStack 
services, but custom resources open things up to third-party services.

cheers,
Zane.