[openstack-dev] [Heat] re: discussion about passing metadata into provider stacks as parameters

Angus Salkeld asalkeld at redhat.com
Thu Jun 6 13:28:09 UTC 2013


On 06/06/13 12:40 +0200, Zane Bitter wrote:
>On 06/06/13 05:55, Angus Salkeld wrote:
>>On 06/06/13 12:24 +1000, Angus Salkeld wrote:
>>>Hi
>>>
>>>So I just want to make sure we are solving this problem in the best
>>>way. I don't claim to know the ulitimate solution...
>>>
>>>Problem (as I understand it)
>>>
>>>Provider templates are templates that are used to implement a resource
>>>type.
>>>A current example is our DBInstance or LoadBalancer (although
>>>the templates are inline with the python).
>>>
>>>Because the template is created as a stack, it obviously has
>>>a Stack() API, but it is implementing a Resource. So this is
>>>a problem of mismatching API's.
>>>
>>>Stack:
>>>- parameters
>>>- template
>>>- some other args (timeouts etc)
>>>
>>>Resource:
>>>- Metadata
>>>- Properties
>>>- DeletionPolicy
>>>- UpdatePolicy
>>>- DependsOn
>>>
>>>Currently we pass the Properties into the parameters (which matches
>>>nicely) DependsOn should not be needed as that is for the creation
>>>ordering. If needed we can pass in the Deletion/Update Policy as
>>>a special parameter (I have no problem with that).
>
>Exactly. We've said that Properties will be passed as Parameters and 
>Fn::GetAtt will read from Outputs, so we just need a way to match up 
>Metadata, DeletionPolicy and UpdatePolicy.
>
>>>
>>>It has been suggested to pass the metadata in as a parameter too.
>>>This doesn't seem like a great fit to me, if there is no other
>>>solution then fine, but first some other options.
>
>Agreed, it's definitely not a perfect fit.
>>>
>>>1) metadata is not used by heat (or shouldn't be), it is stored and is
>>>  made available to the stack's instances. So surely we don't really
>>>need to
>>>  pass in the data just the url to it?
>>>(http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-metadata.html)
>
>I don't see how the author of the provider template can know the URL 
>of the parent resource's parsed metadata in advance, since it will be 
>different for every instantiation of the provider template.

With this option the url would be passed in as a parameter.
(assuming we can generate the url before creating the nested stack)

>>>
>>>
>>>2) if (for some reason) the template writer needs actual access to the
>>>  data within the metadata we could have a template function to get at a
>>>  snippet of a resource. So if we pass in the parent arn (as a
>>>  parameter) we could do something like:
>>>
>>>{'Fn:select': ['Metadata',
>>>              {'Fn::template': {'Ref': 'parent_resource'}}}
>>>
>>
>>Instead of "Fn::template" this should be something like:
>>Fn::DescribeStackResource
>>to indicate that we are retrieving a parsed template not a raw one
>>as it is important in which stack (must be in the parent) the metadata
>>is parsed
>>else we will not be able to resolve the references from the child
>>stack.
>>
>I would support something like this.
>
>On a purely technical level, I think using "Ref" here will be a pain 
>to implement (it already has two meanings; better not to add a 
>third). But I could definitely get behind something like:
>
>  {'Fn::ProvidedResource': 'Metadata'}
>
>except with a better name.

Yea, that seems fine to me.

>>>
>>>3) Metadata is _mostly_ used by Servers/Instances
>>>  Allow one of the above solutions in the general case but rather
>>>encourage the use
>>>  of external (to the resource) files.
>
>The most common case I am envisioning is where the provider template 
>is a generic "configuration management" system provider (e.g. 
>'ChefSoloServer') and its UserData contains a script to bootstrap the 
>"configuration management" system. The Metadata would contain the 
>actual configuration for a particular instance, and would appear in 
>the main template.
>
>We can and should make it easy for that Metadata to be included in 
>the top level template by reference to a file rather than inline. But 
>that doesn't change the fact that the author of the generic provider 
>template needs a way to refer to that metadata to make sure it ends 
>up on the instance.

Just as an idea (not suggesting as apart of this work), what if:
The lowest level Server didn't expose userdata only a property
'ConfigurationUrls': {'Type': 'List'}

These would be like the Cloudformation::Init section in content,
but also having an entrypoint (like a userdata, but for each
configuration). So the very base Server implementation would
have a userdata that retrieved these and run them in order.
It would make "inheriting" the server easier as you just add
your configuration stanza to the end of the list.

>>>
>>>  - I think in the case of Instaces this encourages composition as
>>>    most of the real configuration is within the current Metadata (or
>>>    should be). If we had a library/catalog of metadata files then
>>>    you could just include/reference them from within your template.
>>>    Largely reducing the need for metadata in the first place.
>>>
>>> I'd almost suggest our new native Server resource should not support
>>> the anything like "AWS::CloudFormation::Init" within the metadata
>>> but rather a way of getting this from a catalog.
>
>That seems not so much like a question of what the resource supports 
>as of what the in-instance agent (i.e. cfn-tools, or something else) 
>does with the metadata. Steve H has been advocating persuasively for 
>an OpenStack-native in-guest agent that talks to the ReST API.

Well I think exposing userdata encourages poor template writing.
Even having the metadata as a part of the Server resource discourages
sharing templates and makes it more awkward to reuse your
configurations.

(I know this is probably part of another discussion, but it's fun
thinking about)

-Angus

>cheers,
>Zane.
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



More information about the OpenStack-dev mailing list