Open Stack

> From: Angus Salkeld <asalkeld at redhat.com>
> To: openstack-dev at lists.openstack.org,
> Date: 06.06.2013 15:37
> Subject: Re: [openstack-dev] [Heat] re: discussion about passing
> metadata into provider stacks as parameters
>
> On 06/06/13 12:40 +0200, Zane Bitter wrote:
> >On 06/06/13 05:55, Angus Salkeld wrote:
> >>On 06/06/13 12:24 +1000, Angus Salkeld wrote:
> >>>Hi
> >>>
> >>>So I just want to make sure we are solving this problem in the best
> >>>way. I don't claim to know the ulitimate solution...
> >>>
> >>>Problem (as I understand it)
> >>>
> >>>Provider templates are templates that are used to implement a resource
> >>>type.
> >>>A current example is our DBInstance or LoadBalancer (although
> >>>the templates are inline with the python).
> >>>
> >>>Because the template is created as a stack, it obviously has
> >>>a Stack() API, but it is implementing a Resource. So this is
> >>>a problem of mismatching API's.
> >>>
> >>>Stack:
> >>>- parameters
> >>>- template
> >>>- some other args (timeouts etc)
> >>>
> >>>Resource:
> >>>- Metadata
> >>>- Properties
> >>>- DeletionPolicy
> >>>- UpdatePolicy
> >>>- DependsOn
> >>>
> >>>Currently we pass the Properties into the parameters (which matches
> >>>nicely) DependsOn should not be needed as that is for the creation
> >>>ordering. If needed we can pass in the Deletion/Update Policy as
> >>>a special parameter (I have no problem with that).
> >
> >Exactly. We've said that Properties will be passed as Parameters and
> >Fn::GetAtt will read from Outputs, so we just need a way to match up
> >Metadata, DeletionPolicy and UpdatePolicy.
> >
> >>>
> >>>It has been suggested to pass the metadata in as a parameter too.
> >>>This doesn't seem like a great fit to me, if there is no other
> >>>solution then fine, but first some other options.
> >
> >Agreed, it's definitely not a perfect fit.
> >>>
> >>>1) metadata is not used by heat (or shouldn't be), it is stored and is
> >>>  made available to the stack's instances. So surely we don't really
> >>>need to
> >>>  pass in the data just the url to it?
> >>>(http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/
> aws-attribute-metadata.html)
> >
> >I don't see how the author of the provider template can know the URL
> >of the parent resource's parsed metadata in advance, since it will be
> >different for every instantiation of the provider template.
>
> With this option the url would be passed in as a parameter.
> (assuming we can generate the url before creating the nested stack)
>
> >>>
> >>>
> >>>2) if (for some reason) the template writer needs actual access to the
> >>>  data within the metadata we could have a template function to get at
a
> >>>  snippet of a resource. So if we pass in the parent arn (as a
> >>>  parameter) we could do something like:
> >>>
> >>>{'Fn:select': ['Metadata',
> >>>              {'Fn::template': {'Ref': 'parent_resource'}}}
> >>>
> >>
> >>Instead of "Fn::template" this should be something like:
> >>Fn::DescribeStackResource
> >>to indicate that we are retrieving a parsed template not a raw one
> >>as it is important in which stack (must be in the parent) the metadata
> >>is parsed
> >>else we will not be able to resolve the references from the child
> >>stack.
> >>
> >I would support something like this.
> >
> >On a purely technical level, I think using "Ref" here will be a pain
> >to implement (it already has two meanings; better not to add a
> >third). But I could definitely get behind something like:
> >
> >  {'Fn::ProvidedResource': 'Metadata'}
> >
> >except with a better name.
>
> Yea, that seems fine to me.
>
> >>>
> >>>3) Metadata is _mostly_ used by Servers/Instances
> >>>  Allow one of the above solutions in the general case but rather
> >>>encourage the use
> >>>  of external (to the resource) files.
> >
> >The most common case I am envisioning is where the provider template
> >is a generic "configuration management" system provider (e.g.
> >'ChefSoloServer') and its UserData contains a script to bootstrap the
> >"configuration management" system. The Metadata would contain the
> >actual configuration for a particular instance, and would appear in
> >the main template.
> >
> >We can and should make it easy for that Metadata to be included in
> >the top level template by reference to a file rather than inline. But
> >that doesn't change the fact that the author of the generic provider
> >template needs a way to refer to that metadata to make sure it ends
> >up on the instance.
>
> Just as an idea (not suggesting as apart of this work), what if:
> The lowest level Server didn't expose userdata only a property
> 'ConfigurationUrls': {'Type': 'List'}
>
> These would be like the Cloudformation::Init section in content,
> but also having an entrypoint (like a userdata, but for each
> configuration). So the very base Server implementation would
> have a userdata that retrieved these and run them in order.
> It would make "inheriting" the server easier as you just add
> your configuration stanza to the end of the list.
>
> >>>
> >>>  - I think in the case of Instaces this encourages composition as
> >>>    most of the real configuration is within the current Metadata (or
> >>>    should be). If we had a library/catalog of metadata files then
> >>>    you could just include/reference them from within your template.
> >>>    Largely reducing the need for metadata in the first place.
> >>>
> >>> I'd almost suggest our new native Server resource should not support
> >>> the anything like "AWS::CloudFormation::Init" within the metadata
> >>> but rather a way of getting this from a catalog.
> >
> >That seems not so much like a question of what the resource supports
> >as of what the in-instance agent (i.e. cfn-tools, or something else)
> >does with the metadata. Steve H has been advocating persuasively for
> >an OpenStack-native in-guest agent that talks to the ReST API.
>
> Well I think exposing userdata encourages poor template writing.
> Even having the metadata as a part of the Server resource discourages
> sharing templates and makes it more awkward to reuse your
> configurations.
>
> (I know this is probably part of another discussion, but it's fun
> thinking about)

Agree that this may be a different discussion than the one this thread was
started for, but still a good one.
So on that topic here: I think having too much in a server resource feels
somehow odd to me. A server is a server and should have minimal information
about software (if at all), but the way it is used in CFN is that in gets
provided with metadata for complete software stacks.

In my mental model, I would wish we had something like (don't care about
syntax)

componentA:
  type: to::be::defined
  # ...
  requires:
    hosted_on: my_server

my_server:
  type: os::nova::server
  # ...

... so keeping the native server implementation lean and then maybe having
a native heat agent on it that gets passed the information about everything
hosted on the server would be a nice approach. Also would allow for better
orchestration of different pieces hosted on a server long term.

>
> -Angus
>
> >cheers,
> >Zane.
> >
> >_______________________________________________
> >OpenStack-dev mailing list
> >OpenStack-dev at lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>