Open Stack

On 08/08/14 11:07, Tomas Sedovic wrote:
> On 08/08/14 00:53, Zane Bitter wrote:
>> On 07/08/14 13:22, Tomas Sedovic wrote:
>>> Hi all,
>>>
>>> I have a ResourceGroup which wraps a custom resource defined in another
>>> template:
>>>
>>>       servers:
>>>         type: OS::Heat::ResourceGroup
>>>         properties:
>>>           count: 10
>>>           resource_def:
>>>             type: my_custom_server
>>>             properties:
>>>               prop_1: "..."
>>>               prop_2: "..."
>>>               ...
>>>
>>> And a corresponding provider template and environment file.
>>>
>>> Now I can get say the list of IP addresses or any custom value of each
>>> server from the ResourceGroup by using `{get_attr: [servers,
>>> ip_address]}` and outputs defined in the provider template.
>>>
>>> But I can't figure out how to pass that list back to each server in the
>>> group.
>>>
>>> This is something we use in TripleO for things like building a MySQL
>>> cluster, where each node in the cluster (the ResourceGroup) needs the
>>> addresses of all the other nodes.
>>
>> Yeah, this is kind of the perpetual problem with clusters. I've been
>> hoping that DNSaaS will show up in OpenStack soon and that that will be
>> a way to fix this issue.
>>
>> The other option is to have the cluster members discover each other
>> somehow (mDNS?), but people seem loath to do that.
>>
>>> Right now, we have the servers ungrouped in the top-level template so we
>>> can build this list manually. But if we move to ResourceGroups (or any
>>> other scaling mechanism, I think), this is no longer possible.
>>
>> So I believe the current solution is to abuse a Launch Config resource
>> as a store for the data, and then later retrieve it somehow? Possibly
>> you could do something along similar lines, but it's unclear how the
>> 'later retrieval' part would work... presumably it would have to involve
>> something outside of Heat closing the loop :(
>
> Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble
> figuring out how would that work. LaunchConfig represents an instance,
> right?
>
>>
>>> We can't pass the list to ResourceGroup's `resource_def` section because
>>> that causes a circular dependency.
>>>
>>> And I'm not aware of a way to attach a SoftwareConfig to a
>>> ResourceGroup. SoftwareDeployment only allows attaching a config to a
>>> single server.
>>
>> Yeah, and that would be a tricky thing to implement well, because a
>> resource group may not be a group of servers (but in many cases it may
>> be a group of nested stacks that each contain one or more servers, and
>> you'd want to be able to handle that too).
>
> Yeah, I worried about that, too :-(.
>
> Here's a proposal that might actually work, though:
>
> The provider resource exposes the reference to its inner instance by
> declaring it as one of its outputs. A SoftwareDeployment would learn to
> accept a list of Nova servers, too.
>
> Provider template:
>
>      resources:
>        my_server:
>          type: OS::Nova::Server
>          properties:
>            ...
>
>        ... (some other resource hidden in the provider template)
>
>      outputs:
>        inner_server:
>          value: {get_resource: my_server}
>        ip_address:
>          value: {get_attr: [controller_server, networks, private, 0]}
>
> Based on my limited testing, this already makes it possible to use the
> inner server with a SoftwareDeployment from another template that uses
> "my_server" as a provider resource.
>
> E.g.:
>
>      a_cluster_of_my_servers:
>        type: OS::Heat::ResourceGroup
>        properties:
>          count: 10
>          resource_def:
>            type: custom::my_server
>            ...
>
>      some_deploy:
>        type: OS::Heat::StructuredDeployment
>        properties:
>          server: {get_attr: [a_cluster_of_my_servers,
> resource.0.inner_server]}
>          config: {get_resource: some_config}
>
>
> So what if we allowed SoftwareDeployment to accept a list of servers in
> addition to accepting just one server? Or add another resource that does
> that.

I approve of that in principle. Only Steve Baker can tell us for sure if 
there are any technical roadblocks in the way of that, but I don't see any.

Maybe if we had a new resource type that was internally implemented as a 
nested stack... that might give us a way of tracking the individual 
deployment statuses for free.

cheers,
Zane.

> Then we could do:
>
>      mysql_cluster_deployment:
>        type: OS::Heat::StructuredDeployment
>        properties:
>          server_list: {get_attr: [a_cluster_of_my_servers, inner_server]}
>          config: {get_resource: mysql_cluster_config}
>          input_values:
>            cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers,
> ip_address]}
>
> This isn't that different from having a SoftwareDeployment accepting a
> single server and doesn't have any of the problems of allowing a
> ResourceGroup as a SoftwareDeployment target.
>
> What do you think?
>
>>
>>> Is there a way to do this that I'm missing? And if there isn't, is this
>>> something we could add to Heat? E.g. extending a SoftwareDeployment to
>>> accept ResourceGroups or adding another resource for that purpose.
>>>
>>> Thanks,
>>> Tomas