Open Stack

On 09/08/14 11:15, Zane Bitter wrote:
> On 08/08/14 11:07, Tomas Sedovic wrote:
>> On 08/08/14 00:53, Zane Bitter wrote:
>>> On 07/08/14 13:22, Tomas Sedovic wrote:
>>>> Hi all,
>>>>
>>>> I have a ResourceGroup which wraps a custom resource defined in
>>>> another
>>>> template:
>>>>
>>>>       servers:
>>>>         type: OS::Heat::ResourceGroup
>>>>         properties:
>>>>           count: 10
>>>>           resource_def:
>>>>             type: my_custom_server
>>>>             properties:
>>>>               prop_1: "..."
>>>>               prop_2: "..."
>>>>               ...
>>>>
>>>> And a corresponding provider template and environment file.
>>>>
>>>> Now I can get say the list of IP addresses or any custom value of each
>>>> server from the ResourceGroup by using `{get_attr: [servers,
>>>> ip_address]}` and outputs defined in the provider template.
>>>>
>>>> But I can't figure out how to pass that list back to each server in
>>>> the
>>>> group.
>>>>
>>>> This is something we use in TripleO for things like building a MySQL
>>>> cluster, where each node in the cluster (the ResourceGroup) needs the
>>>> addresses of all the other nodes.
>>>
>>> Yeah, this is kind of the perpetual problem with clusters. I've been
>>> hoping that DNSaaS will show up in OpenStack soon and that that will be
>>> a way to fix this issue.
>>>
>>> The other option is to have the cluster members discover each other
>>> somehow (mDNS?), but people seem loath to do that.
>>>
>>>> Right now, we have the servers ungrouped in the top-level template
>>>> so we
>>>> can build this list manually. But if we move to ResourceGroups (or any
>>>> other scaling mechanism, I think), this is no longer possible.
>>>
>>> So I believe the current solution is to abuse a Launch Config resource
>>> as a store for the data, and then later retrieve it somehow? Possibly
>>> you could do something along similar lines, but it's unclear how the
>>> 'later retrieval' part would work... presumably it would have to
>>> involve
>>> something outside of Heat closing the loop :(
>>
>> Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble
>> figuring out how would that work. LaunchConfig represents an instance,
>> right?
>>
>>>
>>>> We can't pass the list to ResourceGroup's `resource_def` section
>>>> because
>>>> that causes a circular dependency.
>>>>
>>>> And I'm not aware of a way to attach a SoftwareConfig to a
>>>> ResourceGroup. SoftwareDeployment only allows attaching a config to a
>>>> single server.
>>>
>>> Yeah, and that would be a tricky thing to implement well, because a
>>> resource group may not be a group of servers (but in many cases it may
>>> be a group of nested stacks that each contain one or more servers, and
>>> you'd want to be able to handle that too).
>>
>> Yeah, I worried about that, too :-(.
>>
>> Here's a proposal that might actually work, though:
>>
>> The provider resource exposes the reference to its inner instance by
>> declaring it as one of its outputs. A SoftwareDeployment would learn to
>> accept a list of Nova servers, too.
>>
>> Provider template:
>>
>>      resources:
>>        my_server:
>>          type: OS::Nova::Server
>>          properties:
>>            ...
>>
>>        ... (some other resource hidden in the provider template)
>>
>>      outputs:
>>        inner_server:
>>          value: {get_resource: my_server}
>>        ip_address:
>>          value: {get_attr: [controller_server, networks, private, 0]}
>>
>> Based on my limited testing, this already makes it possible to use the
>> inner server with a SoftwareDeployment from another template that uses
>> "my_server" as a provider resource.
>>
>> E.g.:
>>
>>      a_cluster_of_my_servers:
>>        type: OS::Heat::ResourceGroup
>>        properties:
>>          count: 10
>>          resource_def:
>>            type: custom::my_server
>>            ...
>>
>>      some_deploy:
>>        type: OS::Heat::StructuredDeployment
>>        properties:
>>          server: {get_attr: [a_cluster_of_my_servers,
>> resource.0.inner_server]}
>>          config: {get_resource: some_config}
>>
>>
>> So what if we allowed SoftwareDeployment to accept a list of servers in
>> addition to accepting just one server? Or add another resource that does
>> that.
>
> I approve of that in principle. Only Steve Baker can tell us for sure
> if there are any technical roadblocks in the way of that, but I don't
> see any.
>
> Maybe if we had a new resource type that was internally implemented as
> a nested stack... that might give us a way of tracking the individual
> deployment statuses for free.
>
> cheers,
> Zane.
>
>> Then we could do:
>>
>>      mysql_cluster_deployment:
>>        type: OS::Heat::StructuredDeployment
>>        properties:
>>          server_list: {get_attr: [a_cluster_of_my_servers,
>> inner_server]}
>>          config: {get_resource: mysql_cluster_config}
>>          input_values:
>>            cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers,
>> ip_address]}
>>
>> This isn't that different from having a SoftwareDeployment accepting a
>> single server and doesn't have any of the problems of allowing a
>> ResourceGroup as a SoftwareDeployment target.
>>
>> What do you think?
All the other solutions I can think of will result in circular issues.

I'll start looking at a spec to create a resource which is capable of
deploying a config to a list of servers.