Open Stack

On 12/08/14 01:06, Steve Baker wrote:
> On 09/08/14 11:15, Zane Bitter wrote:
>> On 08/08/14 11:07, Tomas Sedovic wrote:
>>> On 08/08/14 00:53, Zane Bitter wrote:
>>>> On 07/08/14 13:22, Tomas Sedovic wrote:
>>>>> Hi all,
>>>>>
>>>>> I have a ResourceGroup which wraps a custom resource defined in
>>>>> another
>>>>> template:
>>>>>
>>>>>       servers:
>>>>>         type: OS::Heat::ResourceGroup
>>>>>         properties:
>>>>>           count: 10
>>>>>           resource_def:
>>>>>             type: my_custom_server
>>>>>             properties:
>>>>>               prop_1: "..."
>>>>>               prop_2: "..."
>>>>>               ...
>>>>>
>>>>> And a corresponding provider template and environment file.
>>>>>
>>>>> Now I can get say the list of IP addresses or any custom value of each
>>>>> server from the ResourceGroup by using `{get_attr: [servers,
>>>>> ip_address]}` and outputs defined in the provider template.
>>>>>
>>>>> But I can't figure out how to pass that list back to each server in
>>>>> the
>>>>> group.
>>>>>
>>>>> This is something we use in TripleO for things like building a MySQL
>>>>> cluster, where each node in the cluster (the ResourceGroup) needs the
>>>>> addresses of all the other nodes.
>>>>
>>>> Yeah, this is kind of the perpetual problem with clusters. I've been
>>>> hoping that DNSaaS will show up in OpenStack soon and that that will be
>>>> a way to fix this issue.
>>>>
>>>> The other option is to have the cluster members discover each other
>>>> somehow (mDNS?), but people seem loath to do that.
>>>>
>>>>> Right now, we have the servers ungrouped in the top-level template
>>>>> so we
>>>>> can build this list manually. But if we move to ResourceGroups (or any
>>>>> other scaling mechanism, I think), this is no longer possible.
>>>>
>>>> So I believe the current solution is to abuse a Launch Config resource
>>>> as a store for the data, and then later retrieve it somehow? Possibly
>>>> you could do something along similar lines, but it's unclear how the
>>>> 'later retrieval' part would work... presumably it would have to
>>>> involve
>>>> something outside of Heat closing the loop :(
>>>
>>> Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble
>>> figuring out how would that work. LaunchConfig represents an instance,
>>> right?
>>>
>>>>
>>>>> We can't pass the list to ResourceGroup's `resource_def` section
>>>>> because
>>>>> that causes a circular dependency.
>>>>>
>>>>> And I'm not aware of a way to attach a SoftwareConfig to a
>>>>> ResourceGroup. SoftwareDeployment only allows attaching a config to a
>>>>> single server.
>>>>
>>>> Yeah, and that would be a tricky thing to implement well, because a
>>>> resource group may not be a group of servers (but in many cases it may
>>>> be a group of nested stacks that each contain one or more servers, and
>>>> you'd want to be able to handle that too).
>>>
>>> Yeah, I worried about that, too :-(.
>>>
>>> Here's a proposal that might actually work, though:
>>>
>>> The provider resource exposes the reference to its inner instance by
>>> declaring it as one of its outputs. A SoftwareDeployment would learn to
>>> accept a list of Nova servers, too.
>>>
>>> Provider template:
>>>
>>>      resources:
>>>        my_server:
>>>          type: OS::Nova::Server
>>>          properties:
>>>            ...
>>>
>>>        ... (some other resource hidden in the provider template)
>>>
>>>      outputs:
>>>        inner_server:
>>>          value: {get_resource: my_server}
>>>        ip_address:
>>>          value: {get_attr: [controller_server, networks, private, 0]}
>>>
>>> Based on my limited testing, this already makes it possible to use the
>>> inner server with a SoftwareDeployment from another template that uses
>>> "my_server" as a provider resource.
>>>
>>> E.g.:
>>>
>>>      a_cluster_of_my_servers:
>>>        type: OS::Heat::ResourceGroup
>>>        properties:
>>>          count: 10
>>>          resource_def:
>>>            type: custom::my_server
>>>            ...
>>>
>>>      some_deploy:
>>>        type: OS::Heat::StructuredDeployment
>>>        properties:
>>>          server: {get_attr: [a_cluster_of_my_servers,
>>> resource.0.inner_server]}
>>>          config: {get_resource: some_config}
>>>
>>>
>>> So what if we allowed SoftwareDeployment to accept a list of servers in
>>> addition to accepting just one server? Or add another resource that does
>>> that.
>>
>> I approve of that in principle. Only Steve Baker can tell us for sure
>> if there are any technical roadblocks in the way of that, but I don't
>> see any.
>>
>> Maybe if we had a new resource type that was internally implemented as
>> a nested stack... that might give us a way of tracking the individual
>> deployment statuses for free.
>>
>> cheers,
>> Zane.
>>
>>> Then we could do:
>>>
>>>      mysql_cluster_deployment:
>>>        type: OS::Heat::StructuredDeployment
>>>        properties:
>>>          server_list: {get_attr: [a_cluster_of_my_servers,
>>> inner_server]}
>>>          config: {get_resource: mysql_cluster_config}
>>>          input_values:
>>>            cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers,
>>> ip_address]}
>>>
>>> This isn't that different from having a SoftwareDeployment accepting a
>>> single server and doesn't have any of the problems of allowing a
>>> ResourceGroup as a SoftwareDeployment target.
>>>
>>> What do you think?
> All the other solutions I can think of will result in circular issues.
> 
> I'll start looking at a spec to create a resource which is capable of
> deploying a config to a list of servers.

Thanks. If there's any way I could help with this, let me know. I'll be
happy to.

> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>