Open Stack

On 08/08/14 00:53, Zane Bitter wrote:
> On 07/08/14 13:22, Tomas Sedovic wrote:
>> Hi all,
>>
>> I have a ResourceGroup which wraps a custom resource defined in another
>> template:
>>
>>      servers:
>>        type: OS::Heat::ResourceGroup
>>        properties:
>>          count: 10
>>          resource_def:
>>            type: my_custom_server
>>            properties:
>>              prop_1: "..."
>>              prop_2: "..."
>>              ...
>>
>> And a corresponding provider template and environment file.
>>
>> Now I can get say the list of IP addresses or any custom value of each
>> server from the ResourceGroup by using `{get_attr: [servers,
>> ip_address]}` and outputs defined in the provider template.
>>
>> But I can't figure out how to pass that list back to each server in the
>> group.
>>
>> This is something we use in TripleO for things like building a MySQL
>> cluster, where each node in the cluster (the ResourceGroup) needs the
>> addresses of all the other nodes.
> 
> Yeah, this is kind of the perpetual problem with clusters. I've been
> hoping that DNSaaS will show up in OpenStack soon and that that will be
> a way to fix this issue.
> 
> The other option is to have the cluster members discover each other
> somehow (mDNS?), but people seem loath to do that.
> 
>> Right now, we have the servers ungrouped in the top-level template so we
>> can build this list manually. But if we move to ResourceGroups (or any
>> other scaling mechanism, I think), this is no longer possible.
> 
> So I believe the current solution is to abuse a Launch Config resource
> as a store for the data, and then later retrieve it somehow? Possibly
> you could do something along similar lines, but it's unclear how the
> 'later retrieval' part would work... presumably it would have to involve
> something outside of Heat closing the loop :(

Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble
figuring out how would that work. LaunchConfig represents an instance,
right?

> 
>> We can't pass the list to ResourceGroup's `resource_def` section because
>> that causes a circular dependency.
>>
>> And I'm not aware of a way to attach a SoftwareConfig to a
>> ResourceGroup. SoftwareDeployment only allows attaching a config to a
>> single server.
> 
> Yeah, and that would be a tricky thing to implement well, because a
> resource group may not be a group of servers (but in many cases it may
> be a group of nested stacks that each contain one or more servers, and
> you'd want to be able to handle that too).

Yeah, I worried about that, too :-(.

Here's a proposal that might actually work, though:

The provider resource exposes the reference to its inner instance by
declaring it as one of its outputs. A SoftwareDeployment would learn to
accept a list of Nova servers, too.

Provider template:

    resources:
      my_server:
        type: OS::Nova::Server
        properties:
          ...

      ... (some other resource hidden in the provider template)

    outputs:
      inner_server:
        value: {get_resource: my_server}
      ip_address:
        value: {get_attr: [controller_server, networks, private, 0]}

Based on my limited testing, this already makes it possible to use the
inner server with a SoftwareDeployment from another template that uses
"my_server" as a provider resource.

E.g.:

    a_cluster_of_my_servers:
      type: OS::Heat::ResourceGroup
      properties:
        count: 10
        resource_def:
          type: custom::my_server
          ...

    some_deploy:
      type: OS::Heat::StructuredDeployment
      properties:
        server: {get_attr: [a_cluster_of_my_servers,
resource.0.inner_server]}
        config: {get_resource: some_config}


So what if we allowed SoftwareDeployment to accept a list of servers in
addition to accepting just one server? Or add another resource that does
that.

Then we could do:

    mysql_cluster_deployment:
      type: OS::Heat::StructuredDeployment
      properties:
        server_list: {get_attr: [a_cluster_of_my_servers, inner_server]}
        config: {get_resource: mysql_cluster_config}
        input_values:
          cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers,
ip_address]}

This isn't that different from having a SoftwareDeployment accepting a
single server and doesn't have any of the problems of allowing a
ResourceGroup as a SoftwareDeployment target.

What do you think?

> 
>> Is there a way to do this that I'm missing? And if there isn't, is this
>> something we could add to Heat? E.g. extending a SoftwareDeployment to
>> accept ResourceGroups or adding another resource for that purpose.
>>
>> Thanks,
>> Tomas
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>