Open Stack

On 12/07/14 06:41, Zane Bitter wrote:
> On 11/07/14 09:37, Tomas Sedovic wrote:
>> Hi all,
>>
>> This is a follow-up to Clint Byrum's suggestion to add the `Map`
>> intrinsic function[0], Zane Bitter's response[1] and Randall Burt's
>> addendum[2].
>>
>> Sorry for bringing it up again, but I'd love to reach consensus on this.
>> The summary of the previous conversation:
> 
> Please keep bringing it up until you get a straight answer ;)
> 
>> 1. TripleO is using some functionality currently not supported by Heat
>> around scaled-out resources
>> 2. Clint proposed a `map` intrinsic function that would solve it
>> 3. Zane said Heat have historically been against a for-loop functionality
>> 4. Randall suggested ResourceGroup's attribute passthrough may do what
>> we need
>>
>> I've looked at the ResourceGroup code and experimented a bit. It does do
>> some of what TripleO needs but not all.
> 
> Many thanks for putting this together Tomas, this is exactly the kind of
> information that is _incredibly_ helpful in knowing what sort of
> features we need in HOT. Fantastic work :)
> 
>> Here's what we're doing with our scaled-out resources (what we'd like to
>> wrap in a ResourceGroup or similar in the future):
>>
>>
>> 1. Building a coma-separated list of RabbitMQ nodes:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L642
>>
>>
>> This one is easy with ResourceGroup's inner attribute support:
>>
>>      list_join:
>>      - ", "
>>      - {get_attr: [controller_group, name]}
>>
>> (controller_group is a ResourceGroup of Nova servers)
>>
>>
>> 2. Get the name of the first Controller node:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L339
>>
>>
>> Possible today:
>>
>>      {get_attr: [controller_group, resource.0.name]}
>>
>>
>> 3. List of IP addresses of all controllers:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L405
>>
>>
>> We cannot do this, because resource group doesn't support extended
>> attributes.
>>
>> Would need something like:
>>
>>      {get_attr: [controller_group, networks, ctlplane, 0]}
>>
>> (ctlplane is the network controller_group servers are on)
> 
> I was going to give an explanation of how we could implement this, but
> then I realised a patch was going to be easier:
> 
> https://review.openstack.org/#/c/106541/
> https://review.openstack.org/#/c/106542/

Thanks, that looks great.

> 
>> 4. IP address of the first node in the resource group:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/swift-deploy.yaml#L29
>>
>>
>> Can't do: extended attributes are not supported for the n-th node for
>> the group either.
> 
> I believe this is possible today using:
> 
>   {get_attr: [controller_group, resource.0.networks, ctlplane, 0]}

Yeah, I've missed this. I have actually checked the ResourceGroup's
GetAtt method but didn't realise the connection with the GetAtt function
so I hadn't tried it before.

> 
>> This can be solved by `get_resource` working with resource IDs:
>>
>>     get_attr:
>>     - {get_attr: [controller_group, resource.0]}
>>     - [networks, ctlplane, 0]
>>
>> (i.e. we get the server's ID from the ResourceGroup and change
>> `get_attr` to work with the ID's too. Would also work if `get_resource`
>> understood IDs).
> 
> This is never going to happen.
> 
> Think of get_resource as returning an object whose string representation
> is the UUID of the named resource (get_attr is similar, but returning
> attributes instead). It doesn't mean that having the UUID of a resource
> is the same as having the resource itself; the UUID could have come from
> anywhere. What you're talking about is a radical departure from the
> existing, very simple but extremely effective, model toward something
> that's extremely difficult to analyse with lots of nasty edge cases.
> It's common for people to think they want this, but it always turns out
> there's a better way to achieve their goal within the existing data model.

Right, that makes sense. I don't think I fully grasped the existing
model so this felt like a nice quick fix.

> 
>> Alternatively, we could extend the ResourceGroup's get_attr behaviour:
>>
>>      {get_attr: [controller_group, resource.0.networks.ctlplane.0]}
>>
>> but the former is a bit cleaner and more generic.
> 
> I wrote a patch that implements this (and also handles (3) above in a
> similar manner), but in the end I decided that this:
> 
>   {get_attr: [controller_group, resource.0, networks, ctlplane, 0]}
> 
> would be better than either that or the current syntax (which was
> obviously obscure enough that you didn't discover it). My only
> reservation was that it might make things a little weird when we have an
> autoscaling API to get attributes from compared with the dotted syntax
> that you suggest, but I soon got over it ;)

So now that I understand how this works, I'm not against keeping things
the way we are. There is a consistency there, we just need to document
it and perhaps show some examples.

> 
>> ---
>>
>>
>> That was the easy stuff, where we can get by with the current
>> functionality (plus a few fixes).
>>
>> What follows are examples that really need new intrinsic functions (or
>> seriously complicating the ResourceGroup attribute code and syntax).
>>
>>
>> 5. Building a list of {ip: ..., name: ...} dictionaries to configure
>> haproxy:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L478
>>
>>
>> This really calls for a mapping/for-each kind of functionality. Trying
>> to invent a ResourceGroup syntax for this would be perverse.
>>
>> Here's what it could look like under Clint's `map` proposal:
>>
>>      map:
>>      - ip: {get_attr: [{get_resource: "$1"}, networks, ctlplane, 0]
>>        name: {get_attr: [{get_resource: "$1"}, name]}
>>      - {get_attr: [compute_group, refs]}
> 
> This has always been the tricky one :D
> 
> IMHO the real problem here is that we're trying to collate the data at
> the point where it is consumed, not the point where it is produced. It's
> not like we were just given a big blob of data and now have to somehow
> extract the useful parts; we produced it ourselves by combining data
> from the scaled units. If we didn't get the right data, we have only
> ourselves to blame ;)
> 
> So if the provider template that defines e.g. a compute node contains
> the section:
> 
>   outputs:
>     host_entry:
>       value:
>         ip: {get_attr: [compute_server, networks, ctlplane, 0]}
>         name: {get_attr: [compute_server, name]}
> 
> Then in your main template all you need to do is:
> 
>   {getattr: [compute_group, host_entry]}

Oh this is absolutely wonderful. I've had all the pieces in my head but
I didn't make the connection.

You're completely right about using a provider template here anyway --
that's what I planned to do, but I didn't fully appreciate the
connection between outputs and attributes (even though I knew about it).

> 
> to get the list of {ip: ..., name: ...} dicts. (As a bonus, this is
> about as straightforward to read as I can imagine it ever getting.)
> 
> Note that you *will* want to be using a provider template as the scaled
> unit _anyway_, because each compute_server will have associated software
> deployments and quite possibly a bunch of other resources that need to
> be in the scaled unit.

Yep, exactly.

> 
> There is one aspect of this that probably doesn't work yet: originally
> outputs and attributes were only allowed to be strings. We changed that
> for attributes, but probably not yet for outputs (even though outputs of
> provider templates become attributes of the facade resource). But that
> should be easy to fix. (And if your data can be returned as a string, it
> should already work.)

Unless I misunderstood what you're saying, it seems to be working now:

controller.yaml:

    outputs:
      hosts_entry:
        description: An IP address and a hostname of the server
        value:
          ip: {get_attr: [controller_server, networks, private, 0]}
          name: {get_attr: [controller_server, name]}

environment.yaml:

    resource_registry:
      OS::TripleO::Controller: controller.yaml

test-resource-group.yaml:

    resources:
      servers:
        type: OS::Heat::ResourceGroup
        properties:
          count: 3
          resource_def:
            type: OS::TripleO::Controller
            properties:
              key_name: {get_param: key_name}
              image: {get_param: image_id}

    outputs:
      hosts:
        description: "/etc/hosts entries for each server"
        value: {get_attr: [servers, hosts_entry]}

Heat stack-show test-resource-group:

   {
     "output_value": [
       "{u'ip': u'10.0.0.4', u'name':
u'rg-7heh-0-tweejsvubaht-controller_server-mscy33sbtirn'}",
       "{u'ip': u'10.0.0.3', u'name':
u'rg-7heh-1-o4szl7lry27d-controller_server-sxpkalgi27ii'}",
       "{u'ip': u'10.0.0.2', u'name':
u'rg-7heh-2-l2y6rqxml2fi-controller_server-u4jcjacjdrea'}"
     ],
     "description": "/etc/hosts entries for each server",
     "output_key": "hosts"
   },


> 
>> (this relies on `get_resource` working with resource IDs. Alternatively,
> 
> That's a show-stopper right out of the gate. If we implemented the map
> thing you would have to use something like:
> 
>       map:
>       - ip:$1
>         name: $2
>       - {get_attr: [compute_group, networks, ctlplane, 0]}
>       - {get_attr: [compute_group, name]}
> 
> (which is IMHO about 50 times easier to read anyway.)
> 
>> we could have a `resources` attribute for ResourceGroup that returns
>> objects that can be used with get_attr.
> 
> Sounds increasingly hacky.
> 
>> 6. Building the /etc/hosts file
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L585
>>
>>
>> Same as above, but also joining two lists together.
>>
>> We can use nested {list_join: ["\n", [...]} just as we're doing now, but
>> having a `concat_list` function would make this and some other cases
>> shorter and clearer.
> 
> I'm not strongly opposed to this one. We were trying to keep the number
> of functions in HOT as small as possible... and it does look like it is
> possible to do what you need already... so I would treat this as a low
> priority.

Agreed.

> 
>> 7. Building the list of Swift devices:
>>
>> https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/swift-deploy.yaml#L23
>>
>>
>> In addition to the abowe, we're adding a single element at the beginning
>> of a list.
>>
>> Asking for a `cons` support is pushing it, right? ;-)
> 
> lol, Greenspun's Tenth Law in action right here :D

I *have* been saying we should switch to s-expressions all along O:-).

> 
>> We could just wrap that in a list and use `concat_list` or keep using
>> nested `list_join`s as in the /etc/hosts case.
> 
> +1
> 
>> ----
>>
>>
>> So this boils down to 4 features proposals:
>>
>> 1. Support extended attributes in ResourceGroup's members
> 
> Sorted.

Yep

> 
>> 2. Allow a way to use a Resource ID (e.g. what you get by {get_attr:
>> [ResourceGroup, refs]} or {get_attr: [ResourceGroup, resource.0]}) with
>> existing intrinsic functions (get_resource, get_attr)
> 
> No dice, but (1) solves the problem anyway.

Agreed

> 
>> 3. A `map` intrinsic function that turns a list of items to another list
>> by doing operations on each item
> 
> There may be a better solution available to us already, so IMO
> investigate that first. If that turns out not to be the case then we'll
> need to reach a consensus on whether map is something we want.

You're right. I no longer think map (or anything like it) is necessary.

> 
>> 4. A `concat_list` intrinsic function that joins multiple lists into one.
> 
> Low priority.

Yeah.

> 
>> I think the first two are not controversial. What about the other two?
>> I've shown you some examples where we would find a good use in the
>> TripleO templates. The lack of `map` actually blocks us from going
>> all-Heat.
> 
> Hopefully that's not actually the case.
> 
>> The alternative would be to say that this sort of stuff to be done
>> inside the instance by os-apply-config et al. It would complicate things
>> for TripleO, but oh well.
> 
> It seems to me that the alternative is not necessarily to modify
> os-apply-config, but rather to provide a software config with a script
> that converts the data from whatever format Heat can supply to whatever
> format is needed by the application. Although I don't think it's even
> required in the specific case you mention, I find that a much better
> answer for the general case than turning the template format into the
> JSON equivalent of XSLT ;)

That's what I thought -- not rewriting os-apply-config but simply
passing the raw data to it and changing the tripleo image elements to
output the configuration format required by the applications.

+1 on the XSLT sentiment, I just didn't realise we pretty much solve
everything with the existing model (hat off to the design btw, I love
how it works together -- we just maybe need to communicate it a bit more).

> 
>> Either way, can we come to a clear answer? I would love to convert our
>> templates to 100% Natural Heat during Juno and I'm willing to put my
>> time towards coding the necessary features.
> 
> +1 that is a worthy goal and we appreciate your contributions toward
> reaching it.

Thanks so much for your answers, Zane. They were most helpful.

> 
> cheers,
> Zane.
> 
>> Tomas
>>
>>
>> [0]:
>> http://lists.openstack.org/pipermail/openstack-dev/2014-February/027536.html
>>
>> [1]:
>> http://lists.openstack.org/pipermail/openstack-dev/2014-April/031944.html
>> [2]:
>> http://lists.openstack.org/pipermail/openstack-dev/2014-April/032074.html
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
>