[openstack-dev] [TripleO][Heat] Conditionally passing properties in Heat
Dan Sneddon
dsneddon at redhat.com
Thu Apr 20 23:11:13 UTC 2017
On 04/20/2017 12:37 AM, Steven Hardy wrote:
> On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:
>> On 04/13/2017 12:01 AM, Rabi Mishra wrote:
>>> On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <dsneddon at redhat.com
>>> <mailto:dsneddon at redhat.com>> wrote:
>>>
>>> On 04/12/2017 01:22 PM, Thomas Herve wrote:
>>> > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <dsneddon at redhat.com
>>> <mailto:dsneddon at redhat.com>> wrote:
>>> >> I'm implementing predictable control plane IPs for spine/leaf,
>>> and I'm
>>> >> running into a problem implementing this in the TripleO Heat
>>> templates.
>>> >>
>>> >> I have a review in progress [1] that works, but fails on upgrade,
>>> so I'm
>>> >> looking for an alternative approach. I'm trying to influence the IP
>>> >> address that is selected for overcloud nodes' Control Plane IP.
>>> Here is
>>> >> the current construct:
>>> >>
>>> >> Controller:
>>> >> type: OS::TripleO::Server
>>> >> metadata:
>>> >> os-collect-config:
>>> >> command: {get_param: ConfigCommand}
>>> >> properties:
>>> >> image: {get_param: controllerImage}
>>> >> image_update_policy: {get_param: ImageUpdatePolicy}
>>> >> flavor: {get_param: OvercloudControlFlavor}
>>> >> key_name: {get_param: KeyName}
>>> >> networks:
>>> >> - network: ctlplane # <- Here's where the port is created
>>> >>
>>> >> If I add fixed_ip: to the networks element at the end of the above, I
>>> >> can select an IP address from the 'ctlplane' network, like this:
>>> >>
>>> >> networks:
>>> >> - network: ctlplane
>>> >> fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>> >>
>>> >> But the problem is that if I pass a blank string to fixed_ip, I
>>> get an
>>> >> error on deployment. This means that the old behavior of
>>> automatically
>>> >> selecting an IP doesn't work.
>>> >>
>>> >> I thought I has solved this by passing an external Neutron port,
>>> like this:
>>> >>
>>> >> networks:
>>> >> - network: ctlplane
>>> >> port: {get_attr: [ControlPlanePort, port_id]}
>>> >>
>>> >> Which works for deployments, but that fails on upgrades, since the
>>> >> original port was created as part of the Nova::Server resource,
>>> instead
>>> >> of being an external resource.
>>> >
>>> > Can you detail how it fails? I was under the impression we never
>>> > replaced servers no matter what (or we try to do that, at least). Is
>>> > the issue that your new port is not the correct one?
>>> >
>>> >> I'm now looking for a way to use Heat conditionals to apply the
>>> fixed_ip
>>> >> only if the value is not unset. Looking at the intrinsic
>>> functions [2],
>>> >> I don't see a way to do this. Is what I'm trying to do with Heat
>>> possible?
>>> >
>>> > You should be able to write something like that (not tested):
>>> >
>>> > networks:
>>> > if:
>>> > - <my condition>
>>> > - network: ctlplane
>>> > fixed_ip: {get_attr: [ControlPlanePort, ip_address]}
>>> > - network: ctlplane
>>> >
>>> > The question is how to define your condition. Maybe:
>>> >
>>> > conditions:
>>> > fixed_ip_condition:
>>> > not:
>>> > equals:
>>> > - {get_attr: [ControlPlanePort, ip_address]}
>>> > - ''
>>> >
>>> > To get back to the problem you stated first.
>>> >
>>> >
>>> >> Another option I'm exploring is conditionally applying resources. It
>>> >> appears that would require duplicating the entire TripleO::Server
>>> stanza
>>> >> in *-role.yaml so that there is one that uses fixed_ip and one
>>> that does
>>> >> not. Which one is applied would be based on a condition that tested
>>> >> whether fixed_ip was blank or not. The downside of that is that
>>> it would
>>> >> make the role definition confusing because there would be a large
>>> >> resource that was implemented twice, with only one line difference
>>> >> between them.
>>> >
>>> > You can define properties with conditions, so you shouldn't need to
>>> > rewrite everything.
>>> >
>>>
>>> Thomas,
>>>
>>> Thanks, I will try your suggestions and that should get me closer.
>>>
>>> The full error log is available here:
>>> http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html
>>> <http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
>>>
>>> We do an interface_detach/attach when a port is replaced.
>>> It seems to be failing[1] as this is not implemented for
>>> ironic/baremetal driver. I could see a patch[2] to add that
>>> functionality though.
>>>
>>> [1]
>>> http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475
>>>
>>> [2] https://review.openstack.org/#/c/419975/
>>>
>>> We retry a few times to check whether the detach/attach is complete(it's
>>> an async operation in nova and takes time), so the cryptic error below
>>> is coming from tenacity library which fails after the configured number
>>> of attempts.
>>>
>>> Here are the errors I am getting:
>>>
>>> 2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
>>> [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k.CephStorage]:
>>> UPDATE_FAILED RetryError: resources.CephStorage: RetryError[<Future at
>>> 0xdd62550 state=finished returned bool>]
>>> 2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
>>> [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED
>>> RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>> state=finished returned bool>]
>>> 2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
>>> [overcloud-CephStorage-bkucn6ign34i.0]: UPDATE_FAILED resources[0]:
>>> RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>> state=finished returned bool>]
>>> 2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
>>> [overcloud-CephStorage-bkucn6ign34i]: UPDATE_FAILED resources[0]:
>>> RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550
>>> state=finished returned bool>]
>>> 2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
>>> [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso.Controller]:
>>> UPDATE_FAILED RetryError: resources.Controller: RetryError[<Future at
>>> 0xdc79b50 state=finished returned bool>]
>>> 2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
>>> [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED
>>> RetryError: resources.Controller: RetryError[<Future at 0xdc79b50
>>> state=finished returned bool>]
>>> 2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:
>>> UPDATE_FAILED resources.CephStorage: resources[0]: RetryError:
>>> resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished
>>> returned bool>]
>>>
>>> --
>>> Dan Sneddon | Senior Principal Software Engineer
>>> dsneddon at redhat.com <mailto:dsneddon at redhat.com> |
>>> redhat.com/openstack <http://redhat.com/openstack>
>>> dsneddon:irc | @dxs:twitter
>>>
>>> --
>>> Regards,
>>> Rabi Misra
>>
>> Rabi,
>>
>> Thanks for the explanation on why the port replacement isn't working.
>>
>> Unfortunately, I tried the recommendation that Thomas Herve made about
>> using conditionals, but that failed. It appears that you can't use a
>> get_attr inside of a conditional statement, I get an error to that
>> effect. I can use a get_param, but that doesn't help me check a value in
>> a nested stack.
>
> Yes I noticed the same thing recently, which is a limitation of conditions
> that IMO we should look at fixing - everywhere else in HOT templates
> get_param and get_attr can be used interchangably.
>
> As a workaround could you move the conditional into the port resource
> nested stack? E.g if you returned the list we pass to "networks" for the
> server resource from the new ControlPlanePort nested stack perhaps, and put
> whatever logic we need to generate it inside the port nested stack?
>
> Attributes from e.g *-role.yaml would then be parameters in the nested
> stack, so you could do the conditional as Thomas suggested, or perhaps
> it's even simpler if you can just return different data from the
> ctlplane.yaml and ctlplane_from_pool.yaml templates?
>
> Steve
Steve,
I tried moving the generation of the network parameters for the role
into the port. That way, I shouldn't need a conditional because I can
select which template to use (with or without a fixed_ip). That didn't
work either, it doesn't appear that I can pass a list as a parameter to
the "networks" property of TripleO::Server.
"controller-role.yaml":
Controller:
type: OS::TripleO::Server
metadata:
os-collect-config:
command: {get_param: ConfigCommand}
properties:
image: {get_param: controllerImage}
image_update_policy: {get_param: ImageUpdatePolicy}
flavor: {get_param: OvercloudControlFlavor}
key_name: {get_param: KeyName}
networks: {get_attr: [ControlPlanePort, port_info]}
Then, the ControlPlanePort has this:
"ctlplane.yaml":
outputs:
port_info:
description: The newtork name to use for port
value:
- network: {get_param: ControlPlaneNetName}
OR "ctlplane_from_pool.yaml" defines the same value as:
port_info:
description: The newtork name and fixed_ip to use for port
value:
- network: {get_param: ControlPlaneNetName}
fixed_ip: {get_param: [IPPool, {get_param: ControlPlaneNetName},
{get_param: NodeIndex}]}
Unfortunately, this fails. For example, when passing only the network:
"""
GET call to orchestration for
http://172.20.0.1:8004/v1/f6d7244fe69c43359c293d2c9edf0ce5/stacks/over
cloud-ObjectStorageIpListMap-kwmthuphny55/1d8b2ef4-8223-412b-b32f-6e00a9651478/resources
used reques
t id req-50c24b3f-2118-444b-878f-94e7d0a3b949
overcloud.Controller.0.Controller:
resource_type: OS::TripleO::Server
physical_resource_id:
status: CREATE_FAILED
status_reason: |
BadRequest: resources.Controller: Bad networks format: network uuid
is not in proper format (ctlplane) (HTTP 400) (Request-ID:
req-e3da2bf7-91bb-4902-bdcc-0ab11d1e2e09)
"""
Is there a way that I can pass a list between a nested resource (port)
and the parent stack (the role in this case), so I can pass either a
network or a network plus fixed IP? I'm not sure why this is failing,
since simply passing ctlplane works in the original role definition:
Controller:
type: OS::TripleO::Server
metadata:
os-collect-config:
command: {get_param: ConfigCommand}
properties:
image: {get_param: controllerImage}
image_update_policy: {get_param: ImageUpdatePolicy}
flavor: {get_param: OvercloudControlFlavor}
key_name: {get_param: KeyName}
networks:
- network: ctlplane
--
Dan Sneddon | Senior Principal Software Engineer
dsneddon at redhat.com | redhat.com/openstack
dsneddon:irc | @dxs:twitter
More information about the OpenStack-dev
mailing list