[openstack-dev] [TripleO][Heat] Conditionally passing properties in Heat
Harald Jensås
hjensas at redhat.com
Tue Jun 13 12:00:30 UTC 2017
On Thu, 2017-04-20 at 16:11 -0700, Dan Sneddon wrote:
> On 04/20/2017 12:37 AM, Steven Hardy wrote:
> > On Wed, Apr 19, 2017 at 02:51:28PM -0700, Dan Sneddon wrote:
> > > On 04/13/2017 12:01 AM, Rabi Mishra wrote:
> > > > On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <dsneddon at redhat.c
> > > > om
> > > > <mailto:dsneddon at redhat.com>> wrote:
> > > >
> > > > On 04/12/2017 01:22 PM, Thomas Herve wrote:
> > > > > On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <dsneddon at re
> > > > dhat.com
> > > > <mailto:dsneddon at redhat.com>> wrote:
> > > > >> I'm implementing predictable control plane IPs for
> > > > spine/leaf,
> > > > and I'm
> > > > >> running into a problem implementing this in the TripleO
> > > > Heat
> > > > templates.
> > > > >>
> > > > >> I have a review in progress [1] that works, but fails on
> > > > upgrade,
> > > > so I'm
> > > > >> looking for an alternative approach. I'm trying to
> > > > influence the IP
> > > > >> address that is selected for overcloud nodes' Control
> > > > Plane IP.
> > > > Here is
> > > > >> the current construct:
> > > > >>
> > > > >> Controller:
> > > > >> type: OS::TripleO::Server
> > > > >> metadata:
> > > > >> os-collect-config:
> > > > >> command: {get_param: ConfigCommand}
> > > > >> properties:
> > > > >> image: {get_param: controllerImage}
> > > > >> image_update_policy: {get_param:
> > > > ImageUpdatePolicy}
> > > > >> flavor: {get_param: OvercloudControlFlavor}
> > > > >> key_name: {get_param: KeyName}
> > > > >> networks:
> > > > >> - network: ctlplane # <- Here's where the port
> > > > is created
> > > > >>
> > > > >> If I add fixed_ip: to the networks element at the end of
> > > > the above, I
> > > > >> can select an IP address from the 'ctlplane' network,
> > > > like this:
> > > > >>
> > > > >> networks:
> > > > >> - network: ctlplane
> > > > >> fixed_ip: {get_attr: [ControlPlanePort,
> > > > ip_address]}
> > > > >>
> > > > >> But the problem is that if I pass a blank string to
> > > > fixed_ip, I
> > > > get an
> > > > >> error on deployment. This means that the old behavior of
> > > > automatically
> > > > >> selecting an IP doesn't work.
> > > > >>
> > > > >> I thought I has solved this by passing an external
> > > > Neutron port,
> > > > like this:
> > > > >>
> > > > >> networks:
> > > > >> - network: ctlplane
> > > > >> port: {get_attr: [ControlPlanePort, port_id]}
> > > > >>
> > > > >> Which works for deployments, but that fails on upgrades,
> > > > since the
> > > > >> original port was created as part of the Nova::Server
> > > > resource,
> > > > instead
> > > > >> of being an external resource.
> > > > >
> > > > > Can you detail how it fails? I was under the impression
> > > > we never
> > > > > replaced servers no matter what (or we try to do that, at
> > > > least). Is
> > > > > the issue that your new port is not the correct one?
> > > > >
> > > > >> I'm now looking for a way to use Heat conditionals to
> > > > apply the
> > > > fixed_ip
> > > > >> only if the value is not unset. Looking at the intrinsic
> > > > functions [2],
> > > > >> I don't see a way to do this. Is what I'm trying to do
> > > > with Heat
> > > > possible?
> > > > >
> > > > > You should be able to write something like that (not
> > > > tested):
> > > > >
> > > > > networks:
> > > > > if:
> > > > > - <my condition>
> > > > > - network: ctlplane
> > > > > fixed_ip: {get_attr: [ControlPlanePort,
> > > > ip_address]}
> > > > > - network: ctlplane
> > > > >
> > > > > The question is how to define your condition. Maybe:
> > > > >
> > > > > conditions:
> > > > > fixed_ip_condition:
> > > > > not:
> > > > > equals:
> > > > > - {get_attr: [ControlPlanePort, ip_address]}
> > > > > - ''
> > > > >
> > > > > To get back to the problem you stated first.
> > > > >
> > > > >
> > > > >> Another option I'm exploring is conditionally applying
> > > > resources. It
> > > > >> appears that would require duplicating the entire
> > > > TripleO::Server
> > > > stanza
> > > > >> in *-role.yaml so that there is one that uses fixed_ip
> > > > and one
> > > > that does
> > > > >> not. Which one is applied would be based on a condition
> > > > that tested
> > > > >> whether fixed_ip was blank or not. The downside of that
> > > > is that
> > > > it would
> > > > >> make the role definition confusing because there would
> > > > be a large
> > > > >> resource that was implemented twice, with only one line
> > > > difference
> > > > >> between them.
> > > > >
> > > > > You can define properties with conditions, so you
> > > > shouldn't need to
> > > > > rewrite everything.
> > > > >
> > > >
> > > > Thomas,
> > > >
> > > > Thanks, I will try your suggestions and that should get me
> > > > closer.
> > > >
> > > > The full error log is available here:
> > > > http://logs.openstack.org/78/413278/11/check-tripleo/gate-t
> > > > ripleo-ci-centos-7-ovb-updates/8d91762/console.html
> > > > <http://logs.openstack.org/78/413278/11/check-tripleo/gate-
> > > > tripleo-ci-centos-7-ovb-updates/8d91762/console.html>
> > > >
> > > > We do an interface_detach/attach when a port is replaced.
> > > > It seems to be failing[1] as this is not implemented for
> > > > ironic/baremetal driver. I could see a patch[2] to add that
> > > > functionality though.
> > > >
> > > > [1]
> > > > http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripl
> > > > eo-ci-centos-7-ovb-
> > > > updates/8d91762/logs/undercloud/var/log/nova/nova-
> > > > compute.txt.gz#_2017-04-12_00_26_15_475
> > > >
> > > > [2] https://review.openstack.org/#/c/419975/
> > > >
> > > > We retry a few times to check whether the detach/attach is
> > > > complete(it's
> > > > an async operation in nova and takes time), so the cryptic
> > > > error below
> > > > is coming from tenacity library which fails after the
> > > > configured number
> > > > of attempts.
> > > >
> > > > Here are the errors I am getting:
> > > >
> > > > 2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z
> > > > [overcloud-CephStorage-bkucn6ign34i-0-
> > > > 2yq2jbtwuu7k.CephStorage]:
> > > > UPDATE_FAILED RetryError: resources.CephStorage:
> > > > RetryError[<Future at
> > > > 0xdd62550 state=finished returned bool>]
> > > > 2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z
> > > > [overcloud-CephStorage-bkucn6ign34i-0-2yq2jbtwuu7k]:
> > > > UPDATE_FAILED
> > > > RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > > state=finished returned bool>]
> > > > 2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z
> > > > [overcloud-CephStorage-bkucn6ign34i.0]:
> > > > UPDATE_FAILED resources[0]:
> > > > RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > > state=finished returned bool>]
> > > > 2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z
> > > > [overcloud-CephStorage-bkucn6ign34i]:
> > > > UPDATE_FAILED resources[0]:
> > > > RetryError: resources.CephStorage: RetryError[<Future at
> > > > 0xdd62550
> > > > state=finished returned bool>]
> > > > 2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z
> > > > [overcloud-Controller-3lf3jauv4cbc-0-
> > > > ydowkb3nwsso.Controller]:
> > > > UPDATE_FAILED RetryError: resources.Controller:
> > > > RetryError[<Future at
> > > > 0xdc79b50 state=finished returned bool>]
> > > > 2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z
> > > > [overcloud-Controller-3lf3jauv4cbc-0-ydowkb3nwsso]:
> > > > UPDATE_FAILED
> > > > RetryError: resources.Controller: RetryError[<Future at
> > > > 0xdc79b50
> > > > state=finished returned bool>]
> > > > 2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z
> > > > [CephStorage]:
> > > > UPDATE_FAILED resources.CephStorage: resources[0]:
> > > > RetryError:
> > > > resources.CephStorage: RetryError[<Future at 0xdd62550
> > > > state=finished
> > > > returned bool>]
> > > >
> > > > --
> > > > Dan Sneddon | Senior Principal Software Engineer
> > > > dsneddon at redhat.com <mailto:dsneddon at redhat.com> |
> > > > redhat.com/openstack <http://redhat.com/openstack>
> > > > dsneddon:irc | @dxs:twitter
> > > >
> > > > --
> > > > Regards,
> > > > Rabi Misra
> > >
> > > Rabi,
> > >
> > > Thanks for the explanation on why the port replacement isn't
> > > working.
> > >
> > > Unfortunately, I tried the recommendation that Thomas Herve made
> > > about
> > > using conditionals, but that failed. It appears that you can't
> > > use a
> > > get_attr inside of a conditional statement, I get an error to
> > > that
> > > effect. I can use a get_param, but that doesn't help me check a
> > > value in
> > > a nested stack.
> >
> > Yes I noticed the same thing recently, which is a limitation of
> > conditions
> > that IMO we should look at fixing - everywhere else in HOT
> > templates
> > get_param and get_attr can be used interchangably.
> >
> > As a workaround could you move the conditional into the port
> > resource
> > nested stack? E.g if you returned the list we pass to "networks"
> > for the
> > server resource from the new ControlPlanePort nested stack perhaps,
> > and put
> > whatever logic we need to generate it inside the port nested stack?
> >
> > Attributes from e.g *-role.yaml would then be parameters in the
> > nested
> > stack, so you could do the conditional as Thomas suggested, or
> > perhaps
> > it's even simpler if you can just return different data from the
> > ctlplane.yaml and ctlplane_from_pool.yaml templates?
> >
> > Steve
>
> Steve,
>
> I tried moving the generation of the network parameters for the role
> into the port. That way, I shouldn't need a conditional because I can
> select which template to use (with or without a fixed_ip). That
> didn't
> work either, it doesn't appear that I can pass a list as a parameter
> to
> the "networks" property of TripleO::Server.
>
> "controller-role.yaml":
> Controller:
> type: OS::TripleO::Server
> metadata:
> os-collect-config:
> command: {get_param: ConfigCommand}
> properties:
> image: {get_param: controllerImage}
> image_update_policy: {get_param: ImageUpdatePolicy}
> flavor: {get_param: OvercloudControlFlavor}
> key_name: {get_param: KeyName}
> networks: {get_attr: [ControlPlanePort, port_info]}
>
> Then, the ControlPlanePort has this:
> "ctlplane.yaml":
> outputs:
> port_info:
> description: The newtork name to use for port
> value:
> - network: {get_param: ControlPlaneNetName}
>
> OR "ctlplane_from_pool.yaml" defines the same value as:
>
> port_info:
> description: The newtork name and fixed_ip to use for port
> value:
> - network: {get_param: ControlPlaneNetName}
> fixed_ip: {get_param: [IPPool, {get_param:
> ControlPlaneNetName},
> {get_param: NodeIndex}]}
>
> Unfortunately, this fails. For example, when passing only the
> network:
>
> """
> GET call to orchestration for
> http://172.20.0.1:8004/v1/f6d7244fe69c43359c293d2c9edf0ce5/stacks/ove
> r
> cloud-ObjectStorageIpListMap-kwmthuphny55/1d8b2ef4-8223-412b-b32f-
> 6e00a9651478/resources
> used reques
> t id req-50c24b3f-2118-444b-878f-94e7d0a3b949
> overcloud.Controller.0.Controller:
> resource_type: OS::TripleO::Server
> physical_resource_id:
> status: CREATE_FAILED
> status_reason: |
> BadRequest: resources.Controller: Bad networks format: network
> uuid
> is not in proper format (ctlplane) (HTTP 400) (Request-ID:
> req-e3da2bf7-91bb-4902-bdcc-0ab11d1e2e09)
> """
>
> Is there a way that I can pass a list between a nested resource
> (port)
> and the parent stack (the role in this case), so I can pass either a
> network or a network plus fixed IP? I'm not sure why this is failing,
> since simply passing ctlplane works in the original role definition:
>
> Controller:
> type: OS::TripleO::Server
> metadata:
> os-collect-config:
> command: {get_param: ConfigCommand}
> properties:
> image: {get_param: controllerImage}
> image_update_policy: {get_param: ImageUpdatePolicy}
> flavor: {get_param: OvercloudControlFlavor}
> key_name: {get_param: KeyName}
> networks:
> - network: ctlplane
>
I have been playing whit this quite a bit. And AFAICT from testing it
does not work to create a template that output a dict or a list and
pass that in as a network using get_attr.
----------------------------
heat_template_version: ocata
parameters:
ControlPlaneNetName:
description: Name of the Control Plane neutron network
default: provider
type: string
ControlPlaneSubnet:
description: Name of the Control Plane neutron subnet
default: provider
type: string
outputs:
portinfo:
value:
- {network: {get_param: ControlPlaneNetName}, subnet: {get_param:
ControlPlaneSubnet}}
----------------------------
heat_template_version: ocata
parameters:
FixedIP:
type: string
default: None
resources:
ctlplane_port:
type: OS::TripleO::Compute::Ports::ControlPlanePort
my_instance:
type: OS::Nova::Server
properties:
image: cirros
flavor: m1.small
networks: {get_attr: [ctlplane_port, portinfo]}
openstack stack create -t ctlplane_port_from_template.yaml stack -e
resource_registry.yaml
Results:
--------
# openstack stack resource show stack my_instance -c
resource_status_reason -f json
{
"resource_status_reason": "NotFound: resources.my_instance: The
resource could not be found.<br /><br />\n\n\n\nNeutron server returns
request_ids: ['req-feb1aa2e-e600-47e1-bed8-df18c556ec27']"
}
# openstack stack resource show stack ctlplane_port -c attributes -f
json
{
"attributes": {
"portinfo": [
{
"subnet": "provider",
"network": "provider"
}
]
}
}
AFAICT the data I try to send in, is correct.
I have also tried to pass only the dict like this:
output:
portinfo:
value: {network: {get_param: ControlPlaneNetName}, subnet:
{get_param: ControlPlaneSubnet}}
networks:
- {get_attr: [ctlplane_port, portinfo]}
In this case I get this error:
ERROR: One of the properties "network", "port" or "subnet" should be
set for the specified network of server "my_instance".
One way to make it work, is to hack the resource constraints.
If I remove the ip_addr constraint of fixed_ip I can do:
networks:
network: {get_attr: [CtlPlanePort, network]}
subnet: {get_attr: [CtlPlanePort, subnet]}
fixed_ip: {get_attr: [CtlPlanePort, fixed_ip]}
If I don't have "fixed_ip" as output in the ctlplane_port template, it
become "None" and the with the constraint removed this works.
Would it be an option to create "IPConstraint_or_None" custom
constraint? To allow either None or Valid IP address? (Unless we can
fix Heat to treat data it get's from get_attr properly...)
--
|Harald Jensås
More information about the OpenStack-dev
mailing list