<div dir="ltr">On Thu, Apr 13, 2017 at 2:14 AM, Dan Sneddon <span dir="ltr"><<a href="mailto:dsneddon@redhat.com" target="_blank">dsneddon@redhat.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail-HOEnZb"><div class="gmail-h5">On 04/12/2017 01:22 PM, Thomas Herve wrote:<br>
> On Wed, Apr 12, 2017 at 9:00 PM, Dan Sneddon <<a href="mailto:dsneddon@redhat.com">dsneddon@redhat.com</a>> wrote:<br>
>> I'm implementing predictable control plane IPs for spine/leaf, and I'm<br>
>> running into a problem implementing this in the TripleO Heat templates.<br>
>><br>
>> I have a review in progress [1] that works, but fails on upgrade, so I'm<br>
>> looking for an alternative approach. I'm trying to influence the IP<br>
>> address that is selected for overcloud nodes' Control Plane IP. Here is<br>
>> the current construct:<br>
>><br>
>> Controller:<br>
>> type: OS::TripleO::Server<br>
>> metadata:<br>
>> os-collect-config:<br>
>> command: {get_param: ConfigCommand}<br>
>> properties:<br>
>> image: {get_param: controllerImage}<br>
>> image_update_policy: {get_param: ImageUpdatePolicy}<br>
>> flavor: {get_param: OvercloudControlFlavor}<br>
>> key_name: {get_param: KeyName}<br>
>> networks:<br>
>> - network: ctlplane # <- Here's where the port is created<br>
>><br>
>> If I add fixed_ip: to the networks element at the end of the above, I<br>
>> can select an IP address from the 'ctlplane' network, like this:<br>
>><br>
>> networks:<br>
>> - network: ctlplane<br>
>> fixed_ip: {get_attr: [ControlPlanePort, ip_address]}<br>
>><br>
>> But the problem is that if I pass a blank string to fixed_ip, I get an<br>
>> error on deployment. This means that the old behavior of automatically<br>
>> selecting an IP doesn't work.<br>
>><br>
>> I thought I has solved this by passing an external Neutron port, like this:<br>
>><br>
>> networks:<br>
>> - network: ctlplane<br>
>> port: {get_attr: [ControlPlanePort, port_id]}<br>
>><br>
>> Which works for deployments, but that fails on upgrades, since the<br>
>> original port was created as part of the Nova::Server resource, instead<br>
>> of being an external resource.<br>
><br>
> Can you detail how it fails? I was under the impression we never<br>
> replaced servers no matter what (or we try to do that, at least). Is<br>
> the issue that your new port is not the correct one?<br>
><br>
>> I'm now looking for a way to use Heat conditionals to apply the fixed_ip<br>
>> only if the value is not unset. Looking at the intrinsic functions [2],<br>
>> I don't see a way to do this. Is what I'm trying to do with Heat possible?<br>
><br>
> You should be able to write something like that (not tested):<br>
><br>
> networks:<br>
> if:<br>
> - <my condition><br>
> - network: ctlplane<br>
> fixed_ip: {get_attr: [ControlPlanePort, ip_address]}<br>
> - network: ctlplane<br>
><br>
> The question is how to define your condition. Maybe:<br>
><br>
> conditions:<br>
> fixed_ip_condition:<br>
> not:<br>
> equals:<br>
> - {get_attr: [ControlPlanePort, ip_address]}<br>
> - ''<br>
><br>
> To get back to the problem you stated first.<br>
><br>
><br>
>> Another option I'm exploring is conditionally applying resources. It<br>
>> appears that would require duplicating the entire TripleO::Server stanza<br>
>> in *-role.yaml so that there is one that uses fixed_ip and one that does<br>
>> not. Which one is applied would be based on a condition that tested<br>
>> whether fixed_ip was blank or not. The downside of that is that it would<br>
>> make the role definition confusing because there would be a large<br>
>> resource that was implemented twice, with only one line difference<br>
>> between them.<br>
><br>
> You can define properties with conditions, so you shouldn't need to<br>
> rewrite everything.<br>
><br>
<br>
</div></div>Thomas,<br>
<br>
Thanks, I will try your suggestions and that should get me closer.<br>
<br>
The full error log is available here:<br>
<a href="http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/console.html" rel="noreferrer" target="_blank">http://logs.openstack.org/78/<wbr>413278/11/check-tripleo/gate-<wbr>tripleo-ci-centos-7-ovb-<wbr>updates/8d91762/console.html</a><br>
<br></blockquote><div>We do an interface_detach/attach when a port is replaced. <br>It seems to be failing[1] as this is not implemented for ironic/baremetal driver. I could see a patch[2] to add that functionality though.<br><br>[1] <a href="http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475">http://logs.openstack.org/78/413278/11/check-tripleo/gate-tripleo-ci-centos-7-ovb-updates/8d91762/logs/undercloud/var/log/nova/nova-compute.txt.gz#_2017-04-12_00_26_15_475</a><br><br>[2] <a href="https://review.openstack.org/#/c/419975/">https://review.openstack.org/#/c/419975/</a><br><br></div><div>We retry a few times to check whether the detach/attach is complete(it's an async operation in nova and takes time), so the cryptic error below is coming from tenacity library which fails after the configured number of attempts.<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Here are the errors I am getting:<br>
<br>
2017-04-12 00:26:34.436655 | 2017-04-12 00:26:29Z<br>
[overcloud-CephStorage-<wbr>bkucn6ign34i-0-2yq2jbtwuu7k.<wbr>CephStorage]:<br>
UPDATE_FAILED RetryError: resources.CephStorage: RetryError[<Future at<br>
0xdd62550 state=finished returned bool>]<br>
2017-04-12 00:26:34.436808 | 2017-04-12 00:26:29Z<br>
[overcloud-CephStorage-<wbr>bkucn6ign34i-0-2yq2jbtwuu7k]: UPDATE_FAILED<br>
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550<br>
state=finished returned bool>]<br>
2017-04-12 00:26:34.436903 | 2017-04-12 00:26:29Z<br>
[overcloud-CephStorage-<wbr>bkucn6ign34i.0]: UPDATE_FAILED resources[0]:<br>
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550<br>
state=finished returned bool>]<br>
2017-04-12 00:26:34.436989 | 2017-04-12 00:26:29Z<br>
[overcloud-CephStorage-<wbr>bkucn6ign34i]: UPDATE_FAILED resources[0]:<br>
RetryError: resources.CephStorage: RetryError[<Future at 0xdd62550<br>
state=finished returned bool>]<br>
2017-04-12 00:26:34.437078 | 2017-04-12 00:26:30Z<br>
[overcloud-Controller-<wbr>3lf3jauv4cbc-0-ydowkb3nwsso.<wbr>Controller]:<br>
UPDATE_FAILED RetryError: resources.Controller: RetryError[<Future at<br>
0xdc79b50 state=finished returned bool>]<br>
2017-04-12 00:26:34.437173 | 2017-04-12 00:26:30Z<br>
[overcloud-Controller-<wbr>3lf3jauv4cbc-0-ydowkb3nwsso]: UPDATE_FAILED<br>
RetryError: resources.Controller: RetryError[<Future at 0xdc79b50<br>
state=finished returned bool>]<br>
2017-04-12 00:26:34.437269 | 2017-04-12 00:26:30Z [CephStorage]:<br>
UPDATE_FAILED resources.CephStorage: resources[0]: RetryError:<br>
resources.CephStorage: RetryError[<Future at 0xdd62550 state=finished<br>
returned bool>]<br>
<span class="gmail-im gmail-HOEnZb"><br>
--<br>
Dan Sneddon | Senior Principal Software Engineer<br>
<a href="mailto:dsneddon@redhat.com">dsneddon@redhat.com</a> | <a href="http://redhat.com/openstack" rel="noreferrer" target="_blank">redhat.com/openstack</a><br>
dsneddon:irc | @dxs:twitter<br>
<br>
</span><div class="gmail-HOEnZb"><div class="gmail-h5">______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.<wbr>openstack.org?subject:<wbr>unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature"><div dir="ltr"><div>Regards,</div>Rabi Misra<div><br></div></div></div>
</div></div>