<div dir="ltr">Hello,<div><br></div><div>It looks similar on issue, which was discussed here [1] </div><div>I suppose, that the root cause is incorrect using get_attr for your case.</div><div>Probably you got "list" instead of "string".</div><div>F.e. if I do something similar:</div><div><div><br></div><div><br></div><div>outputs: </div><div> rg_1: </div><div> value: {get_attr: [rg_a, rg_a_public_ip]} </div><div> rg_2: </div><div> value: {get_attr: [rg_a, rg_a_public_ip, 0]} </div><div> rg_3: </div><div> value: {get_attr: [rg_a]} </div><div> rg_4: </div><div> value: {get_attr: [rg_a, resource.0.rg_a_public_ip]} </div></div><div><br></div><div>where rg_a is also resource group which uses custom template as resource.</div><div>the custom template has output value rg_a_public_ip.</div><div><br></div><div>The output for it looks like [2]</div><div><br></div><div>So as you can see, that in first case (like it is used in your example), get_attr returns list with one element.</div><div>rg_2 is also wrong, because it takes first symbol from sting with IP address.</div><div>rg_3 - does not work at all (because it's custom template resource)</div><div>the right way is rg_4, which returns IP address string .</div><div><br></div><div>[1] <a href="https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg77526.html">https://www.mail-archive.com/openstack-dev@lists.openstack.org/msg77526.html</a></div><div><br></div><div>[2] <a href="http://paste.openstack.org/show/491587/">http://paste.openstack.org/show/491587/</a></div></div><div class="gmail_extra"><br><div class="gmail_quote">On 23 March 2016 at 14:15, Ma, Wen-Tao (Mike, HP Servers-PSC-BJ) <span dir="ltr"><<a href="mailto:wentao.ma@hpe.com" target="_blank">wentao.ma@hpe.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<p>Hi Sergey,<u></u><u></u></p>
<p class="MsoNormal">Here is our tracked logs. we can notice that kube_master resource can return the output value
<span style="color:red">"kube_master_ip": "10.101.58.117" , </span>but It can’t get the kube_master_ip value in kube_minions of
<b>kubecluster-fedora-ironic.yaml.</b><u></u><u></u></p>
<p class="MsoNormal">I found about this heat template composition configuration at
<a href="https://ask.openstack.org/en/question/56988/get-outputs-from-nested-stack/" target="_blank">
https://ask.openstack.org/en/question/56988/get-outputs-from-nested-stack/</a> . It is same with us.<u></u><u></u></p>
<table border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr style="height:153.15pt">
<td width="1397" valign="top" style="width:1047.7pt;border:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:153.15pt">
<p class="MsoNormal"><b><span style="font-size:12.0pt">#heat resource-list --nested-depth 5 cf0e4e53-e703-4d78-b2e3-90c7081c39fe</span><u></u><u></u></b></p>
<p class="MsoNormal">+-----------------------+--------------------------------------+------------------------------------------------------------------------------------+-----------------+---------------------+-------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal">| resource_name | physical_resource_id | resource_type | resource_status | updated_time | stack_name
|<u></u><u></u></p>
<p class="MsoNormal">+-----------------------+--------------------------------------+------------------------------------------------------------------------------------+-----------------+---------------------+-------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal">| kube_master | <span style="color:red">65d68ca7-6629-4203-b40b-359f53be8c79
</span>| OS::Heat::ResourceGroup | CREATE_COMPLETE | 2016-03-23T18:12:44 | k8sbay-rzqvufyi24q5 |<u></u><u></u></p>
<p class="MsoNormal">| kube_minions | 9a3d3d0c-104e-4887-9961-f4d6b6dc392f | OS::Heat::ResourceGroup |
<span style="color:red">CREATE_FAILED </span>| 2016-03-23T18:12:44 | k8sbay-rzqvufyi24q5 |<u></u><u></u></p>
<p class="MsoNormal">+-----------------------+--------------------------------------+------------------------------------------------------------------------------------+-----------------+---------------------+-------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b><span style="font-size:12.0pt">#heat resource-show 65d68ca7-6629-4203-b40b-359f53be8c79 0<u></u><u></u></span></b></p>
<p class="MsoNormal">+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal">| Property | Value |<u></u><u></u></p>
<p class="MsoNormal">+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal"><span style="color:red">| attributes | { |<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:red">| | "kube_master_external_ip": "10.101.58.117", |<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:red">| | "kube_master_ip": "10.101.58.117" |<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:red">| | } </span>|<u></u><u></u></p>
<p class="MsoNormal">…<u></u><u></u></p>
<p class="MsoNormal">| resource_status | CREATE_COMPLETE |<u></u><u></u></p>
<p class="MsoNormal">+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b><span style="font-size:12.0pt">Here is the three k8s heat yaml file.<u></u><u></u></span></b></p>
<table border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr style="height:219.55pt">
<td width="1404" valign="top" style="width:1052.75pt;border:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt;height:219.55pt">
<p class="MsoNormal"><b>kubecluster-fedora-ironic.yaml<u></u><u></u></b></p>
<p class="MsoNormal">kube_master:<u></u><u></u></p>
<p class="MsoNormal"> type: OS::Heat::ResourceGroup<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> count: 1<u></u><u></u></p>
<p class="MsoNormal"> resource_def:<u></u><u></u></p>
<p class="MsoNormal"> type: kubemaster-fedora-ironic.yaml<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> ssh_key_name: {get_param: ssh_key_name}<u></u><u></u></p>
<p class="MsoNormal"> server_image: {get_param: server_image}<u></u><u></u></p>
<p class="MsoNormal"> …<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">kube_minions:<u></u><u></u></p>
<p class="MsoNormal"> type: OS::Heat::ResourceGroup<u></u><u></u></p>
<p class="MsoNormal"> depends_on:<u></u><u></u></p>
<p class="MsoNormal"> - kube_master<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> count: {get_param: number_of_minions}<u></u><u></u></p>
<p class="MsoNormal"> removal_policies: [{resource_list: {get_param: minions_to_remove}}]<u></u><u></u></p>
<p class="MsoNormal"> resource_def:<u></u><u></u></p>
<p class="MsoNormal"> type: kubeminion-fedora-ironic.yaml<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> ssh_key_name: {get_param: ssh_key_name}<u></u><u></u></p>
<p class="MsoNormal"> server_image: {get_param: server_image}<u></u><u></u></p>
<p class="MsoNormal"> …<u></u><u></u></p>
<p class="MsoNormal"><span style="color:red"> kube_master_ip: {get_attr: [kube_master, kube_master_ip]}
</span><span style="font-family:Wingdings;color:red">ß</span><span style="color:red"> kube-minion get kube_master_ip form kube_master resource.
<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b>kubemaster-fedora-ironic.yaml<u></u><u></u></b></p>
<p class="MsoNormal">resources:<u></u><u></u></p>
<p class="MsoNormal"> kube_master:<u></u><u></u></p>
<p class="MsoNormal"> type: OS::Nova::Server<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> image: {get_param: server_image}<u></u><u></u></p>
<p class="MsoNormal"> flavor: {get_param: master_flavor}<u></u><u></u></p>
<p class="MsoNormal"> key_name: {get_param: ssh_key_name}<u></u><u></u></p>
<p class="MsoNormal"> networks:<u></u><u></u></p>
<p class="MsoNormal"> - network: {get_param: fixed_network}<u></u><u></u></p>
<p class="MsoNormal"> user_data_format: RAW<u></u><u></u></p>
<p class="MsoNormal"> user_data: {get_resource: kube_master_init}<u></u><u></u></p>
<p class="MsoNormal"><u></u><u></u></p>
<p class="MsoNormal">outputs:<u></u><u></u></p>
<p class="MsoNormal"> <span style="color:red">kube_master_ip:<u></u><u></u></span></p>
<p class="MsoNormal" style="text-indent:9.0pt"><span style="color:red">value: {get_attr: [kube_master, networks, sharenet1, 0]}
</span><span style="font-family:Wingdings;color:red">ß</span><span style="color:red"> The output value of kube_master_ip. It works.<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b>kubeminion-fedora-ironic.yaml<u></u><u></u></b></p>
<p class="MsoNormal">parameters:<u></u><u></u></p>
<p class="MsoNormal"><span style="color:red"> kube_master_ip:<u></u><u></u></span></p>
<p class="MsoNormal"> type: string<u></u><u></u></p>
<p class="MsoNormal" style="text-indent:9.0pt">description: IP address of the Kubernetes master server.<u></u><u></u></p>
<p class="MsoNormal" style="text-indent:9.0pt"><u></u> <u></u></p>
<p class="MsoNormal">resources:<u></u><u></u></p>
<p class="MsoNormal"> write_heat_params:<u></u><u></u></p>
<p class="MsoNormal"> type: OS::Heat::SoftwareConfig<u></u><u></u></p>
<p class="MsoNormal"> properties:<u></u><u></u></p>
<p class="MsoNormal"> group: ungrouped<u></u><u></u></p>
<p class="MsoNormal"> config:<u></u><u></u></p>
<p class="MsoNormal"> str_replace:<u></u><u></u></p>
<p class="MsoNormal"> template: {get_file: fragments/write-heat-params.yaml}<u></u><u></u></p>
<p class="MsoNormal"> params:<u></u><u></u></p>
<p class="MsoNormal"> $KUBE_ALLOW_PRIV: {get_param: kube_allow_priv}<u></u><u></u></p>
<p class="MsoNormal"><span style="color:red"> $KUBE_MASTER_IP: {get_param: kube_master_ip}
<u></u><u></u></span></p>
<p class="MsoNormal"> $WAIT_HANDLE: {get_resource: minion_wait_handle}<u></u><u></u></p>
<p class="MsoNormal"><b><u></u> <u></u></b></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><b><u></u> <u></u></b></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><b>Here is the traced log in heat engine. It can’t get the kube_master_ip value at resource kube_minions of kubecluster-fedora-ironic.yaml from the kubemaster-fedora-ironic.yaml. About the detail logs, pls refer to attachment.<u></u><u></u></b></p>
<table border="1" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border:none">
<tbody>
<tr>
<td width="1541" valign="top" style="width:1156.1pt;border:solid windowtext 1.0pt;padding:0in 5.4pt 0in 5.4pt">
<p class="MsoNormal"> "Current bay status is: CREATE_IN_PROGRESS", <u></u><u></u></p>
<p class="MsoNormal"> "Current bay status is: CREATE_FAILED", <u></u><u></u></p>
<p class="MsoNormal"> "Magnum bay creating was failed.", <u></u><u></u></p>
<p class="MsoNormal"> "+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - - - + - - - - - - - - - - - - - - - - - - - +",
<u></u><u></u></p>
<p class="MsoNormal"> "| Name | IP | Server_model | ",
<u></u><u></u></p>
<p class="MsoNormal"> "+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - - - + - - - - - - - - - - - - - - - - - - - +",
<u></u><u></u></p>
<p class="MsoNormal"> "| k8-je335tlkev-0-qauies6sqxce-kube_master-7sd5ajc5eckr | 10.101.58.117 | ProLiant DL360 Gen9 |",
<u></u><u></u></p>
<p class="MsoNormal"> "+ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + - - - - - - - - - - - - - + - - - - - - - - - - - - - - - - - - - +",
<u></u><u></u></p>
<p class="MsoNormal"> ""<u></u><u></u></p>
<p><b><span style="font-size:12.0pt">h-eng.log<u></u><u></u></span></b></p>
<p class="MsoNormal">2016-03-22 10:09:20.691 [00;36mINFO heat.engine.stack [[00;36m-[00;36m] [01;35m[00;36mStack UPDATE COMPLETE (k8sbay-nqvfa6muxtlr-kube_master-2eje335tlkev): Stack UPDATE completed successfully[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:20.770 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: a5f220e0b3214c599f3df4ae2bd2ad65 reply to reply_62852b98e7234d12a839c75da108336c[00m [00;33mfrom (pid=16742)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:20.802 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-48435263-67d8-4cca-8d10-57cc2c24dd3e [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: a5f220e0b3214c599f3df4ae2bd2ad65 reply queue: reply_62852b98e7234d12a839c75da108336c[00m
[00;33mfrom (pid=16742) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:20.835 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: 6638fcf42f634778a57fbcb38f61a646 reply to reply_62852b98e7234d12a839c75da108336c[00m [00;33mfrom (pid=16747)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:20.867 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-dcf69ca8-cd48-4001-9aa3-ee951b4680fd [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: 6638fcf42f634778a57fbcb38f61a646 reply queue: reply_62852b98e7234d12a839c75da108336c[00m
[00;33mfrom (pid=16747) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.572 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask stack_task from Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06] running[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.573 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask resource_action running[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.574 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _run_to_completion from ResourceGroup "kube_master" Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06] running[00m [00;33mfrom
(pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.620 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[<span style="color:red">00;32mTask _run_to_completion from ResourceGroup "kube_master" Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06]
complete</span>[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:220[00m
<b><span style="font-family:Wingdings;color:red">ß</span><span style="color:red">[kube_master stack task has completed]</span></b><u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.675 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask resource_action complete[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:220[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.676 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask resource_action starting[00m [00;33mfrom (pid=16747) start /opt/stack/heat/heat/engine/scheduler.py:186[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.676 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask resource_action running[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.680 [00;36mINFO heat.engine.resource [[00;36m-[00;36m] [01;35m[00;<span style="color:red">36mcreating ResourceGroup "kube_minions" Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06</span>][00m <b><span style="color:red"><--[It
is starting the kube_minion statck]<u></u><u></u></span></b></p>
<p class="MsoNormal">2016-03-22 10:09:21.771 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: d933176f9f504b74aa3bb7e38637889c reply to reply_898946fd3eca47669101399e72da7c79[00m [00;33mfrom (pid=16745)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.803 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-1d9f8484-b9fa-4f44-bb17-510f98bfefb2 [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: d933176f9f504b74aa3bb7e38637889c reply queue: reply_898946fd3eca47669101399e72da7c79[00m
[00;33mfrom (pid=16745) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.833 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: abaed3e8f8ce48c2b5c9c64a68d5cda3 reply to reply_898946fd3eca47669101399e72da7c79[00m [00;33mfrom (pid=16746)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:21.864 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-c2edb990-9fe3-49de-9ffa-aec2eaa1d991 [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: abaed3e8f8ce48c2b5c9c64a68d5cda3 reply queue: reply_898946fd3eca47669101399e72da7c79[00m
[00;33mfrom (pid=16746) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.229 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _run_to_completion from ResourceGroup "kube_minions" Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06] starting[00m [00;33mfrom
(pid=16747) start /opt/stack/heat/heat/engine/scheduler.py:186[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.229 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _run_to_completion from ResourceGroup "kube_minions" Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06] running[00m [00;33mfrom
(pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.244 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mCALL msg_id: 5ad18e4645fc49c59b6f7be3025d5b21 exchange 'heat' topic 'engine'[00m [00;33mfrom (pid=16747) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:448[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.253 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: 5ad18e4645fc49c59b6f7be3025d5b21 reply to reply_4ec178bd97a64d5a8680e191df25d840[00m [00;33mfrom (pid=16749)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.259 [00;36mINFO heat.engine.service [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;36m] [01;35m[00;36mCreating stack k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.285 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m<heat.engine.hot.template.HOTemplate20130523 object at 0x7fa55ac68310>[00m [00;33mfrom
(pid=16749) __init__ /opt/stack/heat/heat/engine/parameter_groups.py:30[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.285 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m<heat.engine.hot.parameters.HOTParameters object at 0x7fa55ad51750>[00m [00;33mfrom
(pid=16749) __init__ /opt/stack/heat/heat/engine/parameter_groups.py:31[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.285 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32mValidating Parameter Groups.[00m [00;33mfrom (pid=16749) validate /opt/stack/heat/heat/engine/parameter_groups.py:43[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.286 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m['OS::project_id', 'OS::stack_id'][00m [00;33mfrom (pid=16749) validate /opt/stack/heat/heat/engine/parameter_groups.py:44[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.328 [00;32mDEBUG heat.engine.stack_lock [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32mEngine b3a899bb-a1b0-4537-afff-eaaee66b5fee acquired lock on stack c79e96e0-cc93-4e6d-a980-3f3cdbe3cc5e[00m
[00;33mfrom (pid=16749) acquire /opt/stack/heat/heat/engine/stack_lock.py:68[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.331 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask stack_task from Stack "k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo" [c79e96e0-cc93-4e6d-a980-3f3cdbe3cc5e] starting[00m [00;33mfrom (pid=16749)
start /opt/stack/heat/heat/engine/scheduler.py:186[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.332 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask stack_task from Stack "k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo" [c79e96e0-cc93-4e6d-a980-3f3cdbe3cc5e] running[00m [00;33mfrom (pid=16749)
step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.345 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: 5ad18e4645fc49c59b6f7be3025d5b21 reply queue: reply_4ec178bd97a64d5a8680e191df25d840[00m
[00;33mfrom (pid=16749) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.351 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived reply msg_id: 5ad18e4645fc49c59b6f7be3025d5b21[00m [00;33mfrom (pid=16747) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:294[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.375 [00;36mINFO heat.engine.stack [[00;36m-[00;36m] [01;35m[00;36mStack CREATE IN_PROGRESS (k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo): Stack CREATE started[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.387 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask stack_task from Stack "k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo" [c79e96e0-cc93-4e6d-a980-3f3cdbe3cc5e] complete[00m [00;33mfrom (pid=16749)
step /opt/stack/heat/heat/engine/scheduler.py:220[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.393 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _check_for_completion starting[00m [00;33mfrom (pid=16747) start /opt/stack/heat/heat/engine/scheduler.py:186[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.394 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _check_for_completion running[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:214[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.424 [00;36mINFO heat.engine.stack [[00;36m-[00;36m] [01;35m[00;36mStack CREATE COMPLETE (k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo): Stack CREATE completed successfully[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.452 [00;32mDEBUG heat.engine.scheduler [[00;36m-[00;32m] [01;35m[00;32mTask _check_for_completion complete[00m [00;33mfrom (pid=16747) step /opt/stack/heat/heat/engine/scheduler.py:220[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.473 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mCALL msg_id: 4de0fee3770b4b51ab56a78a49806f2f exchange 'heat' topic 'engine'[00m [00;33mfrom (pid=16747) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:448[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.484 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived message msg_id: 4de0fee3770b4b51ab56a78a49806f2f reply to reply_4ec178bd97a64d5a8680e191df25d840[00m [00;33mfrom (pid=16748)
__call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:193[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.514 [00;36mINFO heat.engine.service [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;36m] [01;35m[00;36mUpdating stack k8sbay-nqvfa6muxtlr-kube_minions-l2nmqmnw7xxo[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.522 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m<heat.engine.hot.template.HOTemplate20150430 object at 0x7fa55abcfa10>[00m [00;33mfrom
(pid=16748) __init__ /opt/stack/heat/heat/engine/parameter_groups.py:30[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.523 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m<heat.engine.hot.parameters.HOTParameters object at 0x7fa55ab9c190>[00m [00;33mfrom
(pid=16748) __init__ /opt/stack/heat/heat/engine/parameter_groups.py:31[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.523 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32mValidating Parameter Groups.[00m [00;33mfrom (pid=16748) validate /opt/stack/heat/heat/engine/parameter_groups.py:43[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.523 [00;32mDEBUG heat.engine.parameter_groups [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32m['OS::project_id', 'OS::stack_id'][00m [00;33mfrom (pid=16748) validate /opt/stack/heat/heat/engine/parameter_groups.py:44[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.524 [00;36mINFO heat.engine.resource [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;36m] [01;35m[00;36mValidating file:///opt/stack/magnum/magnum/templates/kubernetes/kubeminion-fedora-ironic.yaml
"0"[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.525 [00;32mDEBUG heat.engine.stack [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;<b>32m<span style="color:red">Property error: resources[0].properties.kube_master_ip: Value must
be a string[00m [00;33mfrom (pid=16748) validate /opt/stack/heat/heat/engine/stack.py:695[00m
</span></b><b><span style="font-family:Wingdings;color:red">ß</span><span style="color:red">[ It can get the kube_master_ip value in kube_minion]</span></b><u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.526 [00;32mDEBUG oslo_messaging.rpc.dispatcher [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32mExpected exception during message handling (Property error: resources[0].properties.kube_master_ip:
Value must be a string)[00m [00;33mfrom (pid=16748) _dispatch_and_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py:145[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.568 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[01;36mreq-2c3873b4-6d63-4f9a-90b4-94daa3c2388d [00;36mNone admin[00;32m] [01;35m[00;32msending reply msg_id: 4de0fee3770b4b51ab56a78a49806f2f reply queue: reply_4ec178bd97a64d5a8680e191df25d840[00m
[00;33mfrom (pid=16748) _send_reply /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:79[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.571 [00;32mDEBUG oslo_messaging._drivers.amqpdriver [[00;36m-[00;32m] [01;35m[00;32mreceived reply msg_id: 4de0fee3770b4b51ab56a78a49806f2f[00m [00;33mfrom (pid=16747) __call__ /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:294[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.572 [01;31mERROR heat.engine.resources.stack_resource [[00;36m-[01;31m] [01;35m[01;31mupdate_stack[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00mTraceback (most recent call last):<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/engine/resources/stack_resource.py", line 432, in update_with_template<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m args)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/rpc/client.py", line 263, in update_stack<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m args=args))<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/rpc/client.py", line 60, in call<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m return client.call(ctxt, method, **kwargs)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 403, in call<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m return self.prepare().call(ctxt, method, **kwargs)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 158, in call<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m retry=self.retry)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/transport.py", line 90, in _send<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m timeout=timeout, retry=retry)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 464, in send<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m retry=retry)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 455, in _send<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m raise result<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00mStackValidationFailed_Remote: Property error: resources[0].properties.kube_master_ip: Value must be a string<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00mTraceback (most recent call last):<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/common/context.py", line 296, in wrapped<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m return func(self, ctx, *args, **kwargs)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/engine/service.py", line 865, in update_stack<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m cnxt, current_stack, template, params, files, args)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/engine/service.py", line 830, in _prepare_stack_updates<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m updated_stack.validate()<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/usr/local/lib/python2.7/dist-packages/osprofiler/profiler.py", line 105, in wrapper<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m return f(*args, **kwargs)<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m File "/opt/stack/heat/heat/engine/stack.py", line 696, in validate<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m raise ex<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00mStackValidationFailed: Property error: resources[0].properties.kube_master_ip: Value must be a string<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">[01;31m2016-03-22 10:09:22.572 TRACE heat.engine.resources.stack_resource [01;35m[00m<u></u><u></u></p>
<p class="MsoNormal">2016-03-22 10:09:22.601 [00;36mINFO heat.engine.resource [[00;36m-[00;36m] [01;35m[00;36mCREATE: ResourceGroup "kube_minions" [c79e96e0-cc93-4e6d-a980-3f3cdbe3cc5e] Stack "k8sbay-nqvfa6muxtlr" [aebd9eb5-79aa-4005-8eab-3e33c9350f06][00m<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Thanks && Regards<u></u><u></u></p>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span style="font-size:8.0pt;font-family:"Futura Bk",sans-serif;color:#595959">Mike Ma<u></u><u></u></span></p>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span style="font-size:8.0pt;font-family:"Futura Bk",sans-serif;color:#595959">HP Servers Core Platform Software China
<u></u><u></u></span></p>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span style="font-size:8.0pt;font-family:"Futura Hv",sans-serif;color:#595959">Mobile</span><span style="font-size:8.0pt;font-family:"Futura Bk",sans-serif;color:#595959"> +86 18610248322<u></u><u></u></span></p>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span style="font-size:8.0pt;font-family:"Futura Hv",sans-serif;color:#595959">Email</span><span style="font-size:8.0pt;font-family:"Futura Bk",sans-serif;color:#595959">
<u><a href="mailto:wentao.ma@hp.com" target="_blank">wentao.ma@hp.com</a></u><u></u><u></u></span></p>
<p class="MsoNormal" style="text-align:justify;text-justify:inter-ideograph"><span style="font-size:8.0pt;font-family:"Futura Bk",sans-serif;color:#595959"><u></u> <u></u></span></p>
<p class="MsoNormal"><img border="0" width="35" height="32" src="cid:image001.png@01D18535.2E2591B0" alt="cid:image001.png@01CEE093.A27209C0"><span style="color:#1f497d"><u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">Regards,<div>Sergey.</div></div></div>
</div>