<div dir="ltr"><div>Hi,<br><br></div>With my experence, for such "2016-04-08 16:31:38.690 31893 ERROR <span class="">ironic</span>.dhcp.neutron [-] Failed to update Neutron port 7fb98457-90e6-43be-a353-f03ea1959912." issue, some cases are that root cause is we did not register baremetal's mac into ironic, so neutron can not bind mac for baremetal dhcp/pxe, and set DHCP BOOT option for port, so run below command if it is missing:<br><br><pre style="overflow:auto;padding:10px;color:rgb(85,85,85);line-height:1.2em;border:1px solid rgb(198,201,203);font-size:1.1em;margin:1.5em 0px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;text-align:left;text-indent:0px;text-transform:none;word-spacing:0px;background-color:rgb(255,255,255)">ironic port-create -n $NODE_UUID -a $MAC_ADDRESS<br><br></pre>Good luck!<br><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 8, 2016 at 10:00 PM, Senthilprabu Shanmugavel <span dir="ltr"><<a href="mailto:senthilprabu.s@gmail.com" target="_blank">senthilprabu.s@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>I am deploying an ARMv8 board using Ironic integrated with Openstack Liberty on Ubuntu 12.04 host. I am using fake_pxe driver and doing the power on manually. Deploy image created from disk image creator tool</div><div><br></div><div>Nova boot starts the deployment, deployment image is booted, ironic-agent is able to communicate with ironic conductor. After this, scsi connection is established, I can see this in syslog in ironic conductor node</div><div><br></div><div><br></div><div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.620364] scsi host10: iSCSI Initiator over TCP/IP</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.885579] scsi 10:0:0:0: RAID IET Controller 0001 PQ: 0 ANSI: 5</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.886404] scsi 10:0:0:0: Attached scsi generic sg2 type 12</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.887088] scsi 10:0:0:1: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.888157] sd 10:0:0:1: Attached scsi generic sg3 type 0</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.888490] sd 10:0:0:1: [sdc] 15240576 512-byte logical blocks: (7.80 GB/7.26 GiB)</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889114] sd 10:0:0:1: [sdc] Write Protect is off</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889118] sd 10:0:0:1: [sdc] Mode Sense: 69 00 10 08</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889462] sd 10:0:0:1: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.894611] sdc: unknown partition table</div><div>Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.896728] sd 10:0:0:1: [sdc] Attached SCSI disk</div><div>Apr 8 16:31:35 lionfish-ocp-controller iscsid: Connection4:0 to [target: iqn.2008-10.org.openstack:5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0, portal: 192.168.2.161,3260] through [iface: default] is operational now</div><div><br></div><div>On my node console, I see below messages</div><div><br></div><div><div>root@ubuntu:~# SysRq : Emergency Sync</div><div>SysRq : Power Off</div><div>reboot: Power down</div><div><br></div></div><div>I think this is triggered by scsi driver (also seen in ironic conductor syslog) but node did not reboot due to known problem. So I powered off manually and powered on. </div><div><br></div><div>Apr 8 16:31:38 lionfish-ocp-controller kernel: [177776.290404] sd 10:0:0:1: [sdc] Synchronizing SCSI cache</div><div>Apr 8 16:31:38 lionfish-ocp-controller iscsid: Connection4:0 to [target: iqn.2008-10.org.openstack:5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0, portal: 192.168.2.161,3260] through [iface: default] is shutdown.</div></div><div><br></div><div>So far looks good. I was expecting user image download via scsi is successful and should be booted after going down.</div><div><br></div><div>By then ironic conductor log says deployment failed due to neutron failing to set DHCP BOOT option for port. </div><div><br></div><div><br></div><div><div>2016-04-08 16:31:14.229 31893 INFO ironic.drivers.modules.agent_base_vendor [req-c1e50ceb-1e11-47ea-9260-e30a2f10605d - - - - -] Initial lookup for node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 succeeded, agent is running and waiting for commands</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron [-] Failed to update Neutron port 7fb98457-90e6-43be-a353-f03ea1959912.</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron Traceback (most recent call last):</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/ironic/dhcp/neutron.py", line 126, in update_port_dhcp_opts</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron _build_client(token).update_port(port_id, port_req_body)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 102, in with_params</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron ret = self.function(instance, *args, **kwargs)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 562, in update_port</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron return self.put(self.port_path % (port), body=body)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 302, in put</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron headers=headers, params=params)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 270, in retry_request</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron headers=headers, params=params)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 200, in do_request</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron content_type=self.content_type())</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 158, in do_request</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self.authenticate_and_fetch_endpoint_url()</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 123, in authenticate_and_fetch_endpoint_url</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self.authenticate()</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 237, in authenticate</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self._authenticate_keystone()</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 218, in _authenticate_keystone</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron raise exceptions.Unauthorized(message=resp_body)</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron Unauthorized: {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}</div><div>2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron</div><div>2016-04-08 16:31:38.692 31893 ERROR ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance 8be4838b-c7be-411c-abf4-212b42dde17d. Error: Failed to set DHCP BOOT options for any port on node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0.</div><div><div>2016-04-08 16:31:38.891 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power off.</div><div>2016-04-08 16:31:49.208 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power on.</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor [-] Asynchronous exception for node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0: Node failed to get image for deploy. exception: Can not transition from state 'deploy failed' on event 'done' (no defined transition)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last):</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/agent_base_vendor.py", line 339, in heartbeat</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.continue_deploy(task, **kwargs)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 134, in wrapper</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor return f(*args, **kwargs)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/iscsi_deploy.py", line 983, in continue_deploy</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.reboot_and_finish_deploy(task)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/agent_base_vendor.py", line 611, in reboot_and_finish_deploy</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor task.process_event('done')</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 342, in process_event</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.fsm.process_event(event, target_state=target_state)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/common/fsm.py", line 44, in wrapper</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor raise excp.InvalidState(six.text_type(e))</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor InvalidState: Can not transition from state 'deploy failed' on event 'done' (no defined transition)</div><div>2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor</div><div>2016-04-08 16:31:49.869 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power off.</div><div>2016-04-08 16:31:50.012 31893 INFO ironic.conductor.manager [-] Successfully unprovisioned node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 with instance 8be4838b-c7be-411c-abf4-212b42dde17d.</div><div>2016-04-08 16:31:50.621 31893 INFO ironic.conductor.manager [-] Cleaning is disabled, node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 has been successfully moved to AVAILABLE state.</div></div><div><br></div><div><br></div><div>Similar issue reported in bug - 1524779</div><div><a href="https://bugs.launchpad.net/ironic/+bug/1524779" target="_blank">https://bugs.launchpad.net/ironic/+bug/1524779</a><br></div><div><br></div><div><br></div><div>Can someone comment how can I overcome this scsi error?. Is there any workaround. </div><div><br></div><div>Thanks!!!</div><span class="HOEnZb"><font color="#888888"><div><br></div>-- <br><div><div dir="ltr">Best Regards,<br>Senthilprabu Shanmugavel<br></div></div>
</font></span></div></div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>