Looking at the errors here it appears that everything is configured correctly in Ironic (mac addresses etc) because you are getting through the ramdisk boot and deploy successfully.
The error that is of real concern here is "Unauthorized: {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}" which seems to imply that something is wrong with the auth information given to Ironic and its failing to authenticate with Keystone as part of the neutron client. The resource could not be found error is thrown when it can’t find the tenant or user requested in the config file.
Sam
From: "Haomeng, Wang" <wanghaomeng at gmail.com<mailto:wanghaomeng at gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Monday, 11 April 2016 02:45
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [Ironic] Failed to update Neutron port
Hi,
With my experence, for such "2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron [-] Failed to update Neutron port 7fb98457-90e6-43be-a353-f03ea1959912." issue, some cases are that root cause is we did not register baremetal's mac into ironic, so neutron can not bind mac for baremetal dhcp/pxe, and set DHCP BOOT option for port, so run below command if it is missing:
ironic port-create -n $NODE_UUID -a $MAC_ADDRESS
Good luck!
On Fri, Apr 8, 2016 at 10:00 PM, Senthilprabu Shanmugavel <senthilprabu.s at gmail.com<mailto:senthilprabu.s at gmail.com>> wrote:
Hello,
I am deploying an ARMv8 board using Ironic integrated with Openstack Liberty on Ubuntu 12.04 host. I am using fake_pxe driver and doing the power on manually. Deploy image created from disk image creator tool
Nova boot starts the deployment, deployment image is booted, ironic-agent is able to communicate with ironic conductor. After this, scsi connection is established, I can see this in syslog in ironic conductor node
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.620364] scsi host10: iSCSI Initiator over TCP/IP
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.885579] scsi 10:0:0:0: RAID IET Controller 0001 PQ: 0 ANSI: 5
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.886404] scsi 10:0:0:0: Attached scsi generic sg2 type 12
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.887088] scsi 10:0:0:1: Direct-Access IET VIRTUAL-DISK 0001 PQ: 0 ANSI: 5
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.888157] sd 10:0:0:1: Attached scsi generic sg3 type 0
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.888490] sd 10:0:0:1: [sdc] 15240576 512-byte logical blocks: (7.80 GB/7.26 GiB)
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889114] sd 10:0:0:1: [sdc] Write Protect is off
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889118] sd 10:0:0:1: [sdc] Mode Sense: 69 00 10 08
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.889462] sd 10:0:0:1: [sdc] Write cache: enabled, read cache: enabled, supports DPO and FUA
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.894611] sdc: unknown partition table
Apr 8 16:31:35 lionfish-ocp-controller kernel: [177773.896728] sd 10:0:0:1: [sdc] Attached SCSI disk
Apr 8 16:31:35 lionfish-ocp-controller iscsid: Connection4:0 to [target: iqn.2008-10.org.openstack:5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0, portal: 192.168.2.161,3260] through [iface: default] is operational now
On my node console, I see below messages
root at ubuntu:~# SysRq : Emergency Sync
SysRq : Power Off
reboot: Power down
I think this is triggered by scsi driver (also seen in ironic conductor syslog) but node did not reboot due to known problem. So I powered off manually and powered on.
Apr 8 16:31:38 lionfish-ocp-controller kernel: [177776.290404] sd 10:0:0:1: [sdc] Synchronizing SCSI cache
Apr 8 16:31:38 lionfish-ocp-controller iscsid: Connection4:0 to [target: iqn.2008-10.org.openstack:5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0, portal: 192.168.2.161,3260] through [iface: default] is shutdown.
So far looks good. I was expecting user image download via scsi is successful and should be booted after going down.
By then ironic conductor log says deployment failed due to neutron failing to set DHCP BOOT option for port.
2016-04-08 16:31:14.229 31893 INFO ironic.drivers.modules.agent_base_vendor [req-c1e50ceb-1e11-47ea-9260-e30a2f10605d - - - - -] Initial lookup for node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 succeeded, agent is running and waiting for commands
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron [-] Failed to update Neutron port 7fb98457-90e6-43be-a353-f03ea1959912.
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron Traceback (most recent call last):
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/ironic/dhcp/neutron.py", line 126, in update_port_dhcp_opts
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron _build_client(token).update_port(port_id, port_req_body)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 102, in with_params
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron ret = self.function(instance, *args, **kwargs)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 562, in update_port
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron return self.put(self.port_path % (port), body=body)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 302, in put
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron headers=headers, params=params)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 270, in retry_request
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron headers=headers, params=params)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 200, in do_request
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron content_type=self.content_type())
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 158, in do_request
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self.authenticate_and_fetch_endpoint_url()
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 123, in authenticate_and_fetch_endpoint_url
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self.authenticate()
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 237, in authenticate
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron self._authenticate_keystone()
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 218, in _authenticate_keystone
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron raise exceptions.Unauthorized(message=resp_body)
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron Unauthorized: {"error": {"message": "The resource could not be found.", "code": 404, "title": "Not Found"}}
2016-04-08 16:31:38.690 31893 ERROR ironic.dhcp.neutron
2016-04-08 16:31:38.692 31893 ERROR ironic.drivers.modules.iscsi_deploy [-] Deploy failed for instance 8be4838b-c7be-411c-abf4-212b42dde17d. Error: Failed to set DHCP BOOT options for any port on node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0.
2016-04-08 16:31:38.891 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power off.
2016-04-08 16:31:49.208 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power on.
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor [-] Asynchronous exception for node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0: Node failed to get image for deploy. exception: Can not transition from state 'deploy failed' on event 'done' (no defined transition)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor Traceback (most recent call last):
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/agent_base_vendor.py", line 339, in heartbeat
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.continue_deploy(task, **kwargs)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 134, in wrapper
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor return f(*args, **kwargs)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/iscsi_deploy.py", line 983, in continue_deploy
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.reboot_and_finish_deploy(task)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/drivers/modules/agent_base_vendor.py", line 611, in reboot_and_finish_deploy
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor task.process_event('done')
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/conductor/task_manager.py", line 342, in process_event
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor self.fsm.process_event(event, target_state=target_state)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor File "/usr/lib/python2.7/dist-packages/ironic/common/fsm.py", line 44, in wrapper
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor raise excp.InvalidState(six.text_type(e))
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor InvalidState: Can not transition from state 'deploy failed' on event 'done' (no defined transition)
2016-04-08 16:31:49.317 31893 ERROR ironic.drivers.modules.agent_base_vendor
2016-04-08 16:31:49.869 31893 INFO ironic.conductor.utils [-] Successfully set node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 power state to power off.
2016-04-08 16:31:50.012 31893 INFO ironic.conductor.manager [-] Successfully unprovisioned node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 with instance 8be4838b-c7be-411c-abf4-212b42dde17d.
2016-04-08 16:31:50.621 31893 INFO ironic.conductor.manager [-] Cleaning is disabled, node 5044c5ab-a4f5-4ba1-9e3d-01a0cc9bb9e0 has been successfully moved to AVAILABLE state.
Similar issue reported in bug - 1524779
https://bugs.launchpad.net/ironic/+bug/1524779
Can someone comment how can I overcome this scsi error?. Is there any workaround.
Thanks!!!
--
Best Regards,
Senthilprabu Shanmugavel
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe<http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160411/31892b46/attachment.html>