[Openstack] [ironic]Failed to boot ironic instance

Li, Chen chen.li at intel.com
Fri Jun 27 12:18:34 UTC 2014


Hi list,

Based on the warning information:
WARNING ironic.common.neutron [-] No VIFs found for node c9caca46-64a7-44d5-b6db-554077b2473a when attempting to update Neutron DHCP BOOT options.
I changed to run command with nic paramter:
nova boot --flavor baremetal --image cirros-0.3.2-x86_64-uec --nic net-id=b957c908-39bc-4de5-b620-addfd5618680 test

Now I can get IP address by command “nova list”:
nova list
+--------------------------------------+------+--------+------------+-------------+------------------+
| ID                                   | Name | Status | Task State | Power State | Networks         |
+--------------------------------------+------+--------+------------+-------------+------------------+
| 6b513e8d-6f1b-41ce-b3b1-84cbc221e632 | test | BUILD  | spawning   | NOSTATE     | private=10.1.0.4 |
+--------------------------------------+------+--------+------------+-------------+------------------+

But still, the instance stays under  spawning   status.

Then I using VNCviewer directly reach the instance to check status:
I get :
       [cid:image001.png at 01CF9244.F6C8CD20]

Looks like it failed to connect to tftp ???

What should I do next to debug this ???

Thanks.
-chen

From: Li, Chen [mailto:chen.li at intel.com]
Sent: Friday, June 27, 2014 3:14 PM
To: Kai Brennenstuhl
Cc: openstack at lists.openstack.org
Subject: Re: [Openstack] [ironic]Failed to boot ironic instance

Hi Kai,

Thanks for the reply.
I have seem your questions.
And, yes, I think we have met almost the same issue.

I think the dnsmasq process of neutron did not assign an ip to baremetal instance is because neutron-server didn’t allocated any IP.
You can see there is a parameter for  dnsmasq process
--dhcp-hostsfile
dnsmasq process would only response to DHCP request matches MAC address exists in file -dhcp-hostsfile (at least this is how it works when there is no ironic involved).

I checked neutron-server and neutron database, no IP allocation happened.
So, there is no way dnsmasq process would response to any DHCP request.


If another DHCP server can work around this issue, then I guess we might try to solve the  warning in ironic-conductor service log first:
WARNING ironic.common.neutron [-] No VIFs found for node c9caca46-64a7-44d5-b6db-554077b2473a when attempting to update Neutron DHCP BOOT options.

Thanks.
-chen

From: Kai Brennenstuhl [mailto:kai.brennenstuhl at gmail.com]
Sent: Friday, June 27, 2014 2:55 PM
To: Li, Chen
Subject: Re: [Openstack] [ironic]Failed to boot ironic instance

Hi Chen,
I think I had exactly the same problem like you [1]. My problem was that the dnsmasq process of neutron did not assign an ip to the nested baremetal instance. You can check with tcpdump if there are DHCP Boot Requests and if they are replied. I have not solved the issue but did a workaround. I was using Ubuntu 14.04 as controller VM. I configured the dhcp server by myself using isc-dhcp-server [2]. The booting baremetal VM received DHCP offers and everything worked fine. I did not configure PXE or tftp (this was done by ironic).

[1] https://ask.openstack.org/en/question/31253/ironic-node-stuck-in-provisioning-state-wait-call-back/
[2] http://wiki.ubuntuusers.de/ISC-DHCPD

Kai


2014-06-27 8:21 GMT+02:00 Li, Chen <chen.li at intel.com<mailto:chen.li at intel.com>>:
Hi list,

I'm trying to install/using ironic.

I have followed steps here :
http://docs.openstack.org/developer/ironic/dev/dev-quickstart.html#deploying-ironic-with-devstack

While, after I run command "nova boot --flavor baremetal --image cirros-0.3.2-x86_64-uec  test1"

My instance stays in BUILD and spawning for a really long time, and become ERROR at end.

nova list
+--------------------------------------+-------+--------+------------+-------------+----------+
| ID                                   | Name  | Status | Task State | Power State | Networks |
+--------------------------------------+-------+--------+------------+-------------+----------+
| bd10abaa-7cb4-492e-9afd-5027303631f1 | test1  | ERROR  | -          | Running     |          |
| c3e4fccb-dc2b-41dc-8270-2cce5e8fe404 | test2 | BUILD  | spawning   | NOSTATE     |          |
| 3dcfabba-ca32-44ad-988a-c22f7308ed7a | test3 | BUILD  | spawning   | NOSTATE     |          |
+--------------------------------------+-------+--------+------------+-------------+----------+

This is my log for compute.log:
http://paste.openstack.org/
Looks like it is waiting ironic node to become ACTIVE.

This is my ironic-conductor service log:
http://paste.openstack.org/show/85043/

No obvious error message can be find, but there is a warning:
WARNING ironic.common.neutron [-] No VIFs found for node c9caca46-64a7-44d5-b6db-554077b2473a when attempting to update Neutron DHCP BOOT options.

Is this warning caused instance can't start ??

I also check neutron-server, no IP has been allocated at all.

Anyone know why this is happened ???

Thanks.
-chen

Other commands output:
ironic node-list
+--------------------------------------+--------------------------------------+-------------+-----------------+-------------+
| uuid                                 | instance_uuid                        | power_state | provision_state | maintenance |
+--------------------------------------+--------------------------------------+-------------+-----------------+-------------+
| c9caca46-64a7-44d5-b6db-554077b2473a | 3dcfabba-ca32-44ad-988a-c22f7308ed7a | power on    | wait call-back  | False       |
| 6d6c29e0-b81d-4ed9-9e2a-675b5221aba9 | bd10abaa-7cb4-492e-9afd-5027303631f1 | power on    | wait call-back  | False       |
| 9604af4b-a61a-4157-8431-e61235de7b92 | c3e4fccb-dc2b-41dc-8270-2cce5e8fe404 | power on    | wait call-back  | False       |
+--------------------------------------+--------------------------------------+-------------+-----------------+-------------+

ironic node-show c9caca46-64a7-44d5-b6db-554077b2473a

+------------------------+--------------------------------------------------------------------------+
| Property               | Value                                                                    |
+------------------------+--------------------------------------------------------------------------+
| instance_uuid          | 3dcfabba-ca32-44ad-988a-c22f7308ed7a                                     |
| target_power_state     | None                                                                     |
| properties             | {u'memory_mb': u'512', u'cpu_arch': u'x86_64', u'local_gb': u'10',       |
|                        | u'cpus': u'1'}                                                           |
| maintenance            | False                                                                    |
| driver_info            | {u'pxe_deploy_ramdisk': u'cfdbb01b-fa26-4583-a29f-f2f0111499a5',         |
|                        | u'ssh_port': u'22', u'ssh_username': u'stack', u'pxe_deploy_kernel':     |
|                        | u'c5920355-e4ea-4c09-b774-98b85474e24f', u'ssh_virt_type': u'virsh',     |
|                        | u'ssh_address': u'192.168.50.1', u'ssh_key_filename':                    |
|                        | u'/opt/stack/data/ironic/ssh_keys/ironic_key'}                           |
| extra                  | {}                                                                       |
| last_error             | None                                                                     |
| created_at             | 2014-06-27T05:47:27+00:00                                                |
| target_provision_state | deploy complete                                                          |
| driver                 | pxe_ssh                                                                  |
| updated_at             | 2014-06-27T06:03:20+00:00                                                |
| instance_info          | {u'ramdisk': u'9ea0ac76-a07c-44db-a9b7-328839d0d358', u'kernel':         |
|                        | u'cecfc044-6fd5-49af-b91c-8ac90b4e7976', u'root_gb': u'10',              |
|                        | u'image_source': u'94ba7f9a-4de0-4b00-b07a-85c5647b11de', u'deploy_key': |
|                        | u'8GV962GHNSIDQYX30N0SVB94MCSAG4GE', u'swap_mb': u'0'}                   |
| chassis_uuid           | 3183c4f1-b7d6-49a7-8214-1c850d468a7e                                     |
| provision_state        | wait call-back                                                           |
| reservation            | None                                                                     |
| power_state            | power on                                                                 |
| console_enabled        | False                                                                    |
| uuid                   | c9caca46-64a7-44d5-b6db-554077b2473a                                     |
+------------------------+--------------------------------------------------------------------------+


nova image-list
+--------------------------------------+---------------------------------+--------+--------+
| ID                                   | Name                            | Status | Server |
+--------------------------------------+---------------------------------+--------+--------+
| 94ba7f9a-4de0-4b00-b07a-85c5647b11de | cirros-0.3.2-x86_64-uec         | ACTIVE |        |
| cecfc044-6fd5-49af-b91c-8ac90b4e7976 | cirros-0.3.2-x86_64-uec-kernel  | ACTIVE |        |
| 9ea0ac76-a07c-44db-a9b7-328839d0d358 | cirros-0.3.2-x86_64-uec-ramdisk | ACTIVE |        |
| cfdbb01b-fa26-4583-a29f-f2f0111499a5 | ir-deploy.initramfs             | ACTIVE |        |
| c5920355-e4ea-4c09-b774-98b85474e24f | ir-deploy.kernel                | ACTIVE |        |
+--------------------------------------+---------------------------------+--------+--------+

_______________________________________________
Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Post to     : openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140627/b3ffc923/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 22893 bytes
Desc: image001.png
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140627/b3ffc923/attachment.png>


More information about the Openstack mailing list