<div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr">Thanks for your suggestion, <div><br></div><div>here are the details, 1) ipmitool with password, </div><div> 2) node show details </div><div> 3) node validate</div><div> 4) driver list</div><div><br></div><div><div><b>(undercloud) [stack@n001 ~]$ ipmitool -I lanplus -H 10.0.40.6 -L ADMINISTRATOR -p 623 -U admin -R 1 -N 5 -P *** lan print 1</b></div><div>Set in Progress : Set Complete</div><div>Auth Type Support : NONE MD2 MD5 PASSWORD</div><div>Auth Type Enable : Callback : MD2 MD5 PASSWORD</div><div> : User : MD2 MD5 PASSWORD</div><div> : Operator : MD2 MD5 PASSWORD</div><div> : Admin : MD2 MD5 PASSWORD</div><div> : OEM : MD2 MD5 PASSWORD</div><div>IP Address Source : Static Address</div><div>IP Address : 10.0.40.6</div><div>Subnet Mask : 255.255.0.0</div><div>MAC Address : 0c:c4:7a:3c:c0:b9</div><div>SNMP Community String : public</div><div>IP Header : TTL=0x00 Flags=0x00 Precedence=0x00 TOS=0x00</div><div>BMC ARP Control : ARP Responses Enabled, Gratuitous ARP Disabled</div><div>Default Gateway IP : 0.0.0.0</div><div>Default Gateway MAC : 00:00:00:00:00:00</div><div>Backup Gateway IP : 0.0.0.0</div><div>Backup Gateway MAC : 00:00:00:00:00:00</div><div>802.1q VLAN ID : Disabled</div><div>802.1q VLAN Priority : 0</div><div>RMCP+ Cipher Suites : 1,2,3,6,7,8,11,12</div><div>Cipher Suite Priv Max : XaaaXXaaaXXaaXX</div><div> : X=Cipher Suite Unused</div><div> : c=CALLBACK</div><div> : u=USER</div><div> : o=OPERATOR</div><div> : a=ADMIN</div><div> : O=OEM</div><div>Bad Password Threshold : Not Available</div></div><div><br></div><div><br></div><div><br></div><div><div><b>(undercloud) [stack@n001 ~]$ openstack baremetal node show 23a17f0f-d683-4f90-8696-f947485900f9</b></div><div>/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!</div><div> RequestsDependencyWarning)</div><div>+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div><div>| Field | Value |</div><div>+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div><div>| allocation_uuid | None |</div><div>| automated_clean | None |</div><div>| bios_interface | no-bios |</div><div>| boot_interface | ipxe |</div><div>| chassis_uuid | None |</div><div>| clean_step | {} |</div><div>| conductor | n001.localdomain |</div><div>| conductor_group | |</div><div>| console_enabled | False |</div><div>| console_interface | ipmitool-socat |</div><div>| created_at | 2021-04-26T08:22:49+00:00 |</div><div>| deploy_interface | direct |</div><div>| deploy_step | {} |</div><div>| description | None |</div><div>| driver | ipmi |</div><div>| driver_info | {'deploy_kernel': 'file:///var/lib/ironic/httpboot/agent.kernel', 'rescue_kernel': 'file:///var/lib/ironic/httpboot/agent.kernel', 'deploy_ramdisk': 'file:///var/lib/ironic/httpboot/agent.ramdisk', 'rescue_ramdisk': 'file:///var/lib/ironic/httpboot/agent.ramdisk', 'ipmi_address': '10.0.1.11', 'ipmi_username': 'admin', 'ipmi_password': '******', 'ipmi_port': 623} |</div><div>| driver_internal_info | {} |</div><div>| extra | {} |</div><div>| fault | None |</div><div>| inspect_interface | inspector |</div><div>| inspection_finished_at | None |</div><div>| inspection_started_at | None |</div><div>| instance_info | {} |</div><div>| instance_uuid | None |</div><div>| last_error | None |</div><div>| lessee | None |</div><div>| maintenance | False |</div><div>| maintenance_reason | None |</div><div>| management_interface | ipmitool |</div><div>| name | None |</div><div>| network_data | {} |</div><div>| network_interface | flat |</div><div>| owner | None |</div><div>| power_interface | ipmitool |</div><div>| power_state | power off |</div><div>| properties | {'capabilities': 'boot_option:local', 'vendor': 'supermicro'} |</div><div>| protected | False |</div><div>| protected_reason | None |</div><div>| provision_state | manageable |</div><div>| provision_updated_at | 2021-04-26T08:22:55+00:00 |</div><div>| raid_config | {} |</div><div>| raid_interface | no-raid |</div><div>| rescue_interface | agent |</div><div>| reservation | None |</div><div>| resource_class | baremetal |</div><div>| retired | False |</div><div>| retired_reason | None |</div><div>| storage_interface | noop |</div><div>| target_power_state | None |</div><div>| target_provision_state | None |</div><div>| target_raid_config | {} |</div><div>| traits | [] |</div><div>| updated_at | 2021-04-26T08:35:27+00:00 |</div><div>| uuid | 23a17f0f-d683-4f90-8696-f947485900f9 |</div><div>| vendor_interface | ipmitool |</div><div>+------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div></div><div>and here is the validate output, i could see the pxe assign ipaddress to node and loaded the operating system. </div><div><br></div><div><div><b>(undercloud) [stack@n001 ~]$ openstack baremetal node validate 23a17f0f-d683-4f90-8696-f947485900f9</b></div><div>/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!</div><div> RequestsDependencyWarning)</div><div>+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div><div>| Interface | Result | Reason |</div><div>+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div><div>| bios | False | Driver ipmi does not support bios (disabled or not implemented). |</div><div>| boot | False | Cannot validate image information for node 23a17f0f-d683-4f90-8696-f947485900f9 because one or more parameters are missing from its instance_info and insufficent information is present to boot from a remote volume. Missing are: ['image_source', 'kernel', 'ramdisk'] |</div><div>| console | False | Either missing 'ipmi_terminal_port' parameter in node's driver_info or [console]port_range is not configured |</div><div>| deploy | False | Cannot validate image information for node 23a17f0f-d683-4f90-8696-f947485900f9 because one or more parameters are missing from its instance_info and insufficent information is present to boot from a remote volume. Missing are: ['image_source', 'kernel', 'ramdisk'] |</div><div>| inspect | True | |</div><div>| management | True | |</div><div>| network | True | |</div><div>| power | True | |</div><div>| raid | False | Driver ipmi does not support raid (disabled or not implemented). |</div><div>| rescue | False | Cannot validate image information for node 23a17f0f-d683-4f90-8696-f947485900f9 because one or more parameters are missing from its instance_info and insufficent information is present to boot from a remote volume. Missing are: ['image_source', 'kernel', 'ramdisk'] |</div><div>| storage | True | |</div><div>+------------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+</div></div><div><br></div><div><div><b>(undercloud) [stack@n001 ~]$ openstack baremetal driver list</b></div><div>/usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!</div><div> RequestsDependencyWarning)</div><div>+---------------------+------------------+</div><div>| Supported driver(s) | Active host(s) |</div><div>+---------------------+------------------+</div><div>| idrac | n001.localdomain |</div><div>| ilo | n001.localdomain |</div><div>| ipmi | n001.localdomain |</div><div>| redfish | n001.localdomain |</div><div>+---------------------+------------------+</div></div><div><br></div><div>-Vinesh</div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 26, 2021 at 6:59 PM Julia Kreger <<a href="mailto:juliaashleykreger@gmail.com">juliaashleykreger@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">Greetings,<br>
<br>
In all likelihood, the credentials are wrong for the baremetal node<br>
and the lock is being held by the conductor who is still trying to<br>
record the power state. The lock is an intentional behavior clients<br>
should retry if they encounter the lock. This is because BMC's often<br>
cannot handle concurrent requests.<br>
<br>
I would first manually verify:<br>
<br>
* That the nodes are not in maintenance state (openstack baremetal<br>
node show). The node last_error field may have a hint or indication to<br>
the actual error, but visit the next two bullet points.<br>
* That a power state of on or off has been recorded. If it has not<br>
been recorded, the supplied credentials or or access is correct.<br>
* If you're sure about the credentials, verify basic connectivity to<br>
the BMC address. Some BMCs are very particular about *how* the<br>
networking is configured, specifically to help limit attacks from the<br>
network itself.<br>
<br>
-Julia<br>
<br>
<br>
On Wed, Apr 21, 2021 at 7:25 PM Vinesh N <<a href="mailto:vineshnellaiappan@gmail.com" target="_blank">vineshnellaiappan@gmail.com</a>> wrote:<br>
><br>
> hi,<br>
> i am facing an issue while introspect the bare metal nodes,<br>
><br>
> error message<br>
> "4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf | 2021-04-22T01:41:32 | 2021-04-22T01:41:35 | Failed to set boot device to PXE: Failed to set boot device for node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf: Client Error for url: <a href="http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/management/boot_device" rel="noreferrer" target="_blank">http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/management/boot_device</a>, Node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf is locked by host undercloud.localdomain, please retry after the current operation is completed"<br>
><br>
><br>
> (undercloud) [stack@undercloud ~]$ cat /etc/*release<br>
> CentOS Linux release 8.3.2011<br>
><br>
> ussuri version<br>
><br>
> (undercloud) [stack@undercloud ~]$ openstack image list<br>
> /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!<br>
> RequestsDependencyWarning)<br>
> +--------------------------------------+------------------------+--------+<br>
> | ID | Name | Status |<br>
> +--------------------------------------+------------------------+--------+<br>
> | 8ddcd168-cc18-4ce2-97c5-c3502ac471a4 | overcloud-full | active |<br>
> | 8d9cfac9-400b-4570-b0b1-baeb175b16c4 | overcloud-full-initrd | active |<br>
> | c561f1d5-41ae-4599-81ea-de2c1e74eae7 | overcloud-full-vmlinuz | active |<br>
> +--------------------------------------+------------------------+--------+<br>
><br>
> Using the command to introspect the node, it was able to discover the node and I could provision the node boot via pxe, and load the image on the node. I could see the login prompt on the server, after some time of provision shut the node down.<br>
><br>
> openstack overcloud node discover --range 10.0.40.5 --credentials admin:XXXX --introspect --provide<br>
><br>
> /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!<br>
> RequestsDependencyWarning)<br>
> Successfully probed node IP 10.0.40.5<br>
> Successfully registered node UUID 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf<br>
> /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!<br>
> RequestsDependencyWarning)<br>
><br>
> PLAY [Baremetal Introspection for multiple Ironic Nodes] ***********************<br>
> 2021-04-22 07:04:28.978299 | 002590fe-0d22-76eb-1a70-000000000008 | TASK | Check for required inputs<br>
> 2021-04-22 07:04:29.002729 | 002590fe-0d22-76eb-1a70-000000000008 | SKIPPED | Check for required inputs | localhost | item=node_uuids<br>
> 2021-04-22 07:04:29.004468 | 002590fe-0d22-76eb-1a70-000000000008 | TIMING | Check for required inputs | localhost | 0:00:00.069134 | 0.0<br>
> ....<br>
> ....<br>
> ....<br>
><br>
> 2021-04-22 07:11:43.261714 | 002590fe-0d22-76eb-1a70-000000000016 | TASK | Nodes that failed introspection<br>
> 2021-04-22 07:11:43.296417 | 002590fe-0d22-76eb-1a70-000000000016 | FATAL | Nodes that failed introspection | localhost | error={<br>
> "msg": " 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf"<br>
> }<br>
> 2021-04-22 07:11:43.297359 | 002590fe-0d22-76eb-1a70-000000000016 | TIMING | Nodes that failed introspection | localhost | 0:07:14.362025 | 0.03s<br>
><br>
> NO MORE HOSTS LEFT *************************************************************<br>
><br>
> PLAY RECAP *********************************************************************<br>
> localhost : ok=4 changed=1 unreachable=0 failed=1 skipped=5 rescued=0 ignored=0<br>
> 2021-04-22 07:11:43.301553 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.302101 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Total Tasks: 10 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.302609 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Elapsed Time: 0:07:14.367265 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.303162 | UUID | Info | Host | Task Name | Run Time<br>
> 2021-04-22 07:11:43.303740 | 002590fe-0d22-76eb-1a70-000000000014 | SUMMARY | localhost | Start baremetal introspection | 434.03s<br>
> 2021-04-22 07:11:43.304248 | 002590fe-0d22-76eb-1a70-000000000015 | SUMMARY | localhost | Nodes that passed introspection | 0.04s<br>
> 2021-04-22 07:11:43.304814 | 002590fe-0d22-76eb-1a70-000000000016 | SUMMARY | localhost | Nodes that failed introspection | 0.03s<br>
> 2021-04-22 07:11:43.305341 | 002590fe-0d22-76eb-1a70-000000000008 | SUMMARY | localhost | Check for required inputs | 0.03s<br>
> 2021-04-22 07:11:43.305854 | 002590fe-0d22-76eb-1a70-00000000000a | SUMMARY | localhost | Set node_uuids_intro fact | 0.02s<br>
> 2021-04-22 07:11:43.306397 | 002590fe-0d22-76eb-1a70-000000000010 | SUMMARY | localhost | Check if validation enabled | 0.02s<br>
> 2021-04-22 07:11:43.306904 | 002590fe-0d22-76eb-1a70-000000000012 | SUMMARY | localhost | Fail if validations are disabled | 0.02s<br>
> 2021-04-22 07:11:43.307379 | 002590fe-0d22-76eb-1a70-00000000000e | SUMMARY | localhost | Set concurrency fact | 0.02s<br>
> 2021-04-22 07:11:43.307913 | 002590fe-0d22-76eb-1a70-00000000000c | SUMMARY | localhost | Notice | 0.02s<br>
> 2021-04-22 07:11:43.308417 | 002590fe-0d22-76eb-1a70-000000000011 | SUMMARY | localhost | Run Validations | 0.02s<br>
> 2021-04-22 07:11:43.308926 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ End Summary Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.309423 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ State Information ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.310021 | ~~~~~~~~~~~~~~~~~~ Number of nodes which did not deploy successfully: 1 ~~~~~~~~~~~~~~~~~<br>
> 2021-04-22 07:11:43.310545 | The following node(s) had failures: localhost<br>
> 2021-04-22 07:11:43.311080 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br>
> Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2<br>
> Exception occured while running the command<br>
> Traceback (most recent call last):<br>
> File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run<br>
> super(Command, self).run(parsed_args)<br>
> File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 41, in run<br>
> return super(Command, self).run(parsed_args)<br>
> File "/usr/lib/python3.6/site-packages/cliff/command.py", line 187, in run<br>
> return_code = self.take_action(parsed_args) or 0<br>
> File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_node.py", line 462, in take_action<br>
> retry_timeout=parsed_args.retry_timeout,<br>
> File "/usr/lib/python3.6/site-packages/tripleoclient/workflows/baremetal.py", line 193, in introspect<br>
> "retry_timeout": retry_timeout,<br>
> File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 728, in run_ansible_playbook<br>
> raise RuntimeError(err_msg)<br>
> RuntimeError: Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2<br>
> Ansible execution failed. playbook: /usr/share/ansible/tripleo-playbooks/cli-baremetal-introspect.yaml, Run Status: failed, Return Code: 2<br>
><br>
><br>
> (undercloud) [stack@undercloud ~]$ openstack baremetal introspection list<br>
> /usr/lib/python3.6/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!<br>
> RequestsDependencyWarning)<br>
> +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<br>
> | UUID | Started at | Finished at | Error |<br>
> +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<br>
> | 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf | 2021-04-22T01:41:32 | 2021-04-22T01:41:35 | Failed to set boot device to PXE: Failed to set boot device for node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf: Client Error for url: <a href="http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/management/boot_device" rel="noreferrer" target="_blank">http://10.0.1.202:6385/v1/nodes/4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf/management/boot_device</a>, Node 4bdd55bd-c4d9-4af0-b619-2e0b7b0107cf is locked by host undercloud.localdomain, please retry after the current operation is completed. |<br>
> | 3d091348-e9c7-4e99-80e3-df72d332d935 | 2021-04-21T12:36:30 | 2021-04-21T12:36:32 | Failed to set boot device to PXE: Failed to set boot device for node 3d091348-e9c7-4e99-80e3-df72d332d935: Client Error for url: <a href="http://10.0.1.202:6385/v1/nodes/3d091348-e9c7-4e99-80e3-df72d332d935/management/boot_device" rel="noreferrer" target="_blank">http://10.0.1.202:6385/v1/nodes/3d091348-e9c7-4e99-80e3-df72d332d935/management/boot_device</a>, Node 3d091348-e9c7-4e99-80e3-df72d332d935 is locked by host undercloud.localdomain, please retry after the current operation is completed. |<br>
> +--------------------------------------+---------------------+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+<br>
</blockquote></div>