[ironic][tripleo][kayobe][tenks][vbmc] Do you also experience random issues with vbmc?
Dear fellow upstream OpenStackers,
I tagged the main projects that use vbmc intensively because I have a question regarding the stability of vbmc. I have been working on adding baremetal testing in Kolla Ansible using Tenks and thus also VirtualBMC. I am encountering random failures that I am at the moment unable to debug. It seems vbmc likes to reject the "chassis bootdev pxe" ipmitool command from time to time (the command is issued when trying to deploy a host). This seems permanent per case as ironic retries this call several times only to fail. The error message is "Error setting Chassis Boot Parameter 5". I can't see any trace in vbmc logs (with DEBUG enabled). Other ipmitool commands seem to work fine (like periodic power status). I was wondering if any of you also experienced this issue. I am providing a link to an excerpt from the logs. [1] It also shows a raw ipmitool command succeeding only to let the next one fail.
[1] https://paste.ubuntu.com/p/MgHHpDvVGf/
-yoctozepto
On Sun, 6 Mar 2022 at 08:51, Radosław Piliszek radoslaw.piliszek@gmail.com wrote:
Dear fellow upstream OpenStackers,
I tagged the main projects that use vbmc intensively because I have a question regarding the stability of vbmc. I have been working on adding baremetal testing in Kolla Ansible using Tenks and thus also VirtualBMC. I am encountering random failures that I am at the moment unable to debug. It seems vbmc likes to reject the "chassis bootdev pxe" ipmitool command from time to time (the command is issued when trying to deploy a host). This seems permanent per case as ironic retries this call several times only to fail. The error message is "Error setting Chassis Boot Parameter 5". I can't see any trace in vbmc logs (with DEBUG enabled). Other ipmitool commands seem to work fine (like periodic power status). I was wondering if any of you also experienced this issue. I am providing a link to an excerpt from the logs. [1] It also shows a raw ipmitool command succeeding only to let the next one fail.
Radek,
Try this on for size: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829006
Mark
-yoctozepto
On Mon, 7 Mar 2022 at 09:51, Mark Goddard mark@stackhpc.com wrote:
On Sun, 6 Mar 2022 at 08:51, Radosław Piliszek radoslaw.piliszek@gmail.com wrote:
Dear fellow upstream OpenStackers,
I tagged the main projects that use vbmc intensively because I have a question regarding the stability of vbmc. I have been working on adding baremetal testing in Kolla Ansible using Tenks and thus also VirtualBMC. I am encountering random failures that I am at the moment unable to debug. It seems vbmc likes to reject the "chassis bootdev pxe" ipmitool command from time to time (the command is issued when trying to deploy a host). This seems permanent per case as ironic retries this call several times only to fail. The error message is "Error setting Chassis Boot Parameter 5". I can't see any trace in vbmc logs (with DEBUG enabled). Other ipmitool commands seem to work fine (like periodic power status). I was wondering if any of you also experienced this issue. I am providing a link to an excerpt from the logs. [1] It also shows a raw ipmitool command succeeding only to let the next one fail.
Radek,
Try this on for size: https://review.opendev.org/c/openstack/kayobe-config-dev/+/829006
Thanks, Mark! Trying it out now in [2].
[2] https://review.opendev.org/c/openstack/kolla-ansible/+/832204
-yoctozepto
participants (2)
-
Mark Goddard
-
Radosław Piliszek