Open Stack

Dell - Internal Use - Confidential
Hi Mike
I do see this problem this week. See logs.


https://stash.opencrowbar.org/logs/52/456952/2/check/dell-hw-tempest-dsvm-ironic-pxe_ipmitool/315bd85/



We were running 5 builds on a one node devstack-cloud and now made it 6 and started seeing this problem. The server must be running out of resources for the VMs.



Regards

Rajini


-----Original Message-----
From: Michael Turek [mailto:mjturek at linux.vnet.ibm.com]
Sent: Friday, April 14, 2017 10:52 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [ironic] [third-party-ci] pkvmci ironic job breakage details

Hey ironic-ers,

So our third party CI job for ironic has been, and remains, broken. I was able to do some investigation today and here's a summary of what we're seeing. I'm hoping someone might know the root of the problem.

For reference, please see this paste and the logs of the job that I was working in:
http://paste.openstack.org/show/606564/
https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/

I've redacted the credentials in the ironic node-show for obvious reasons but rest assured they are properly set. These commands are run while '/opt/stack/new/ironic/devstack/lib/ironic:wait_for_nova_resources' is looping.

Basically, the ironic hypervisor for the node doesn't appear. As well, none of the node's properties make it to the hypervisor stats.

Some more strangeness is that the 'count' value from the 'openstack hypervisor stats show'. Though no hypervisors appear, the count is still 1. Since the run was broken, I decided to delete node-0 (about 3-5 minutes before the run failed) and see if it updated the count. It did.

Does anyone have any clue what might be happening here? Any advice would be appreciated!

Thanks,
mjturek


__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170417/3148ebd6/attachment.html>