<div dir="ltr">Hey Michael,<div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 14, 2017 at 6:51 PM, Michael Turek <span dir="ltr"><<a href="mailto:mjturek@linux.vnet.ibm.com" target="_blank">mjturek@linux.vnet.ibm.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hey ironic-ers,<br>
<br>
So our third party CI job for ironic has been, and remains, broken. I was able to do some investigation today and here's a summary of what we're seeing. I'm hoping someone might know the root of the problem.<br>
<br>
For reference, please see this paste and the logs of the job that I was working in:<br>
<a href="http://paste.openstack.org/show/606564/" rel="noreferrer" target="_blank">http://paste.openstack.org/sho<wbr>w/606564/</a><br>
<a href="https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/" rel="noreferrer" target="_blank">https://dal05.objectstorage.so<wbr>ftlayer.net/v1/AUTH_3d8e6ecb-<wbr>f597-448c-8ec2-164e9f710dd6/<wbr>pkvmci/ironic/25/454625/10/<wbr>check-ironic/tempest-dsvm-<wbr>ironic-agent_ipmitool/0520958/</a><br>
<br>
I've redacted the credentials in the ironic node-show for obvious reasons but rest assured they are properly set. These commands are run while '/opt/stack/new/ironic/devstac<wbr>k/lib/ironic:wait_for_nova_<wbr>resources' is looping.<br>
<br>
Basically, the ironic hypervisor for the node doesn't appear. As well, none of the node's properties make it to the hypervisor stats.<br>
<br>
Some more strangeness is that the 'count' value from the 'openstack hypervisor stats show'. Though no hypervisors appear, the count is still 1. Since the run was broken, I decided to delete node-0 (about 3-5 minutes before the run failed) and see if it updated the count. It did.<br>
<br>
Does anyone have any clue what might be happening here? Any advice would be appreciated!<br></blockquote><div><br></div><div>So the failure seems to be here -- <a href="https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/screen-ir-api.txt.gz">https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/screen-ir-api.txt.gz</a>, API and conductor are not able to communicate via RPC for some reason. Need to investigate this more. Do you mind filing a bug about this?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Thanks,<br>
mjturek<br>
<br>
<br>
______________________________<wbr>______________________________<wbr>______________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.op<wbr>enstack.org?subject:unsubscrib<wbr>e</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-dev</a><br>
</blockquote></div><br></div></div>