[openstack-dev] [ironic] [third-party-ci] pkvmci ironic job breakage details

Michael Turek mjturek at linux.vnet.ibm.com
Tue Apr 18 20:18:40 UTC 2017


On 04/18/2017 07:56 AM, Vladyslav Drok wrote:
> Hey Michael,
>
> On Fri, Apr 14, 2017 at 6:51 PM, Michael Turek 
> <mjturek at linux.vnet.ibm.com <mailto:mjturek at linux.vnet.ibm.com>> wrote:
>
>     Hey ironic-ers,
>
>     So our third party CI job for ironic has been, and remains,
>     broken. I was able to do some investigation today and here's a
>     summary of what we're seeing. I'm hoping someone might know the
>     root of the problem.
>
>     For reference, please see this paste and the logs of the job that
>     I was working in:
>     http://paste.openstack.org/show/606564/
>     <http://paste.openstack.org/show/606564/>
>     https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/
>     <https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/>
>
>     I've redacted the credentials in the ironic node-show for obvious
>     reasons but rest assured they are properly set. These commands are
>     run while
>     '/opt/stack/new/ironic/devstack/lib/ironic:wait_for_nova_resources'
>     is looping.
>
>     Basically, the ironic hypervisor for the node doesn't appear. As
>     well, none of the node's properties make it to the hypervisor stats.
>
>     Some more strangeness is that the 'count' value from the
>     'openstack hypervisor stats show'. Though no hypervisors appear,
>     the count is still 1. Since the run was broken, I decided to
>     delete node-0 (about 3-5 minutes before the run failed) and see if
>     it updated the count. It did.
>
>     Does anyone have any clue what might be happening here? Any advice
>     would be appreciated!
>
>
> So the failure seems to be here -- 
> https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/screen-ir-api.txt.gz, 
> API and conductor are not able to communicate via RPC for some reason. 
> Need to investigate this more. Do you mind filing a bug about this?
>
>
>     Thanks,
>     mjturek
>
>
>     __________________________________________________________________________
>     OpenStack Development Mailing List (not for usage questions)
>     Unsubscribe:
>     OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>     <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Thanks vdrok,

Bug is opened here - https://bugs.launchpad.net/ironic/+bug/1683902
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170418/4f5301ab/attachment.html>


More information about the OpenStack-dev mailing list