[openstack-dev] [ironic] [third-party-ci] pkvmci ironic job breakage details
Michael Turek
mjturek at linux.vnet.ibm.com
Tue Apr 18 20:18:40 UTC 2017
On 04/18/2017 07:56 AM, Vladyslav Drok wrote:
> Hey Michael,
>
> On Fri, Apr 14, 2017 at 6:51 PM, Michael Turek
> <mjturek at linux.vnet.ibm.com <mailto:mjturek at linux.vnet.ibm.com>> wrote:
>
> Hey ironic-ers,
>
> So our third party CI job for ironic has been, and remains,
> broken. I was able to do some investigation today and here's a
> summary of what we're seeing. I'm hoping someone might know the
> root of the problem.
>
> For reference, please see this paste and the logs of the job that
> I was working in:
> http://paste.openstack.org/show/606564/
> <http://paste.openstack.org/show/606564/>
> https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/
> <https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/>
>
> I've redacted the credentials in the ironic node-show for obvious
> reasons but rest assured they are properly set. These commands are
> run while
> '/opt/stack/new/ironic/devstack/lib/ironic:wait_for_nova_resources'
> is looping.
>
> Basically, the ironic hypervisor for the node doesn't appear. As
> well, none of the node's properties make it to the hypervisor stats.
>
> Some more strangeness is that the 'count' value from the
> 'openstack hypervisor stats show'. Though no hypervisors appear,
> the count is still 1. Since the run was broken, I decided to
> delete node-0 (about 3-5 minutes before the run failed) and see if
> it updated the count. It did.
>
> Does anyone have any clue what might be happening here? Any advice
> would be appreciated!
>
>
> So the failure seems to be here --
> https://dal05.objectstorage.softlayer.net/v1/AUTH_3d8e6ecb-f597-448c-8ec2-164e9f710dd6/pkvmci/ironic/25/454625/10/check-ironic/tempest-dsvm-ironic-agent_ipmitool/0520958/screen-ir-api.txt.gz,
> API and conductor are not able to communicate via RPC for some reason.
> Need to investigate this more. Do you mind filing a bug about this?
>
>
> Thanks,
> mjturek
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Thanks vdrok,
Bug is opened here - https://bugs.launchpad.net/ironic/+bug/1683902
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20170418/4f5301ab/attachment.html>
More information about the OpenStack-dev
mailing list