Open Stack

Hi all,

I'll disable the shelve/unshelve failing tests in Intel NFV CI for the time being till we have clarity on the problem.
Stephen Finucane will be working on that in next week or so.

 >-----Original Message-----
 >From: Chris Friesen [mailto:chris.friesen at windriver.com]
 >Sent: Wednesday, May 25, 2016 5:40 PM
 >To: openstack-dev at lists.openstack.org
 >Subject: Re: [openstack-dev] [nova] Intel NFV CI failing all shelve/unshelve
 >tests
 >
 >On 05/22/2016 05:41 PM, Jay Pipes wrote:
 >> Hello Novaites,
 >>
 >> I've noticed that the Intel NFV CI has been failing all test runs for
 >> quite some time (at least a few days), always failing the same tests
 >> around shelve/unshelve operations.
 >
 ><snip>
 >
 >> I looked through the conductor and compute logs to see if I could find
 >> any possible reasons for the errors and found a number of the
 >> following errors in the compute logs:
 >>
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52] Traceback (most recent call last):
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52]   File
 >> "/opt/stack/new/nova/nova/compute/manager.py", line 4230, in
 >> _unshelve_instance
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52]     with rt.instance_claim(context,
 >> instance, limits):
 >
 ><snip>
 >
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52]
 >newcell.unpin_cpus(pinned_cpus)
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52]   File
 >> "/opt/stack/new/nova/nova/objects/numa.py", line 94, in unpin_cpus
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52]     pinned=list(self.pinned_cpus))
 >> 2016-05-22 22:18:59.403 8145 ERROR nova.compute.manager [instance:
 >> cae6fd47-0968-4922-a03e-3f2872e4eb52] CPUPinningInvalid: Cannot
 >> pin/unpin cpus [6] from the following pinned set [0, 2, 4]
 >>
 >> on or around the time of the failures in Tempest.
 >>
 >> Perhaps tomorrow morning we can look into handling the above exception
 >> properly from the compute manager, since clearly we shouldn't be
 >> allowing CPUPinningInvalid to be raised in the resource tracker's
 >_update_usage() call....
 >
 >First, it seems wrong to me that an _unshelve_instance() call would result in
 >unpinning any CPUs.  If the instance was using pinned CPUs then I would
 >expect the CPUs to be unpinned when doing the "shelve" operation.  When
 >we do an instance claim as part of the "unshelve" operation we should be
 >pinning CPUs, not unpinning them.
 >
 >Second, the reason why CPUPinningInvalid gets raised in _update_usage() is
 >that it has discovered an inconsistency in its view of resources.  In this case,
 >it's trying to unpin CPU 6 from a set of pinned cpus that doesn't include CPU
 >6.  I think this is a valid concern and should result in an error log.  Whether it
 >should cause the unshelve operation to fail is a separate question, but it's
 >definitely a symptom that something is wrong with resource tracking on this
 >compute node.
 >
 >Chris
 >
 >
 >__________________________________________________________
 >________________
 >OpenStack Development Mailing List (not for usage questions)
 >Unsubscribe: OpenStack-dev-
 >request at lists.openstack.org?subject:unsubscribe
 >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.