Open Stack

On 21-01-21 09:22:44, Lee Yarwood wrote:
> On Tue, 19 Jan 2021 at 20:28, aditi Dukle <aditi.Dukle at ibm.com> wrote:
> >
> > Hi Mike,
> >
> > I have started nova unit test jobs in the periodic pipeline(runs everyday at UTC- 12) for each openstack branch as follows:
> > periodic:
> >       jobs:
> >         - openstack-tox-py27:
> >             branches:
> >               - stable/ocata
> >               - stable/pike
> >               - stable/queens
> >               - stable/rocky
> >               - stable/stein
> >               - stable/train
> >         - openstack-tox-py36:
> >             branches:
> >               - stable/train
> >               - stable/ussuri
> >               - stable/victoria
> >         - openstack-tox-py37:
> >             branches:
> >               - stable/train
> >               - stable/ussuri
> >         - openstack-tox-py38:
> >             branches:
> >               - stable/victoria
> >         - openstack-tox-py39:
> >             branches:
> >               - master
> >
> > I have observed a few failures in the unit test cases mostly all related to volume drivers. Please have a look at the 14 test cases that are failing in openstack-tox-py39 job( https://oplab9.parqtec.unicamp.br/pub/ppc64el/openstack/nova/periodic/openstack-tox-py39/2021-01-19-0058-38c70ae/job-output.txt).  Most of the 14 failures report these errors:
> > InvalidConnectorProtocol: Invalid InitiatorConnector protocol specified NVME
> > InvalidConnectorProtocol: Invalid InitiatorConnector protocol specified SCALEIO
> >
> > I would need some help in understanding if these connectors(NVME,SCALEIO) are supported on ppc64.
> 
> Right os-brick doesn't support these connectors on ppc64 but the real
> issue here is with the unit tests that don't mock out calls os-brick,
> an out of tree lib. I've filed a bug for this below:
> 
> LibvirtNVMEVolumeDriverTestCase and LibvirtScaleIOVolumeDriverTestCase
> unit tests fail on ppc64
> https://bugs.launchpad.net/nova/+bug/1912608
> 
> I'll push a fix for this shortly.

libvirt: Stop NVMe and ScaleIO unit tests from calling os-brick
https://review.opendev.org/c/openstack/nova/+/771806/
 
> In addition to these failures I also see stable rescue tests failing,
> these should be fixed by the following change:
> 
> libvirt: Mock get_arch during some stable rescue unit tests
> https://review.opendev.org/c/openstack/nova/+/769916/

I also noticed some additional failures caused by the way in which the
libvirt virt driver loads its volume drivers at startup that also
attempt to load the underlying os-brick connector. As above this results
in volume drivers failing to load and being dropped.

We've actually had a long standing TODO in the driver to move to loading
these drivers on-demand so I've proposed the following:

libvirt: Load and cache volume drivers on-demand
https://review.opendev.org/c/openstack/nova/+/741545/

Can you rerun your tests using the above change and I'll try to address
any additional failures.
 
> Finally, in an effort to root out any further issues caused by tests
> looking up the arch of the test host I've pushed the following change
> to poison  nova.objects.fields.Architecture.from_host:
> 
> tests: Poison nova.objects.fields.Architecture.from_host
> https://review.opendev.org/c/openstack/nova/+/769920
> 
> There's a huge amount of fallout from this that I'll try to address in
> the coming weeks ahead of M3.
> 
> Hope this helps!
> 
> Lee
> 
> > ----- Original message -----
> > From: aditi Dukle/India/Contr/IBM
> > To: Michael J Turek/Poughkeepsie/IBM at IBM
> > Cc: Sajauddin Mohammad/India/Contr/IBM at IBM, openstack-discuss at lists.openstack.org
> > Subject: Re: [EXTERNAL] [nova] unit testing on ppc64le
> > Date: Tue, Jan 12, 2021 5:41 PM
> >
> > Hi Mike,
> >
> > I have created these unit test jobs - openstack-tox-py27, openstack-tox-py35, openstack-tox-py36, openstack-tox-py37, openstack-tox-py38, openstack-tox-py39
> > by referring to the upstream CI( https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/zuul.d/jobs.yaml) and these jobs are triggered for every patchset in the Openstack CI.
> >
> > I checked the code for old CI for Power, we didn't have any unit test jobs that were run for every patchset for nova. We had one "nova-python27" job that was run in a periodic pipeline. So, I wanted to know if we need to run the unit test jobs on ppc for every patchset for nova? and If yes, should these be reporting to the Openstack community?
> >
> >
> > Thanks,
> > Aditi Dukle
> >
> > ----- Original message -----
> > From: Michael J Turek/Poughkeepsie/IBM
> > To: balazs.gibizer at est.tech, aditi Dukle/India/Contr/IBM at IBM, Sajauddin Mohammad/India/Contr/IBM at IBM
> > Cc: openstack-discuss at lists.openstack.org
> > Subject: Re: [EXTERNAL] [nova] unit testing on ppc64le
> > Date: Sat, Jan 9, 2021 12:52 AM
> >
> > Thanks for the heads up,
> >
> > We should have the capacity to add them. At one point I think we ran unit tests for nova but the job may have been culled in the move to zuul v3. I've CC'd the maintainers of the CI, Aditi Dukle and Sajauddin Mohammad.
> >
> > Aditi and Sajauddin, could we add a job to pkvmci to run unit tests for nova?
> >
> > Michael Turek
> > Software Engineer
> > Power Cloud Department
> > 1 845 433 1290 Office
> > mjturek at us.ibm.com
> > He/Him/His
> >
> > IBM
> >
> >
> >
> >
> > ----- Original message -----
> > From: Balazs Gibizer <balazs.gibizer at est.tech>
> > To: OpenStack Discuss <openstack-discuss at lists.openstack.org>
> > Cc: mjturek at us.ibm.com
> > Subject: [EXTERNAL] [nova] unit testing on ppc64le
> > Date: Fri, Jan 8, 2021 7:59 AM
> >
> > Hi,
> >
> > We have a bugreport[1] showing that our unit tests are not passing on
> > ppc. In the upstream CI we don't have test capability to run our tests
> > on ppc. But we have the IBM Power KVM CI[2] that runs integration tests
> > on ppc. I'm wondering if IBM could extend the CI to run nova unit and
> > functional tests too. I've added Michael Turek (mjturek at us.ibm.com) to
> > CC. Michael is listed as the contact person for the CI.
> >
> > Cheers,
> > gibi
> >
> > [1]https://bugs.launchpad.net/nova/+bug/1909972
> > [2]https://wiki.openstack.org/wiki/ThirdPartySystems/IBMPowerKVMCI
> >
> >
> >
> >
> >
> >
> >
> >
> >

-- 
Lee Yarwood                 A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672 2D76