[openstack-dev] [nova] unit tests result in false negatives on system z platform CI
mikal at stillhq.com
Wed Apr 1 21:01:51 UTC 2015
Thanks for the detailed email on this. How about we add this to the
agenda for this weeks nova meeting?
One option would be to add a fixture to some higher level test class,
but perhaps someone has a better idea than that.
On Wed, Apr 1, 2015 at 8:54 PM, Markus Zoeller <mzoeller at de.ibm.com> wrote:
> During the Kilo development cycle the "KVM/libvirt on system z" platform
> made some effort to be supported by the libvirt driver .
> Our first tests in a prototype platform CI showed some false negatives
> because some unit tests don't seem to be fully platform independent.
> For example the result of test:
> test_get_guest_config_without_qga_through_image_meta [0.016369s]
> ... FAILED
> Captured traceback:
> Traceback (most recent call last):
> File "nova/tests/unit/virt/libvirt/test_driver.py", line 3112,
> in test_get_guest_config_without_qga_through_image_meta
> raise mismatch_error
> '<nova.virt.libvirt.config.LibvirtConfigGuestConsole object>'
> is not an instance of LibvirtConfigGuestSerial
> This mismatch makes fully sense if x86 is assumed as default underlying
> platform for unit test execution. The root cause (in this case) is that
> the call of "libvirt_utils.get_arch()" is not mocked and actually speaks
> to the underlying platform. On our system z CI this call returns "s390x"
> which hits platform switches in the code (a search for "arch.S390X" will
> show you all this platform specific code).
> A first test
> A change of this specific test could look like this:
> # git diff nova/tests/unit/virt/libvirt/test_driver.py
> diff --git a/nova/tests/unit/virt/libvirt/test_driver.py
> index 5fbe5e1..ebcc9ed 100644
> --- a/nova/tests/unit/virt/libvirt/test_driver.py
> +++ b/nova/tests/unit/virt/libvirt/test_driver.py
> @@ -3091,7 +3091,9 @@ class LibvirtConnTestCase(test.NoDBTestCase):
> - def test_get_guest_config_without_qga_through_image_meta(self):
> + @mock.patch.object(libvirt_driver.libvirt_utils, 'get_arch',
> + return_value=arch.X86_64)
> + def test_get_guest_config_without_qga_through_image_meta(self,
> self.flags(virt_type='kvm', group='libvirt')
> drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True)
> Open questions
> We currently discovered around 30 test cases (mostly the class
> "LibvirtConnTestCase") which could be treated that way, which seems to
> be cumbersome from my point of view. I'm looking for a way to express
> the assumption that x86 should be the default platform in the unit tests
> and prevent calls to the underlying system. This has to be rewritable if
> platform specific code like in  has to be tested.
> I'd like to discuss how that could be achieved in a maintainable way.
>  https://blueprints.launchpad.net/nova/+spec/libvirt-kvm-systemz
>  test_driver.py; test_get_guest_config_with_type_kvm_on_s390;
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev