[openstack-dev] [ironic] some notes on ironic functional testing
agordeev at mirantis.com
Tue Jan 14 14:28:05 UTC 2014
We (agordeev and vkozhukalov) had a discussion about functional testing
flow described here
perspective it has some non-critical, but still notable flaws. If
you ask me, it looks a bit strange to create testing networks and VMs
during devstack run using virsh and shell. Maybe more suitable to use
libvirt python API for this purpose and create test environment right
before launching testing scenario (aka setUp stage). We know that according
to tempest development requirements it is not allowed to hit hypervisors
directly ( http://docs.openstack.org/developer/tempest/overview.html ), but
using libvirt is not hitting hypervisor directly. We described alternative
testing flow at the same document
There are some reasons why it is more correct to create VMs inside tempest
itself, not inside devstack.
- Firstly, we possibly want to have several functional testing scenarios
and every scenario needs to have clean testing environment, so we can just
add setUp and tearDown stages for creating and destroying testing envs.
- Secondly, virsh has some performance issues if you deal with >30 VMs
(it is not our case for now but who knows). Besides, as we found out ssh
power driver also uses virsh for VM power management and does it
ineffectively looking over all VMs (virsh dumpxml for every VM and grep
MAC). It is possible to significantly improve testing power management
performance if we substitute ssh power driver with driver which uses
libvirt python API and lookups VM by UUID, not by MAC.
- Third point here is that adding nodes into ironic is also a part of
ironic functionality and it is supposed to be tested as well as others.
Besides, if something goes wrong during adding nodes into ironic, we'll
get testing scenario failed, not devstack failed. It is what we usually
expect from testing procedure.
- And finally, using tempest with python libvirt API we can create
widely customizable testing envs, it is not possible if we use devstack for
Ok. It is just our point of view. If the decision about how to implement
creating testing VMs already made, we'll make it according to above
mentioned description. As far as original flow was suggested by infra and
QA gangs, it means that they a priori agree with that flow and we'll merge
it much faster than any other one.
Let us know what you think. Your feedback is highly appreciated :)
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev