Open Stack

Hello,

We (agordeev and vkozhukalov) had a discussion about functional testing
flow described here
https://etherpad.openstack.org/p/IronicDevstackTesting.From our
perspective it has some non-critical, but still notable flaws. If
you ask me, it looks a bit strange to create testing networks and VMs
during devstack run using virsh and shell. Maybe more suitable to use
libvirt python API for this purpose and create test environment right
before launching testing scenario (aka setUp stage). We know that according
to tempest development requirements it is not allowed to hit hypervisors
directly ( http://docs.openstack.org/developer/tempest/overview.html ), but
using libvirt is not hitting hypervisor directly. We described alternative
testing flow at the same document
https://etherpad.openstack.org/p/IronicDevstackTesting.

There are some reasons why it is more correct to create VMs inside tempest
itself, not inside devstack.

   - Firstly, we possibly want to have several functional testing scenarios
   and every scenario needs to have clean testing environment, so we can just
   add setUp and tearDown stages for creating and destroying testing envs.
   - Secondly, virsh has some performance issues if you deal with >30 VMs
   (it is not our case for now but who knows). Besides, as we found out ssh
   power driver also uses virsh for VM power management and does it
   ineffectively looking over all VMs (virsh dumpxml for every VM and grep
   MAC). It is possible to significantly improve testing power management
   performance if we substitute ssh power driver with driver which uses
   libvirt python API and lookups VM by UUID, not by MAC.
   - Third point here is that adding nodes into ironic is also a part of
   ironic functionality and it is supposed to be tested as well as others.
   Besides, if something goes wrong during adding nodes into ironic, we'll
   get testing scenario failed, not devstack failed. It is what we usually
   expect from testing procedure.
   - And finally, using tempest with python libvirt API we can create
   widely customizable testing envs, it is not possible if we use devstack for
   that purpose.


Ok. It is just our point of view. If the decision about how to implement
creating testing VMs already made, we'll make it according to above
mentioned description. As far as original flow was suggested by infra and
QA gangs, it means that they a priori agree with that flow and we'll merge
it much faster than any other one.

Let us know what you think. Your feedback is highly appreciated :)

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140114/c22674d1/attachment.html>