Open Stack

On Thu, Aug 21, 2014 at 10:52:42AM -0400, Solly Ross wrote:
> FYI, the context of this is that I would like to be able to test some 
> of the libvirt storage pool code against a live file system, as we
> currently test the storage pool code.  To do this, we need at least to
> be able to get a proper connection to a session daemon.  IMHO, since
> these calls aren't "expensive", so to speak, it should be fine to have
> them run against a real libvirt.

No it really isn't OK to run against the real libvirt host system when
in the unit tests. Unit tests must *not* rely on external system state
in this way because it will lead to greater instability and unreliability
of our unit tests. If you want to test stuff against the real libvirt
storage pools then that becomes a functional / integration test suite
which is pretty much what tempest is targetting.
 
> > So If we require libvirt-python for tests and that requires
> > libvirt-bin, what's stopping us from just removing fakelibvirt since
> > it's kind of useless now anyway, right?
> 
> The thing about fakelibvirt is that it allows us to operate against
> against a libvirt API without actually doing libvirt-y things like
> launching VMs.  Now, libvirt does have a "test:///default" URI that
> IIRC has similar functionality, so we could start to phase out fake
> libvirt in favor of that.  However, there are probably still some
> spots where we'll want to use fakelibvirt.

I'm actually increasingly of the opinion that we should not in fact
be trying to use the real libvirt library in the unit tests at all
as it is not really adding any value. We typically nmock out all the
actual API calls we exercise so despite "using" libvirt-python we
are not in fact exercising its code or even validating that we're
passing the correct numbers of parameters to API calls. Pretty much
all we really relying on is the existance of the various global
constants that are defined, and that has been nothing but trouble
because the constants may or may not be defined depending on the
version.

The downside of fakelibvirt is that it is a half-assed implementation
of libvirt that we evolve in an adhoc fashion. I'm exploring the idea
of using pythons introspection abilities to query the libvirt-python
API and automatically generate a better 'fakelibvirt' that we can
guarantee to match the signatures of the real libvirt library. If we
had something like that which we had more confidence in, then we could
make the unit tests use that unconditionally. This would make our unit
tests more reliable since we would not be suspectible to different API
coverage in different libvirt module versions which have tripped us up
so many times

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|