Open Stack

On Tue, Apr 4, 2017, at 06:36 AM, Daniel P. Berrange wrote:
> On Mon, Apr 03, 2017 at 04:06:53PM -0700, Clark Boylan wrote:
> > I have pushed a change to devstack [3] to enable using UCA which pulls
> > in new Libvirt and mostly seems to work. I think we should consider
> > switching to UCA as this may fix our Libvirt problems and if it doesn't,
> > we will be closer to a version of Libvirt that upstream should be
> > willing to fix.
> > 
> > This isn't the most straightfoward switch as UCA has a different repo
> > for each OpenStack release. libvirt-python is sensitve to the underlying
> > library changing; it is backward compatible but libvirt-python built
> > against older libvirt won't work against new libvirt. The result is a
> > libvirt-python wheel built on our wheel mirror does not work with UCA.
> 
> I'm surprised about that - could you elaborate on what's broken for you.
> The libvirt.so provides a stable public API, and the standalone python
> binding only uses public APIs from libvirt.so.  IOW you should be able
> to build libvirt-python against 1.3.0 and then use it against 2.5.0 with
> no problems.
> 
> NB, *before* libvirt-python lived on pypi, it used some non-public
> libvirt.so symbols, so was tied to the exact libvirt.so it was built
> against. Assuming you're using the pypi version this should no longer
> apply.

The specific issue was "AttributeError: 'module' object has no attribute
'VIR_MIGRATE_POSTCOPY'" where module here is libvirt (full log and
traceback at [0]). The libvirt-python module here was built against
Libvirt 1.3.1 turned into a wheel and copied into our wheel mirror. Then
when running against Libvirt 2.5.0 Nova seems to have detected that
newer features should be present that are not reflected in the compiled
libvirt-python resulting in the error. This crashed nova compute.

Problem was easily corrected by preventing devstack from using our wheel
mirror for libvirt-python which resulted in a new installation built
against Libvirt 2.5.0.

It seems like the API is stable enough for backward compatibility but
not forward compatibility. Its also possible that Nova is doing version
checking in a buggy way and should be checking what the libvirt-python
version is and what it supports?

[0]
http://logs.openstack.org/92/451492/4/check/gate-devstack-dsvm-updown-ubuntu-xenial/19dca66/logs/screen-n-cpu.txt.gz?level=ERROR#_2017-03-29_22_55_19_038

Hope this helps,
Clark