<div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Fri, Dec 16, 2016 at 4:08 PM Steve Gordon <<a href="mailto:sgordon@redhat.com">sgordon@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">----- Original Message -----<br class="gmail_msg">
> From: "Neil Jerram" <<a href="mailto:neil@tigera.io" class="gmail_msg" target="_blank">neil@tigera.io</a>><br class="gmail_msg">
> To: "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org" class="gmail_msg" target="_blank">openstack-dev@lists.openstack.org</a>><br class="gmail_msg">
> Sent: Friday, December 16, 2016 10:53:02 AM<br class="gmail_msg">
> Subject: Re: [openstack-dev] Issues with libvirt transition 1.2.7 -> 2.0.0 (CentOS 7.3)<br class="gmail_msg">
><br class="gmail_msg">
> On Fri, Dec 16, 2016 at 3:35 PM Steve Gordon <<a href="mailto:sgordon@redhat.com" class="gmail_msg" target="_blank">sgordon@redhat.com</a>> wrote:<br class="gmail_msg">
><br class="gmail_msg">
> ><br class="gmail_msg">
> ><br class="gmail_msg">
> > ----- Original Message -----<br class="gmail_msg">
> > > From: "Neil Jerram" <<a href="mailto:neil@tigera.io" class="gmail_msg" target="_blank">neil@tigera.io</a>><br class="gmail_msg">
> > > To: "OpenStack Development Mailing List (not for usage questions)" <<br class="gmail_msg">
> > <a href="mailto:openstack-dev@lists.openstack.org" class="gmail_msg" target="_blank">openstack-dev@lists.openstack.org</a>><br class="gmail_msg">
> > > Sent: Friday, December 16, 2016 6:40:57 AM<br class="gmail_msg">
> > > Subject: [openstack-dev] Issues with libvirt transition 1.2.7 -> 2.0.0<br class="gmail_msg">
> > (CentOS 7.3)<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > I appreciate that even libvirt 2.0.0 will be ancient history by now, to<br class="gmail_msg">
> > its<br class="gmail_msg">
> > > developers, but I am seeing further issues that look associated with the<br class="gmail_msg">
> > > recent CentOS 7 transition from libvirt 1.2.7 to libvirt 2.0.0, and would<br class="gmail_msg">
> > > appreciate any comments on them that people may have. I believe these<br class="gmail_msg">
> > > issues are independent of those that have already been discussed on other<br class="gmail_msg">
> > > threads.<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > First, this traceback in nova-compute.log:<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > Traceback (most recent call last):<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line<br class="gmail_msg">
> > > 2156, in _build_resources<br class="gmail_msg">
> > > yield resources<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line<br class="gmail_msg">
> > > 2009, in _build_and_run_instance<br class="gmail_msg">
> > > block_device_info=block_device_info)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",<br class="gmail_msg">
> > line<br class="gmail_msg">
> > > 2534, in spawn<br class="gmail_msg">
> > > block_device_info=block_device_info)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",<br class="gmail_msg">
> > line<br class="gmail_msg">
> > > 4620, in _create_domain_and_network<br class="gmail_msg">
> > > xml, pause=pause, power_on=power_on)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",<br class="gmail_msg">
> > line<br class="gmail_msg">
> > > 4550, in _create_domain<br class="gmail_msg">
> > > guest.launch(pause=pause)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py",<br class="gmail_msg">
> > line<br class="gmail_msg">
> > > 142, in launch<br class="gmail_msg">
> > > self._encoded_xml, errors='ignore')<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line<br class="gmail_msg">
> > 195,<br class="gmail_msg">
> > > in __exit__<br class="gmail_msg">
> > > six.reraise(self.type_, self.value, self.tb)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/guest.py",<br class="gmail_msg">
> > line<br class="gmail_msg">
> > > 137, in launch<br class="gmail_msg">
> > > return self._domain.createWithFlags(flags)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 186, in<br class="gmail_msg">
> > > doit<br class="gmail_msg">
> > > result = proxy_call(self._autowrap, f, *args, **kwargs)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 144, in<br class="gmail_msg">
> > > proxy_call<br class="gmail_msg">
> > > rv = execute(f, *args, **kwargs)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 125, in<br class="gmail_msg">
> > > execute<br class="gmail_msg">
> > > six.reraise(c, e, tb)<br class="gmail_msg">
> > > File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 83, in<br class="gmail_msg">
> > > tworker<br class="gmail_msg">
> > > rv = meth(*args, **kwargs)<br class="gmail_msg">
> > > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1065, in<br class="gmail_msg">
> > > createWithFlags<br class="gmail_msg">
> > > if ret == -1: raise libvirtError ('virDomainCreateWithFlags()<br class="gmail_msg">
> > failed',<br class="gmail_msg">
> > > dom=self)<br class="gmail_msg">
> > > libvirtError: Cannot find '' in path: No such file or directory<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > which I believe is caused by the empty path attribute in this part of the<br class="gmail_msg">
> > > XML:<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > <interface type='ethernet'><br class="gmail_msg">
> > > <mac address='fa:16:3e:3c:96:33'/><br class="gmail_msg">
> > > <script path=''/><br class="gmail_msg">
> > > <target dev='tap06992dfb-5d'/><br class="gmail_msg">
> > > <model type='virtio'/><br class="gmail_msg">
> > > <driver name='qemu'/><br class="gmail_msg">
> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x03'<br class="gmail_msg">
> > > function='0x0'/><br class="gmail_msg">
> > > </interface><br class="gmail_msg">
> > ><br class="gmail_msg">
> > > which is in turn caused, I think, by<br class="gmail_msg">
> > ><br class="gmail_msg">
> > <a href="https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61</a><br class="gmail_msg">
> > ><br class="gmail_msg">
> > > Is it plausible that libvirt 1.2.7 would have avoided trying to invoke a<br class="gmail_msg">
> > > script with an empty path, whereas libvirt 2.0.0 does not?<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > Secondly - if I move past the problem above by changing<br class="gmail_msg">
> > ><br class="gmail_msg">
> > <a href="https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/openstack/nova/blob/master/nova/virt/libvirt/designer.py#L61</a><br class="gmail_msg">
> > > to say 'conf.script = None' - I then find:<br class="gmail_msg">
> > > - no apparent error in nova-compute.log<br class="gmail_msg">
> > > - but my instances don't boot<br class="gmail_msg">
> > > - the following messages in the libvirt log:<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > Domain id=4 is tainted: high-privileges<br class="gmail_msg">
> > > char device redirected to /dev/pts/1 (label charserial1)<br class="gmail_msg">
> > > CPU feature tsc_adjust not found<br class="gmail_msg">
> > ><br class="gmail_msg">
> > > I guess it's the last message that is the critical one here - can anyone<br class="gmail_msg">
> > > tell me more about it?<br class="gmail_msg">
> ><br class="gmail_msg">
> > Hi Neil,<br class="gmail_msg">
> ><br class="gmail_msg">
> > The second seems likely to be related to the addition of support for VMX<br class="gmail_msg">
> > TSC scaling. What version of qemu are you running?<br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
> Thanks for your reply, Steve. I have these qemu packages installed:<br class="gmail_msg">
><br class="gmail_msg">
> [root@neil-fv-0-centos-liberty-compute-node01 qemu]# yum list installed |<br class="gmail_msg">
> grep qemu<br class="gmail_msg">
> ipxe-roms-qemu.noarch 20160127-5.git6366fa7a.el7<br class="gmail_msg">
> libvirt-daemon-driver-qemu.x86_64 2.0.0-10.el7_3.2<br class="gmail_msg">
> @updates<br class="gmail_msg">
> qemu-img.x86_64 10:1.5.3-126.el7<br class="gmail_msg">
> @base<br class="gmail_msg">
> qemu-kvm.x86_64 10:1.5.3-126.el7<br class="gmail_msg">
> @base<br class="gmail_msg">
> qemu-kvm-common.x86_64 10:1.5.3-126.el7<br class="gmail_msg">
> @base<br class="gmail_msg">
><br class="gmail_msg">
> I should probably also mention that I'm using virt_type = qemu, because my<br class="gmail_msg">
> compute nodes are GCE instances.<br class="gmail_msg">
><br class="gmail_msg">
> [root@neil-fv-0-centos-liberty-compute-node01 qemu]# cat<br class="gmail_msg">
> /etc/nova/nova-compute.conf<br class="gmail_msg">
> [libvirt]<br class="gmail_msg">
> virt_type = qemu<br class="gmail_msg">
><br class="gmail_msg">
> If you haven't already I would suggest grabbing qemu-kvm-ev from the CentOS<br class="gmail_msg">
> > Virt SIG repos:<br class="gmail_msg">
> ><br class="gmail_msg">
> > <a href="http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/" rel="noreferrer" class="gmail_msg" target="_blank">http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/</a><br class="gmail_msg">
> ><br class="gmail_msg">
> > You can enable the repository using this release RPM:<br class="gmail_msg">
> ><br class="gmail_msg">
> ><br class="gmail_msg">
> > <a href="http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/centos-release-qemu-ev-1.0-1.el7.noarch.rpm" rel="noreferrer" class="gmail_msg" target="_blank">http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/centos-release-qemu-ev-1.0-1.el7.noarch.rpm</a><br class="gmail_msg">
> ><br class="gmail_msg">
> ><br class="gmail_msg">
> Would you expect that to help with virt_type = qemu (as well as with<br class="gmail_msg">
> virt_type = kvm, which I assume is the more common setting)? If so I'll be<br class="gmail_msg">
> very excited to try this!<br class="gmail_msg">
<br class="gmail_msg">
For this particular flag issue I am not 100% sure yet as I'm still checking with some of the qemu folks, but I think it would still be worth a try.<br class="gmail_msg"></blockquote><div><br></div><div>Well I have at least one booting instance now, and there is no mention of 'tsc_adjust not found' in the instance's log.<br><br></div><div>So looking more promising - thanks!<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br class="gmail_msg">
Thanks,<br class="gmail_msg">
<br class="gmail_msg">
--<br class="gmail_msg">
Steve Gordon,<br class="gmail_msg">
Principal Product Manager,<br class="gmail_msg">
Red Hat OpenStack Platform<br class="gmail_msg">
<br class="gmail_msg">
__________________________________________________________________________<br class="gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail_msg" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="gmail_msg">
</blockquote></div></div>