<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>This is the part that determines the hypervisor:</p>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova-compute.conf<br>
[DEFAULT]<br>
compute_driver=libvirt.LibvirtDriver<br>
[libvirt]<br>
virt_type=xen<br>
</blockquote>
<br>
Nova uses libvirt to manage VMs, and libvirt uses Xen. According to
the <a
href="https://docs.openstack.org/nova/latest/admin/configuration/hypervisor-xen-libvirt.html">Nova/Xen
configuration guide</a>, it is enough to set these variables.
However, if you have not read that config page, do it now. Are you
using correct image formats? Do you find anything in the other
logfiles mentioned on that page?<br>
<br>
The references to <i>qemu-img </i>in the nova-compute log don't
mean that qemu is used. Nova uses <i>qemu-img</i> as a general tool
for converting VM images, no matter which hypervisor is configured.<br>
<br>
However, something prevents Nova to access one of the disk images,
namely /home/caslab/xenguest1/domains/guest1/disk.img. Is <i>guest1</i>
the VM you started manually?<br>
<br>
I see:<br>
<br>
Unexpected error while running command.<br>
Command: /usr/bin/python2 -m oslo_concurrency.prlimit
--as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C
<b> qemu-img info /home/caslab/xenguest1/domains/guest1/disk.img</b><br>
<br>
and<br>
<br>
Could not open '/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied<br>
<br>
I don't know, though, whether these errors are related to your
problem. Why is Libvirt or Xen unable to start your instances? I
suggest you look for other errors in the nova-compute.log files.<br>
<br>
Bernd<br>
<br>
<div class="moz-cite-prefix">On 11/2/2018 6:30 PM, Minjun Hong
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAFX_q-EARxvrWjaiPyhfn-zfcK-Rk2Wbg1wsdhcj6QrWGmB4eA@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Fri, Nov 2, 2018 at 9:05 AM Bernd
Bausch <<a href="mailto:berndbausch@gmail.com"
moz-do-not-send="true">berndbausch@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Welcome to OpenStack! I think the joy of
troubleshooting it is one of its main selling
points. <br>
</p>
<p>The conductor log says "<i>libxenlight failed
to create new domain</i>" on various nodes.
You should check the nova-compute logs on
those nodes for relevant messages. I suspect
an ill-configured interface between Nova
Compute and libvirt/Xen. In other words,
double-check the libvirt and xen sections of
nova.conf on those nodes.<br>
</p>
<p>Then there is this message "<i>Invalid </i><i>input
for field 'identity/password/user/password':
None is not of type 'string' ....</i>". This
is often caused by incompatible software. I.e.
you installed Openstack components that don't
fit together. Might also be caused by a
configuration error.<br>
Or it's an ordinary bug, but it seems to occur
on node 2 only.<br>
</p>
On 11/1/2018 11:01 PM, Minjun Hong wrote:<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi everyone.
<div>I'm really new to OpenStack.</div>
<div>After I install essential
components of OpenStack, such as
Nova, Keystone, etc, I attempted
creating an instance through
Openstack command in the terminal.</div>
<div>But a trouble has occurred. It
was logged in "nova-conductor.log":</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">2018-11-01
22:18:59.831 2570 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node2 (node
node2): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: Invalid input
for field
'identity/password/user/password':
None is not of type
'string'\n\nFailed validating
'type' in
schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:\n
{'type': 'string'}\n\nOn
instance['identity']['password']['user']['password']:\n
None (HTTP 400) (Request-ID:
req-350beee4-2fed-4645-9e21-79806d7ebfe7)\n"]<br>
2018-11-01 22:18:59.833 2570
WARNING oslo_config.cfg
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] Option
"os_region_name" from group
"placement" is deprecated. Use
option "region-name" from group
"placement".<br>
2018-11-01 22:19:10.936 2571 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node4 (node
node4): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: internal error:
libxenlight failed to create new
domain 'instance-00000005'\n"]<br>
2018-11-01 22:19:21.783 2567 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node5 (node
node5): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: internal error:
libxenlight failed to create new
domain 'instance-00000005'\n"]<br>
2018-11-01 22:19:21.783 2567
WARNING nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] Failed to
compute_task_build_instances:
Exceeded maximum number of
retries. Exceeded max scheduling
attempts 3 for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005':
MaxRetriesExceeded: Exceeded
maximum number of retries.
Exceeded max scheduling attempts 3
for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005'<br>
2018-11-01 22:19:21.784 2567
WARNING nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Setting instance to ERROR state.:
MaxRetriesExceeded: Exceeded
maximum number of retries.
Exceeded max scheduling attempts 3
for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005'</blockquote>
<div><br>
</div>
<div> I'm not sure which component
is involved in this trouble.</div>
</div>
<div>And libvirt and Xen have been
successfully installed on all
compute node I have without any
problem.</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">nickeys@node2:~$
virsh create ./test.xml<br>
Domain guest1 created from
./test.xml<br>
nickeys@node2:~$ virsh list<br>
Id Name State<br>
--------------------------<br>
0 Domain-0 running<br>
1 guest1 running<br>
nickeys@node2:~$</blockquote>
<div><br>
</div>
<div>What should I check first for
that issue ? </div>
</div>
<div>Your hint would be big help for
me.</div>
<div><br>
</div>
<div>Thanks!</div>
</div>
</div>
</div>
</div>
<br>
<fieldset
class="gmail-m_-6848492476500160991mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Mailing list: <a class="gmail-m_-6848492476500160991moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
Post to : <a class="gmail-m_-6848492476500160991moz-txt-link-abbreviated" href="mailto:openstack@lists.openstack.org" target="_blank" moz-do-not-send="true">openstack@lists.openstack.org</a>
Unsubscribe : <a class="gmail-m_-6848492476500160991moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
<div>
<h3 class="gmail-iw"
style="overflow:hidden;white-space:nowrap;max-width:92%;font-size:0.75rem;font-weight:inherit;margin:inherit;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><br>
</h3>
</div>
<div>Thanks for your kind answer, Bernd.</div>
<div>I checked the configure files which you
mentioned like it can cause the trouble. following
files are my nova.conf and nova-compute.conf of
the compute node ('node2').</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova.conf </blockquote>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">[DEFAULT]<br>
dhcpbridge_flagfile=/etc/nova/nova.conf<br>
dhcpbridge=/usr/bin/nova-dhcpbridge<br>
state_path=/var/lib/nova<br>
lock_path=/var/lock/nova<br>
force_dhcp_release=True<br>
libvirt_use_virtio_for_bridges=True<br>
verbose=True<br>
ec2_private_dns_show_ip=True<br>
api_paste_config=/etc/nova/api-paste.ini<br>
enabled_apis=osapi_compute,metadata<br>
transport_url = rabbit://<a
href="mailto:admin%3A1234@10.150.3.88"
moz-do-not-send="true">admin:1234@10.150.3.88</a><br>
my_ip = 10.150.21.182<br>
use_neutron = True<br>
firewall_driver =
nova.virt.firewall.NoopFirewallDriver<br>
[api]<br>
auth_strategy = keystone<br>
[vnc]<br>
enabled = True<br>
server_listen = 0.0.0.0<br>
vncserver_proxyclient_address = 10.150.21.182<br>
novncproxy_base_url = <a
href="http://10.150.3.88:6080/vnc_auth.html"
moz-do-not-send="true">http://10.150.3.88:6080/vnc_auth.html</a><br>
[glance]<br>
api_servers = <a href="http://10.150.3.88:9292"
moz-do-not-send="true">http://10.150.3.88:9292</a><br>
[oslo_concurrency]<br>
lock_path = /var/lib/nova/tmp<br>
[keystone_authtoken]<br>
auth_url = <a href="http://10.150.3.88:5000/v3"
moz-do-not-send="true">http://10.150.3.88:5000/v3</a><br>
memcached_servers = <a
href="http://10.150.3.88:11211"
moz-do-not-send="true">10.150.3.88:11211</a><br>
auth_type = password<br>
project_domain_name = default<br>
user_domain_name = default<br>
project_name = service<br>
username = nova<br>
password = <KEYSTONE_PASS><br>
[placement]<br>
os_region_name = RegionOne<br>
project_domain_name = Default<br>
project_name = service<br>
auth_type = password<br>
user_domain_name = Default<br>
auth_url = <a href="http://10.150.3.88:5000/v3"
moz-do-not-send="true">http://10.150.3.88:5000/v3</a><br>
username = placement<br>
password = <PLACEMENT_PASS><br>
[database]<br>
connection=sqlite:////var/lib/nova/nova.sqlite<br>
[api_database]<br>
connection=sqlite:////var/lib/nova/nova.sqlite<br>
[neutron]<br>
url = <a href="http://10.150.3.88:9696"
moz-do-not-send="true">http://10.150.3.88:9696</a><br>
auth_url = <a href="http://10.150.3.88:5000"
moz-do-not-send="true">http://10.150.3.88:5000</a><br>
auth_type = password<br>
project_domain_name = default<br>
user_domain_name = default<br>
region_name = RegionOne<br>
project_name = service<br>
username = neutron<br>
passsword = <NEUTRON_PASS></blockquote>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova-compute.conf<br>
[DEFAULT]<br>
compute_driver=libvirt.LibvirtDriver<br>
[libvirt]<br>
virt_type=xen</blockquote>
<div><br>
</div>
<div>And I set all the compute nodes I'm using with
the same configure file above.</div>
<div>Also, I installed the OpenStack components,
which belong to Queens version, following the
official installation guide (<a
href="https://docs.openstack.org/install-guide/"
moz-do-not-send="true">https://docs.openstack.org/install-guide/</a>).</div>
<div><br>
</div>
<div>By the way, I found something weird in
'/var/log/nova/nova-compute.log' of the 'node2'.</div>
<div>After I manually created a VM by 'virsh'
command (i.g. $ virsh create test.xml),
nova-compute of 'node2' started to complain.</div>
<div>And on an other node which have not created a
virtual machine by the 'virsh' command, there is
not below error.</div>
<div>I guess that the nova-compute component was
configured to qemu as virtualization type, but I
have no idea where the setting value is.</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px
0px 0px 0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">2018-11-02
18:19:13.048 16305 ERROR nova.compute.manager
[req-ec324126-6dfc-4011-9f45-41d014e6f900 - - -
- -] Error updating resources for node node2.:
InvalidDiskInfo: Disk info file is invalid:
qemu-img failed to execute on
/home/caslab/xenguest1/domains/guest1/disk.img :
Unexpected error while running command.<br>
Command: /usr/bin/python2 -m
oslo_concurrency.prlimit --as=1073741824
--cpu=30 -- env LC_ALL=C LANG=C qemu-img info
/home/caslab/xenguest1/domains/guest1/disk.img<br>
Exit code: 1<br>
Stdout: u''<br>
Stderr: u"qemu-img: Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied\n"<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Traceback (most recent call
last):<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 7284, in update_available_resource_for_node<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager
rt.update_available_resource(context, nodename)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py",
line 664, in update_available_resource<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager resources =
self.driver.get_available_resource(nodename)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 6440, in get_available_resource<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager disk_over_committed =
self._get_disk_over_committed_size_total()<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 8019, in
_get_disk_over_committed_size_total<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager config,
block_device_info)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 7918, in
_get_instance_disk_info_from_config<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager dk_size =
disk_api.get_allocated_disk_size(path)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py",
line 147, in get_allocated_disk_size<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager return
images.qemu_img_info(path).disk_size<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/images.py",
line 87, in qemu_img_info<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager raise
exception.InvalidDiskInfo(reason=msg)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager InvalidDiskInfo: Disk info
file is invalid: qemu-img failed to execute on
/home/caslab/xenguest1/domains/guest1/disk.img :
Unexpected error while running command.<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Command: /usr/bin/python2
-m oslo_concurrency.prlimit --as=1073741824
--cpu=30 -- env LC_ALL=C LANG=C qemu-img info
/home/caslab/xenguest1/domains/guest1/disk.img<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Exit code: 1<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Stdout: u''<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Stderr: u"qemu-img: Could
not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied\n"<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager</blockquote>
</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>