<div dir="ltr"><div dir="ltr"><div class="gmail_quote"><div dir="ltr">On Fri, Nov 2, 2018 at 9:12 PM Bernd Bausch <<a href="mailto:berndbausch@gmail.com">berndbausch@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>This is the part that determines the hypervisor:</p>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova-compute.conf<br>
[DEFAULT]<br>
compute_driver=libvirt.LibvirtDriver<br>
[libvirt]<br>
virt_type=xen<br>
</blockquote>
<br>
Nova uses libvirt to manage VMs, and libvirt uses Xen. According to
the <a href="https://docs.openstack.org/nova/latest/admin/configuration/hypervisor-xen-libvirt.html" target="_blank">Nova/Xen
configuration guide</a>, it is enough to set these variables.
However, if you have not read that config page, do it now. Are you
using correct image formats? Do you find anything in the other
logfiles mentioned on that page?<br>
<br>
The references to <i>qemu-img </i>in the nova-compute log don't
mean that qemu is used. Nova uses <i>qemu-img</i> as a general tool
for converting VM images, no matter which hypervisor is configured.<br>
<br>
However, something prevents Nova to access one of the disk images,
namely /home/caslab/xenguest1/domains/guest1/disk.img. Is <i>guest1</i>
the VM you started manually?<br>
<br>
I see:<br>
<br>
Unexpected error while running command.<br>
Command: /usr/bin/python2 -m oslo_concurrency.prlimit
--as=1073741824 --cpu=30 -- env LC_ALL=C LANG=C
<b> qemu-img info /home/caslab/xenguest1/domains/guest1/disk.img</b><br>
<br>
and<br>
<br>
Could not open '/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied<br>
<br>
I don't know, though, whether these errors are related to your
problem. Why is Libvirt or Xen unable to start your instances? I
suggest you look for other errors in the nova-compute.log files.<br>
<br>
Bernd<br>
<br>
<div class="gmail-m_-5140474126947329416moz-cite-prefix">On 11/2/2018 6:30 PM, Minjun Hong
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Fri, Nov 2, 2018 at 9:05 AM Bernd
Bausch <<a href="mailto:berndbausch@gmail.com" target="_blank">berndbausch@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Welcome to OpenStack! I think the joy of
troubleshooting it is one of its main selling
points. <br>
</p>
<p>The conductor log says "<i>libxenlight failed
to create new domain</i>" on various nodes.
You should check the nova-compute logs on
those nodes for relevant messages. I suspect
an ill-configured interface between Nova
Compute and libvirt/Xen. In other words,
double-check the libvirt and xen sections of
nova.conf on those nodes.<br>
</p>
<p>Then there is this message "<i>Invalid </i><i>input
for field 'identity/password/user/password':
None is not of type 'string' ....</i>". This
is often caused by incompatible software. I.e.
you installed Openstack components that don't
fit together. Might also be caused by a
configuration error.<br>
Or it's an ordinary bug, but it seems to occur
on node 2 only.<br>
</p>
On 11/1/2018 11:01 PM, Minjun Hong wrote:<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi everyone.
<div>I'm really new to OpenStack.</div>
<div>After I install essential
components of OpenStack, such as
Nova, Keystone, etc, I attempted
creating an instance through
Openstack command in the terminal.</div>
<div>But a trouble has occurred. It
was logged in "nova-conductor.log":</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">2018-11-01
22:18:59.831 2570 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node2 (node
node2): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: Invalid input
for field
'identity/password/user/password':
None is not of type
'string'\n\nFailed validating
'type' in
schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:\n
{'type': 'string'}\n\nOn
instance['identity']['password']['user']['password']:\n
None (HTTP 400) (Request-ID:
req-350beee4-2fed-4645-9e21-79806d7ebfe7)\n"]<br>
2018-11-01 22:18:59.833 2570
WARNING oslo_config.cfg
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] Option
"os_region_name" from group
"placement" is deprecated. Use
option "region-name" from group
"placement".<br>
2018-11-01 22:19:10.936 2571 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node4 (node
node4): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: internal error:
libxenlight failed to create new
domain 'instance-00000005'\n"]<br>
2018-11-01 22:19:21.783 2567 ERROR
nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Error from last host: node5 (node
node5): [u'Traceback (most recent
call last):\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 1840, in
_do_build_and_run_instance\n
filter_properties,
request_spec)\n', u' File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 2120, in
_build_and_run_instance\n
instance_uuid=instance.uuid,
reason=six.text_type(e))\n',
u"RescheduledException: Build of
instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee
was re-scheduled: internal error:
libxenlight failed to create new
domain 'instance-00000005'\n"]<br>
2018-11-01 22:19:21.783 2567
WARNING nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] Failed to
compute_task_build_instances:
Exceeded maximum number of
retries. Exceeded max scheduling
attempts 3 for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005':
MaxRetriesExceeded: Exceeded
maximum number of retries.
Exceeded max scheduling attempts 3
for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005'<br>
2018-11-01 22:19:21.784 2567
WARNING nova.scheduler.utils
[req-12aae2ff-4186-4ab0-964c-35b335c3188a
cc22ec575cb44e53aced9ddf58d9e8d7
965ff1c2002d4c278b5f838dbdbbb780 -
default default] [instance:
684b0a7d-22b9-4c87-88f8-b1474d3f9cee]
Setting instance to ERROR state.:
MaxRetriesExceeded: Exceeded
maximum number of retries.
Exceeded max scheduling attempts 3
for instance
684b0a7d-22b9-4c87-88f8-b1474d3f9cee.
Last exception: internal error:
libxenlight failed to create new
domain 'instance-00000005'</blockquote>
<div><br>
</div>
<div> I'm not sure which component
is involved in this trouble.</div>
</div>
<div>And libvirt and Xen have been
successfully installed on all
compute node I have without any
problem.</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">nickeys@node2:~$
virsh create ./test.xml<br>
Domain guest1 created from
./test.xml<br>
nickeys@node2:~$ virsh list<br>
Id Name State<br>
--------------------------<br>
0 Domain-0 running<br>
1 guest1 running<br>
nickeys@node2:~$</blockquote>
<div><br>
</div>
<div>What should I check first for
that issue ? </div>
</div>
<div>Your hint would be big help for
me.</div>
<div><br>
</div>
<div>Thanks!</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="gmail-m_-5140474126947329416gmail-m_-6848492476500160991mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Mailing list: <a class="gmail-m_-5140474126947329416gmail-m_-6848492476500160991moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
Post to : <a class="gmail-m_-5140474126947329416gmail-m_-6848492476500160991moz-txt-link-abbreviated" href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a>
Unsubscribe : <a class="gmail-m_-5140474126947329416gmail-m_-6848492476500160991moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a>
</pre>
</blockquote>
<br>
</div>
</blockquote>
<div>
<h3 class="gmail-m_-5140474126947329416gmail-iw" style="overflow:hidden;white-space:nowrap;max-width:92%;font-size:0.75rem;font-weight:inherit;margin:inherit;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><br>
</h3>
</div>
<div>Thanks for your kind answer, Bernd.</div>
<div>I checked the configure files which you
mentioned like it can cause the trouble. following
files are my nova.conf and nova-compute.conf of
the compute node ('node2').</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova.conf </blockquote>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">[DEFAULT]<br>
dhcpbridge_flagfile=/etc/nova/nova.conf<br>
dhcpbridge=/usr/bin/nova-dhcpbridge<br>
state_path=/var/lib/nova<br>
lock_path=/var/lock/nova<br>
force_dhcp_release=True<br>
libvirt_use_virtio_for_bridges=True<br>
verbose=True<br>
ec2_private_dns_show_ip=True<br>
api_paste_config=/etc/nova/api-paste.ini<br>
enabled_apis=osapi_compute,metadata<br>
transport_url = rabbit://<a href="mailto:admin%3A1234@10.150.3.88" target="_blank">admin:1234@10.150.3.88</a><br>
my_ip = 10.150.21.182<br>
use_neutron = True<br>
firewall_driver =
nova.virt.firewall.NoopFirewallDriver<br>
[api]<br>
auth_strategy = keystone<br>
[vnc]<br>
enabled = True<br>
server_listen = 0.0.0.0<br>
vncserver_proxyclient_address = 10.150.21.182<br>
novncproxy_base_url = <a href="http://10.150.3.88:6080/vnc_auth.html" target="_blank">http://10.150.3.88:6080/vnc_auth.html</a><br>
[glance]<br>
api_servers = <a href="http://10.150.3.88:9292" target="_blank">http://10.150.3.88:9292</a><br>
[oslo_concurrency]<br>
lock_path = /var/lib/nova/tmp<br>
[keystone_authtoken]<br>
auth_url = <a href="http://10.150.3.88:5000/v3" target="_blank">http://10.150.3.88:5000/v3</a><br>
memcached_servers = <a href="http://10.150.3.88:11211" target="_blank">10.150.3.88:11211</a><br>
auth_type = password<br>
project_domain_name = default<br>
user_domain_name = default<br>
project_name = service<br>
username = nova<br>
password = <KEYSTONE_PASS><br>
[placement]<br>
os_region_name = RegionOne<br>
project_domain_name = Default<br>
project_name = service<br>
auth_type = password<br>
user_domain_name = Default<br>
auth_url = <a href="http://10.150.3.88:5000/v3" target="_blank">http://10.150.3.88:5000/v3</a><br>
username = placement<br>
password = <PLACEMENT_PASS><br>
[database]<br>
connection=sqlite:////var/lib/nova/nova.sqlite<br>
[api_database]<br>
connection=sqlite:////var/lib/nova/nova.sqlite<br>
[neutron]<br>
url = <a href="http://10.150.3.88:9696" target="_blank">http://10.150.3.88:9696</a><br>
auth_url = <a href="http://10.150.3.88:5000" target="_blank">http://10.150.3.88:5000</a><br>
auth_type = password<br>
project_domain_name = default<br>
user_domain_name = default<br>
region_name = RegionOne<br>
project_name = service<br>
username = neutron<br>
passsword = <NEUTRON_PASS></blockquote>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">#
/etc/nova/nova-compute.conf<br>
[DEFAULT]<br>
compute_driver=libvirt.LibvirtDriver<br>
[libvirt]<br>
virt_type=xen</blockquote>
<div><br>
</div>
<div>And I set all the compute nodes I'm using with
the same configure file above.</div>
<div>Also, I installed the OpenStack components,
which belong to Queens version, following the
official installation guide (<a href="https://docs.openstack.org/install-guide/" target="_blank">https://docs.openstack.org/install-guide/</a>).</div>
<div><br>
</div>
<div>By the way, I found something weird in
'/var/log/nova/nova-compute.log' of the 'node2'.</div>
<div>After I manually created a VM by 'virsh'
command (i.g. $ virsh create test.xml),
nova-compute of 'node2' started to complain.</div>
<div>And on an other node which have not created a
virtual machine by the 'virsh' command, there is
not below error.</div>
<div>I guess that the nova-compute component was
configured to qemu as virtualization type, but I
have no idea where the setting value is.</div>
<div><br>
</div>
<div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">2018-11-02
18:19:13.048 16305 ERROR nova.compute.manager
[req-ec324126-6dfc-4011-9f45-41d014e6f900 - - -
- -] Error updating resources for node node2.:
InvalidDiskInfo: Disk info file is invalid:
qemu-img failed to execute on
/home/caslab/xenguest1/domains/guest1/disk.img :
Unexpected error while running command.<br>
Command: /usr/bin/python2 -m
oslo_concurrency.prlimit --as=1073741824
--cpu=30 -- env LC_ALL=C LANG=C qemu-img info
/home/caslab/xenguest1/domains/guest1/disk.img<br>
Exit code: 1<br>
Stdout: u''<br>
Stderr: u"qemu-img: Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied\n"<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Traceback (most recent call
last):<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py",
line 7284, in update_available_resource_for_node<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager
rt.update_available_resource(context, nodename)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/compute/resource_tracker.py",
line 664, in update_available_resource<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager resources =
self.driver.get_available_resource(nodename)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 6440, in get_available_resource<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager disk_over_committed =
self._get_disk_over_committed_size_total()<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 8019, in
_get_disk_over_committed_size_total<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager config,
block_device_info)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py",
line 7918, in
_get_instance_disk_info_from_config<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager dk_size =
disk_api.get_allocated_disk_size(path)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/disk/api.py",
line 147, in get_allocated_disk_size<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager return
images.qemu_img_info(path).disk_size<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager File
"/usr/lib/python2.7/dist-packages/nova/virt/images.py",
line 87, in qemu_img_info<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager raise
exception.InvalidDiskInfo(reason=msg)<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager InvalidDiskInfo: Disk info
file is invalid: qemu-img failed to execute on
/home/caslab/xenguest1/domains/guest1/disk.img :
Unexpected error while running command.<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Command: /usr/bin/python2
-m oslo_concurrency.prlimit --as=1073741824
--cpu=30 -- env LC_ALL=C LANG=C qemu-img info
/home/caslab/xenguest1/domains/guest1/disk.img<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Exit code: 1<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Stdout: u''<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager Stderr: u"qemu-img: Could
not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Could not open
'/home/caslab/xenguest1/domains/guest1/disk.img':
Permission denied\n"<br>
2018-11-02 18:19:13.048 16305 ERROR
nova.compute.manager</blockquote>
</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<br></div></blockquote><div><br></div><div><div>I found a crucial clue.</div><div>When I tried to create an instance, every time, I got the error messages repeated continuously.</div><div>It is from the log file of node2, /var/log/nova/nova-compute.log:</div><div><br></div><div><blockquote>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager [req-f93b17df-6718-457a-9ce0-916b4e051619 cc22ec575cb44e53aced9ddf58d9e8d7 965ff1c2002d4c278b5f838dbdbbb780 - default default] Instance failed network setup after 1 attempt(s): BadRequest: Invalid input for field 'identity/password/user/password': None is not of type 'string'<br>Failed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:<br> {'type': 'string'}<br>On instance['identity']['password']['user']['password']:<br> None (HTTP 400) (Request-ID: req-c9fac551-ebb4-4b2c-af67-5bad51d403fc)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager Traceback (most recent call last):<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1399, in _allocate_network_async<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager bind_host_id=bind_host_id)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 954, in allocate_for_instance<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager bind_host_id, available_macs, requested_ports_dict)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1087, in _update_ports_for_instance<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager vif.destroy()<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager self.force_reraise()<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in force_reraise<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager six.reraise(self.type_, self.value, self.tb)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1057, in _update_ports_for_instance<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager port_client, instance, port_id, port_req_body)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 468, in _update_port<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager port_response = port_client.update_port(port_id, port_req_body)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 799, in update_port<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager revision_number=revision_number)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 2375, in _update_resource<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self.put(path, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 363, in put<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager headers=headers, params=params)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 331, in retry_request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager headers=headers, params=params)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 282, in do_request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager headers=headers)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 343, in do_request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self.request(url, method, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 331, in request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager resp = super(SessionClient, self).request(*args, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 189, in request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self.session.request(url, method, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 573, in request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager auth_headers = self.get_auth_headers(auth)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 900, in get_auth_headers<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return auth.get_headers(self, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/plugin.py", line 95, in get_headers<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager token = self.get_token(session)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 88, in get_token<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self.get_access(session).auth_token<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager self.auth_ref = self.get_auth_ref(session)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 201, in get_auth_ref<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self._plugin.get_auth_ref(session, **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/v3/base.py", line 177, in get_auth_ref<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager authenticated=False, log=False, **rkwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 848, in post<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager return self.request(url, 'POST', **kwargs)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 737, in request<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager raise exceptions.from_response(resp, method, url)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager BadRequest: Invalid input for field 'identity/password/user/password': None is not of type 'string'<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager Failed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager {'type': 'string'}<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager On instance['identity']['password']['user']['password']:<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager None (HTTP 400) (Request-ID: req-c9fac551-ebb4-4b2c-af67-5bad51d403fc)<br>2018-11-03 21:44:24.231 16305 ERROR nova.compute.manager<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [req-f93b17df-6718-457a-9ce0-916b4e051619 cc22ec575cb44e53aced9ddf58d9e8d7 965ff1c2002d4c278b5f838dbdbbb780 - default default] [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] Instance failed to spawn: BadRequest: Invalid input for field 'identity/password/user/password': None is not of type 'string'<br>Failed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:<br> {'type': 'string'}<br>On instance['identity']['password']['user']['password']:<br> None (HTTP 400) (Request-ID: req-c9fac551-ebb4-4b2c-af67-5bad51d403fc)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] Traceback (most recent call last):<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2251, in _build_resources<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] yield resources<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2031, in _build_and_run_instance<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] block_device_info=block_device_info)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3084, in spawn<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] mdevs=mdevs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5368, in _get_guest_xml<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] network_info_str = str(network_info)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 568, in __str__<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self._sync_wrapper(fn, *args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 551, in _sync_wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] self.wait()<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/model.py", line 583, in wait<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] self[:] = self._gt.wait()<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 175, in wait<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self._exit_event.wait()<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 125, in wait<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] current.throw(*self._exc)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 214, in main<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] result = function(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 906, in context_wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return func(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1416, in _allocate_network_async<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] six.reraise(*exc_info)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1399, in _allocate_network_async<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] bind_host_id=bind_host_id)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 954, in allocate_for_instance<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] bind_host_id, available_macs, requested_ports_dict)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1087, in _update_ports_for_instance<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] vif.destroy()<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in __exit__<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] self.force_reraise()<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] six.reraise(self.type_, self.value, self.tb)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 1057, in _update_ports_for_instance<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] port_client, instance, port_id, port_req_body)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 468, in _update_port<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] port_response = port_client.update_port(port_id, port_req_body)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 799, in update_port<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] revision_number=revision_number)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 2375, in _update_resource<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self.put(path, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 363, in put<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] headers=headers, params=params)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/nova/network/neutronv2/api.py", line 114, in wrapper<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] ret = obj(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/v2_0/client.py", line 282, in do_request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] headers=headers)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 343, in do_request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self.request(url, method, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/neutronclient/client.py", line 331, in request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] resp = super(SessionClient, self).request(*args, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 189, in request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self.session.request(url, method, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 573, in request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] auth_headers = self.get_auth_headers(auth)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 900, in get_auth_headers<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return auth.get_headers(self, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/plugin.py", line 95, in get_headers<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] token = self.get_token(session)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 88, in get_token<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self.get_access(session).auth_token<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/base.py", line 134, in get_access<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] self.auth_ref = self.get_auth_ref(session)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/generic/base.py", line 201, in get_auth_ref<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self._plugin.get_auth_ref(session, **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/identity/v3/base.py", line 177, in get_auth_ref<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] authenticated=False, log=False, **rkwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 848, in post<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] return self.request(url, 'POST', **kwargs)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] File "/usr/lib/python2.7/dist-packages/keystoneauth1/session.py", line 737, in request<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] raise exceptions.from_response(resp, method, url)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] BadRequest: Invalid input for field 'identity/password/user/password': None is not of type 'string'<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762]<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] Failed validating 'type' in schema['properties']['identity']['properties']['password']['properties']['user']['properties']['password']:<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] {'type': 'string'}<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762]<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] On instance['identity']['password']['user']['password']:<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762] None (HTTP 400) (Request-ID: req-c9fac551-ebb4-4b2c-af67-5bad51d403fc)<br>2018-11-03 21:44:24.983 16305 ERROR nova.compute.manager [instance: 2ef5e55c-4868-4ff1-bbc9-3d82c4f99762]</blockquote></div></div><div> Above logs all occurred with only one instance creation.</div><div>Although I already searched the internet, I found nothing meaningful. </div><div><br></div><div>Just, I'm guessing that when I requested an instance creation on the controller node, it delivered the request with an empty parameter, which was deemed to None type. I mean, the empty one is password.</div><div><div><br></div><div>However, I'm confused because I have set all the config files which the official documentation guided to add.</div><div>As I think, the main issue is the reason the instance creation request was sent to 'node2' without password parameter.</div><div>Have you ever seen the error message I had?</div></div><div><br></div></div></div></div>