Hi-<br><br><br>While bringing up an instance, I encountered this error at Spawning.<br><br>Please find the log details below...<br><br>Please help me resolve the issue.<br><br>2012-06-28 16:23:55 INFO nova.virt.libvirt.connection [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] Injecting key into image 83e47bfa-5eca-40cd-8bf1-785d52450d5a<br>
2012-06-28 16:23:55 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap qemu-nbd -c /dev/nbd15 /var/lib/nova/instances/instance-0000000c/disk from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap kpartx -a /dev/nbd15 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 32 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:235<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.<br>Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm<br>
Exit code: 32<br>Stdout: ''<br>Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n' from (pid=27972) trycmd /usr/lib/python2.7/dist-packages/nova/utils.py:277<br>2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap kpartx -d /dev/nbd15 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap qemu-nbd -d /dev/nbd15 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.virt.disk.api [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Failed to mount filesystem: Unexpected error while running command.<br>
Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm<br>Exit code: 32<br>Stdout: ''<br>Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n' from (pid=27972) mount /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:205<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap guestmount --rw -a /var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 1 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:235<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.<br>Command: sudo nova-rootwrap guestmount --rw -a /var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm<br>
Exit code: 1<br>Stdout: ''<br>Stderr: 'Traceback (most recent call last):\n File "/usr/bin/nova-rootwrap", line 69, in <module>\n env=filtermatch.get_environment(userargs))\n File "/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread, errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child\n raise child_exception\nOSError: [Errno 2] No such file or directory\n' from (pid=27972) trycmd /usr/lib/python2.7/dist-packages/nova/utils.py:277<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Running cmd (subprocess): sudo nova-rootwrap fusermount -u /tmp/tmpvusFCm from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Result was 1 from (pid=27972) execute /usr/lib/python2.7/dist-packages/nova/utils.py:235<br>
2012-06-28 16:23:56 DEBUG nova.utils [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Unexpected error while running command.<br>Command: sudo nova-rootwrap fusermount -u /tmp/tmpvusFCm<br>
Exit code: 1<br>Stdout: ''<br>Stderr: '/bin/fusermount: failed to unmount /tmp/tmpvusFCm: Invalid argument\n' from (pid=27972) trycmd /usr/lib/python2.7/dist-packages/nova/utils.py:277<br>2012-06-28 16:23:56 DEBUG nova.virt.disk.api [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Failed to mount filesystem: Unexpected error while running command.<br>
Command: sudo nova-rootwrap guestmount --rw -a /var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm<br>Exit code: 1<br>Stdout: ''<br>Stderr: 'Traceback (most recent call last):\n File "/usr/bin/nova-rootwrap", line 69, in <module>\n env=filtermatch.get_environment(userargs))\n File "/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread, errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child\n raise child_exception\nOSError: [Errno 2] No such file or directory\n' from (pid=27972) mount /usr/lib/python2.7/dist-packages/nova/virt/disk/api.py:205<br>
2012-06-28 16:23:56 WARNING nova.virt.libvirt.connection [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] Ignoring error injecting data into image 83e47bfa-5eca-40cd-8bf1-785d52450d5a (<br>
--<br>Failed to mount filesystem: Unexpected error while running command.<br>Command: sudo nova-rootwrap mount /dev/mapper/nbd15p1 /tmp/tmpvusFCm<br>Exit code: 32<br>Stdout: ''<br>Stderr: 'mount: special device /dev/mapper/nbd15p1 does not exist\n'<br>
--<br>Failed to mount filesystem: Unexpected error while running command.<br>Command: sudo nova-rootwrap guestmount --rw -a /var/lib/nova/instances/instance-0000000c/disk -m /dev/sda1 /tmp/tmpvusFCm<br>Exit code: 1<br>Stdout: ''<br>
Stderr: 'Traceback (most recent call last):\n File "/usr/bin/nova-rootwrap", line 69, in <module>\n env=filtermatch.get_environment(userargs))\n File "/usr/lib/python2.7/subprocess.py", line 679, in __init__\n errread, errwrite)\n File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child\n raise child_exception\nOSError: [Errno 2] No such file or directory\n')<br>
2012-06-28 16:24:00 ERROR nova.compute.manager [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] Instance failed to spawn<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] Traceback (most recent call last):<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] self._legacy_nw_info(network_info), block_device_info)<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] return f(*args, **kw)<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 922, in spawn<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] self._create_new_domain(xml)<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1575, in _create_new_domain<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] domain.createWithFlags(launch_flags)<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] File "/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)<br>2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] libvirtError: Unable to read from monitor: Connection reset by peer<br>
2012-06-28 16:24:00 TRACE nova.compute.manager [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] <br>2012-06-28 16:24:00 DEBUG nova.compute.manager [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] [instance: 7741f67f-ad78-4777-a5a0-6636eb8b460e] Deallocating network for instance from (pid=27972) _deallocate_network /usr/lib/python2.7/dist-packages/nova/compute/manager.py:616<br>
2012-06-28 16:24:00 DEBUG nova.rpc.amqp [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Making asynchronous cast on network... from (pid=27972) cast /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:346<br>
2012-06-28 16:24:01 ERROR nova.rpc.amqp [req-11e2e10a-f136-4182-b67a-4eb9e663cac0 668f5d9ca716417fa16208828c5acea8 8f2e0a43c2bc43a8b75b2c08d2598ca4] Exception during message handling<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp Traceback (most recent call last):<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 252, in _process_data<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp return f(*args, **kw)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 177, in decorated_function<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp sys.exc_info())<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 171, in decorated_function<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp return function(self, context, instance_uuid, *args, **kwargs)<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 651, in run_instance<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp do_run_instance()<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 945, in inner<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp retval = f(*args, **kwargs)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 650, in do_run_instance<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp self._run_instance(context, instance_uuid, **kwargs)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 451, in _run_instance<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp self._set_instance_error_state(context, instance_uuid)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 432, in _run_instance<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp self._deallocate_network(context, instance)<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp self.gen.next()<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 429, in _run_instance<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp injected_files, admin_password)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 592, in _spawn<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp self._legacy_nw_info(network_info), block_device_info)<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp return f(*args, **kw)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 922, in spawn<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp self._create_new_domain(xml)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1575, in _create_new_domain<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp domain.createWithFlags(launch_flags)<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/libvirt.py", line 581, in createWithFlags<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)<br>
2012-06-28 16:24:01 TRACE nova.rpc.amqp libvirtError: Unable to read from monitor: Connection reset by peer<br>2012-06-28 16:24:01 TRACE nova.rpc.amqp <br>2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task ComputeManager._publish_service_capabilities from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:27 DEBUG nova.manager [-] Notifying Schedulers of capabilities ... from (pid=27972) _publish_service_capabilities /usr/lib/python2.7/dist-packages/nova/manager.py:203<br>2012-06-28 16:24:27 DEBUG nova.rpc.amqp [-] Making asynchronous fanout cast... from (pid=27972) fanout_cast /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:354<br>
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_rescued_instances from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>2012-06-28 16:24:27 DEBUG nova.manager [-] Skipping ComputeManager._sync_power_states, 7 ticks left until next run from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:147<br>
2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_bandwidth_usage from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>2012-06-28 16:24:27 DEBUG nova.manager [-] Running periodic task ComputeManager.update_available_resource from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:29 INFO nova.virt.libvirt.connection [-] Compute_service record updated for openstackHost1 <br>2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_rebooting_instances from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:29 DEBUG nova.manager [-] Skipping ComputeManager._cleanup_running_deleted_instances, 27 ticks left until next run from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:147<br>2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task ComputeManager._heal_instance_info_cache from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:29 DEBUG nova.rpc.amqp [-] Making asynchronous call on network ... from (pid=27972) multicall /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:321<br>2012-06-28 16:24:29 DEBUG nova.rpc.amqp [-] MSG_ID is 8afecb4fea3b4df7938556b4da364590 from (pid=27972) multicall /usr/lib/python2.7/dist-packages/nova/rpc/amqp.py:324<br>
^C<br>root@openstackHost1:~# 2012-06-28 16:24:29 DEBUG nova.compute.manager [-] Updated the info_cache for instance 7741f67f-ad78-4777-a5a0-6636eb8b460e from (pid=27972) _heal_instance_info_cache /usr/lib/python2.7/dist-packages/nova/compute/manager.py:2227<br>
2012-06-28 16:24:29 DEBUG nova.manager [-] Skipping ComputeManager._run_image_cache_manager_pass, 37 ticks left until next run from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:147<br>2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task ComputeManager._reclaim_queued_deletes from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:29 DEBUG nova.compute.manager [-] FLAGS.reclaim_instance_interval <= 0, skipping... from (pid=27972) _reclaim_queued_deletes /usr/lib/python2.7/dist-packages/nova/compute/manager.py:2380<br>2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task ComputeManager._report_driver_status from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br>
2012-06-28 16:24:29 DEBUG nova.manager [-] Running periodic task ComputeManager._poll_unconfirmed_resizes from (pid=27972) periodic_tasks /usr/lib/python2.7/dist-packages/nova/manager.py:152<br><br><br>Thanking you<br clear="all">
<br>-- <br>Regards,<br>----------------------------------------------<br>Trinath Somanchi,<div>+91 9866 235 130</div><br>