<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Hi, <div>I'm thinking about two things here </div><div>- disk full</div><div>- permission issues</div><div>Can you run the following command : </div><div></div><blockquote type="cite"><div><font face="Consolas">su - nova -c "touch /data/openstack/nova/instance/test"</font></div></blockquote><div><br></div><div>thanks</div><div>Razique</div><div><br></div><div><br></div><div><blockquote type="cite">Command: qemu-img convert -O raw /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.converted</blockquote><div apple-content-edited="true">
<span class="Apple-style-span" style="border-collapse: separate; font-family: 'Lucida Grande'; border-spacing: 0px; "><span class="Apple-style-span" style="font-family: Helvetica; "><span class="Apple-style-span" style="font-family: Helvetica; "><b style="color: rgb(19, 112, 138); ">Nuage & Co - Razique Mahroua</b></span><span class="Apple-style-span" style="color: rgb(19, 112, 138); font-family: Helvetica; "><b> </b></span><span class="Apple-style-span" style="border-collapse: separate; font-family: Helvetica; font-style: normal; font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><span class="Apple-style-span" style="border-collapse: separate; font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><span class="Apple-style-span" style="border-collapse: separate; font-variant: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; "><div style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-weight: normal; font-style: normal; "><font class="Apple-style-span" color="#13708a"><a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a></font></div></span><br class="Apple-interchange-newline"><span></span><span style="color: rgb(0, 0, 0); font-family: Helvetica; font-size: medium; font-weight: normal; font-style: normal; "></span><span style="color: rgb(0, 0, 0); font-weight: normal; "></span></span><span><img height="125" width="125" id="2e924be8-4f38-40e3-8795-82d655beee0a" apple-width="yes" apple-height="yes" src="cid:00E07450-433E-43E6-9005-48B8FD5C669F@la.fabrique"></span>
</span></span></span></div>
<br><div><div>Le 24 sept. 2012 à 05:21, "Wangpan"<<a href="mailto:hzwangpan@corp.netease.com">hzwangpan@corp.netease.com</a>> a écrit :</div><br class="Apple-interchange-newline"><blockquote type="cite">Hi all,<br>I have a problem to consult, this problem occurs in _create_image when I boot a new instance(someone else may boot one at the same time).<br>The instance UUID of mine is e8752971-d390-450f-a083-819b7716fd53, and it booted failed, the other instance UUID is 79e7b88f-2302-4387-a27e-ffdbd483f6d2, it booted successfully.<br>the problem occurs when these two instances convert the same base image from qcow2 to raw(in the fetch_to_raw method) at the same time, the convert cmd of my instance is failed with error: 'qemu-img: error while writing sector 634880: Input/output error', the convertion starting time of my instance is 2012-09-21 17:54:58, one second ahead of the other instance 2012-09-21 17:54:59.<br>So I think this is a concurrency issue, what about guys think about it?<br>the log message is listed below:<br>1. receive 'run_instance' message from AMQP:<br>2012-09-21 17:48:42 DEBUG nova.rpc.amqp [-] received {u'_context_roles': [u'', u'admin'], u'_context_request_id': u'req-65ffd7d5-f615-4ee0-8007-f71657227632', u'_context_read_deleted': u'no', u'args': {u'instance_uuid': u'e8752971-d390-450f-a083-819b7716fd53', u'requested_networks': None, u'availability_zone': u'nova12', u'admin_password': '<SANITIZED>', u'injected_files': [[u'/etc/vm_monitor/info', u'{"service":"RDS","ori_user":"cc130e62151540c09e1745841e8b3ff6","resource_type":"RDS","resource_id":"889","aggregation_item_name":"","aggregation_item_value":""}']], u'is_first_time': True}, u'_context_auth_token': '<SANITIZED>', u'_context_is_admin': True, u'_context_project_id': u'844dc9fb4ad9499ea97b07c7b6cee137', u'_context_timestamp': u'2012-09-21T09:48:42.304584', u'_context_user_id': u'64d4d53b99f84cf1ba6becd72ed10982', u'method': u'run_instance', u'_context_remote_address': u'10.120.40.28'} from (pid=28544) _safe_log /usr/lib/python2.7/dist-packages/nova/rpc/common.py:160<br>2012-09-21 17:48:43 DEBUG nova.rpc.amqp [-] received {u'_context_roles': [u'', u'admin'], u'_context_request_id': u'req-4202dc5f-022f-4c1b-bfbf-71dae31f5476', u'_context_read_deleted': u'no', u'args': {u'instance_uuid': u'79e7b88f-2302-4387-a27e-ffdbd483f6d2', u'requested_networks': None, u'availability_zone': u'nova12', u'admin_password': '<SANITIZED>', u'injected_files': [[u'/etc/vm_monitor/info', u'{"service":"RDS","ori_user":"cc130e62151540c09e1745841e8b3ff6","resource_type":"RDS","resource_id":"890","aggregation_item_name":"","aggregation_item_value":""}']], u'is_first_time': True}, u'_context_auth_token': '<SANITIZED>', u'_context_is_admin': True, u'_context_project_id': u'844dc9fb4ad9499ea97b07c7b6cee137', u'_context_timestamp': u'2012-09-21T09:48:43.024286', u'_context_user_id': u'64d4d53b99f84cf1ba6becd72ed10982', u'method': u'run_instance', u'_context_remote_address': u'10.120.40.28'} from (pid=28544) _safe_log /usr/lib/python2.7/dist-packages/nova/rpc/common.py:160<br>2. create image:<br>2012-09-21 17:48:48 INFO nova.virt.libvirt.connection [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] [instance: e8752971-d390-450f-a083-819b7716fd53] Creating image<br>2012-09-21 17:54:51 DEBUG nova.utils [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>2012-09-21 17:54:51 DEBUG nova.utils [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>2012-09-21 17:54:58 DEBUG nova.virt.images [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] 2fd9c9e7-115d-4a39-8ea3-3adc9f0dda68 was qcow2, converting to raw from (pid=28544) fetch_to_raw /usr/lib/python2.7/dist-packages/nova/virt/images.py:104<br>2012-09-21 17:54:58 DEBUG nova.utils [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Running cmd (subprocess): qemu-img convert -O raw /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.converted from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>2012-09-21 17:54:59 DEBUG nova.utils [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Result was 1 from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:235<br>2012-09-21 17:54:59 ERROR nova.compute.manager [req-65ffd7d5-f615-4ee0-8007-f71657227632 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] [instance: e8752971-d390-450f-a083-819b7716fd53] Instance failed to spawn<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] Traceback (most recent call last):<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 606, in _spawn<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] self._legacy_nw_info(network_info), block_device_info)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 114, in wrapped<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] return f(*args, **kw)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1018, in spawn<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] block_device_info=block_device_info)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 1439, in _create_image<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] basepath('disk'), size)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/utils.py", line 296, in fetch_image<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] images.fetch_to_raw(context, image_id, target, user_id, project_id)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/virt/images.py", line 106, in fetch_to_raw<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] path_tmp, staged)<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 242, in execute<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] cmd=' '.join(cmd))<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] ProcessExecutionError: Unexpected error while running command.<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] Command: qemu-img convert -O raw /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.converted<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] Exit code: 1<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] Stdout: ''<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] Stderr: 'qemu-img: error while writing sector 634880: Input/output error\n' ###ERROR<br>2012-09-21 17:54:59 TRACE nova.compute.manager [instance: e8752971-d390-450f-a083-819b7716fd53] <br><br>2012-09-21 17:48:49 INFO nova.virt.libvirt.connection [req-4202dc5f-022f-4c1b-bfbf-71dae31f5476 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] [instance: 79e7b88f-2302-4387-a27e-ffdbd483f6d2] Creating image<br>2012-09-21 17:54:59 DEBUG nova.utils [req-4202dc5f-022f-4c1b-bfbf-71dae31f5476 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Running cmd (subprocess): env LC_ALL=C LANG=C qemu-img info /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>2012-09-21 17:54:59 DEBUG nova.virt.images [req-4202dc5f-022f-4c1b-bfbf-71dae31f5476 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] 2fd9c9e7-115d-4a39-8ea3-3adc9f0dda68 was qcow2, converting to raw from (pid=28544) fetch_to_raw /usr/lib/python2.7/dist-packages/nova/virt/images.py:104<br>2012-09-21 17:54:59 DEBUG nova.utils [req-4202dc5f-022f-4c1b-bfbf-71dae31f5476 64d4d53b99f84cf1ba6becd72ed10982 844dc9fb4ad9499ea97b07c7b6cee137] Running cmd (subprocess): qemu-img convert -O raw /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.part /data/openstack/nova/instances/_base/3554d26efea057ec9dd8029a0a3ceddc0359b70d.converted from (pid=28544) execute /usr/lib/python2.7/dist-packages/nova/utils.py:219<br>2012-09-21 17:57:50 INFO nova.virt.libvirt.connection [-] [instance: 79e7b88f-2302-4387-a27e-ffdbd483f6d2] Instance spawned successfully. ###OK<br><br><br>_______________________________________________<br>Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br></blockquote></div><br></div></body></html>