[Openstack] Cinder migrate between pools fails

Mārtiņš Jakubovičs martins-lists at hostnet.lv
Fri May 6 09:35:42 UTC 2016


Hello all,

I try to test cinder migration between two storage pools and so far 
without success.

What I have:

~$ cinder get-pools
+----------+----------------------------------------------------------------------------------------+
| Property | Value                                          |
+----------+----------------------------------------------------------------------------------------+
|   name   | 
glance.os.local at sata-generic#10.10.10.140:/vol/Cinder_SATA_1_vol_30032016_174641 
|
+----------+----------------------------------------------------------------------------------------+
+----------+----------------------------------------------------------------------------------------+
| Property | Value                                          |
+----------+----------------------------------------------------------------------------------------+
|   name   | 
glance.os.local at sata-generic#10.10.10.140:/vol/Cinder_SATA_2_vol_05052016_154020 
|
+----------+----------------------------------------------------------------------------------------+

And detached volume in first pool.
Basically they are two different NFS shares in same host. When I start 
migrate, in cinder-volume host starts dd operation, which is fine, but 
in the end migration fails with error.
So does cinder migration works between same storage but different pools?

http://pastebin.com/E74ff1Xn

2016-05-06 11:45:00.941 5348 INFO cinder.volume.utils 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Volume copy 1024.00 MB at 92.26 MB/s
2016-05-06 11:45:01.005 5348 INFO cinder.volume.manager 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Terminate volume connection completed successfully.
2016-05-06 11:45:01.151 5348 INFO cinder.volume.manager 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Terminate volume connection completed successfully.
2016-05-06 11:45:01.230 5348 INFO cinder.volume.manager 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Remove volume export completed successfully.
2016-05-06 11:45:01.246 5348 INFO cinder.volume.manager 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Remove volume export completed successfully.
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Failed to copy volume 79e19f6c-ee9d-48ea-aa0c-37fef899c592 to 
0b09aea3-93fe-4a34-9a14-23120e435a7f
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager Traceback (most 
recent call last):
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1774, 
in _migrate_volume_generic
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager error=False)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1886, 
in migrate_volume_completion
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager orig_volume_status)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/volume/rpcapi.py", line 367, in 
update_migrated_volume
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
volume_status=original_volume_status)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 
145, in call
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager     msg = 
self._make_message(ctxt, method, kwargs)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 
106, in _make_message
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
msg['args'][argname] = self.serializer.serialize_entity(ctxt, arg)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/rpc.py", line 118, in 
serialize_entity
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager     return 
self._base.serialize_entity(context, entity)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/objects/base.py", line 429, in 
serialize_entity
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager     entity = 
entity.obj_to_primitive(backport_ver)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
548, in obj_to_primitive
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager version_manifest)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
520, in obj_make_compatible_from_manifest
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager     return 
self.obj_make_compatible(primitive, target_version)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/cinder/objects/volume.py", line 226, 
in obj_make_compatible
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager super(Volume, 
self).obj_make_compatible(primitive, target_version)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
507, in obj_make_compatible
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
self._obj_make_obj_compatible(primitive, target_version, key)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
459, in _obj_make_obj_compatible
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
relationship_map = self._obj_relationship_for(field, target_version)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager   File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
436, in _obj_relationship_for
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager reason='No rule 
for %s' % field)
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager 
ObjectActionError: Object action obj_make_compatible failed because: No 
rule for volume_type
2016-05-06 11:45:01.411 5348 ERROR cinder.volume.manager
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Exception during message handling: Object action obj_make_compatible 
failed because: No rule for volume_type
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
Traceback (most recent call last):
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 138, in _dispatch_and_reply
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
incoming.message))
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 185, in _dispatch
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher return 
self._do_dispatch(endpoint, method, ctxt, args)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 127, in _do_dispatch
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher result 
= func(ctxt, **new_args)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 3545, 
in migrate_volume
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
new_type_id=new_type_id, volume=volume)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1982, 
in migrate_volume
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
volume.save()
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in 
__exit__
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
self.force_reraise()
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in 
force_reraise
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
six.reraise(self.type_, self.value, self.tb)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1975, 
in migrate_volume
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
new_type_id)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1790, 
in _migrate_volume_generic
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher new_volume)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 220, in 
__exit__
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
self.force_reraise()
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 196, in 
force_reraise
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
six.reraise(self.type_, self.value, self.tb)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1774, 
in _migrate_volume_generic
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
error=False)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1886, 
in migrate_volume_completion
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
orig_volume_status)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/volume/rpcapi.py", line 367, in 
update_migrated_volume
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
volume_status=original_volume_status)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 
145, in call
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher msg = 
self._make_message(ctxt, method, kwargs)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/client.py", line 
106, in _make_message
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
msg['args'][argname] = self.serializer.serialize_entity(ctxt, arg)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/rpc.py", line 118, in 
serialize_entity
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher return 
self._base.serialize_entity(context, entity)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/objects/base.py", line 429, in 
serialize_entity
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher entity 
= entity.obj_to_primitive(backport_ver)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
548, in obj_to_primitive
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
version_manifest)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
520, in obj_make_compatible_from_manifest
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher return 
self.obj_make_compatible(primitive, target_version)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/cinder/objects/volume.py", line 226, 
in obj_make_compatible
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
super(Volume, self).obj_make_compatible(primitive, target_version)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
507, in obj_make_compatible
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
self._obj_make_obj_compatible(primitive, target_version, key)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
459, in _obj_make_obj_compatible
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
relationship_map = self._obj_relationship_for(field, target_version)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher File 
"/usr/lib/python2.7/dist-packages/oslo_versionedobjects/base.py", line 
436, in _obj_relationship_for
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
reason='No rule for %s' % field)
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher 
ObjectActionError: Object action obj_make_compatible failed because: No 
rule for volume_type
2016-05-06 11:45:01.688 5348 ERROR oslo_messaging.rpc.dispatcher
2016-05-06 11:45:01.915 5348 INFO os_brick.remotefs.remotefs 
[req-6dd52d1a-b6a0-4f68-a0f5-f42f108cb191 
96354895967c45cdbc57e28e4fbad851 45861366ce354525bb24703292ee3858 - - -] 
Already mounted: /var/lib/cinder/mnt/5a6a1a8bffc1eab46d9bd2ef74a08caf

Best regards,
Martins




More information about the Openstack mailing list