Do not worry, I know queens is very old. I am going to move volumes with another procedure. Ignazio
Il giorno gio 10 mar 2022 alle ore 13:58 Gorka Eguileor geguileo@redhat.com ha scritto:
On 07/03, Ignazio Cassano wrote:
Hello again I found the error but it is in cinder api log: 2022-03-07 15:42:57.330 156916 INFO cinder.api.openstack.wsgi [req-01eb2613-4aad-4809-ac91-037de3148fbb
5134ed3b93284af5ba2f05d7361edf53
0e760ccde5d24af5a571de40220fbf80 - default default]
http://10.138.144.16:8776/v3/0e760ccde5d24af5a571de40220fbf80/attachments/04...
returned with HTTP 404
and in nova compute log: 15:42:57.502 280431 ERROR oslo_messaging.rpc.server [req-dfeac1f0-870b-4aef-af40-cda52f4552e0
5134ed3b93284af5ba2f05d7361edf53
0e760ccde5d24af5a571de40220fbf80 - default default] Exception during message handling: VolumeAttachmentNotFound: Volume attachment 04ae01f6-e164-40cb-8d31-63f78e69c8db could not be found.
Hi,
Retyping with migration of attached volumes is a complex flow, so doing the RCA usually requires full logs of ALL involved services:
- All nova-api, cinder-api, cinder-scheduler services
- nova-cpu of the host with the instance
- Cinder-volume service
For example in the logs you gave above we don't even know if it was a GET or PUT operation and what was nova trying to do, and we also don't have context, so we cannot tell at which stage of the whole process this happened.
If I remember correctly the high level steps are something like:
- Cinder-volume receives request
- Creates new volume
- Calls nova to do the volume swap
- Nova attaches new volume with os-brick
- Nova asks the hypervisor to do the swap
- Nova disconnects the original volume
- Nova calls cinder telling it that the swap is completed
- Cinder detaches the original volume and makes DB changes
Since you are running a 4 years old code it's not easy to see the code differences to see if there was a bug that has been fixed in a later release.
Sorry for not being able to provide actual help. :-(
Cheers, Gorka.
Il giorno lun 7 mar 2022 alle ore 17:44 Ignazio Cassano < ignaziocassano@gmail.com> ha scritto:
Hello, volume.log does not report errors: 2022-03-07 15:41:44.435 159559 INFO cinder.volume.manager [req-68444973-24eb-4b21-95b2-626c9b2e2aa0
5134ed3b93284af5ba2f05d7361edf53
0e760ccde5d24af5a571de40220fbf80 - default default] Created volume successfully. 2022-03-07 15:41:45.358 159099 INFO cinder.volume.manager [req-68444973-24eb-4b21-95b2-626c9b2e2aa0
5134ed3b93284af5ba2f05d7361edf53
0e760ccde5d24af5a571de40220fbf80 - default default] Migrate volume completed successfully. 2022-03-07 15:41:48.970 159559 INFO cinder.volume.manager [req-72ca9242-25e9-4bb0-9bd6-9857f42805ee
5134ed3b93284af5ba2f05d7361edf53
0e760ccde5d24af5a571de40220fbf80 - default default] attachment_update complete
Il giorno lun 7 mar 2022 alle ore 17:25 Eugen Block eblock@nde.ag ha scritto:
Hi,
since you're retyping a volume, what do the cinder logs reveal?
Zitat von Ignazio Cassano ignaziocassano@gmail.com:
Hello, sometimes the volume retype from a netapp nfs storage to
another
netapp nfs storage does not work. I do not know the reason but I think something is going wrong in
nova:
/var/log/nova/nova-compute.log:2022-02-02 14:23:39.992 194288 ERROR
root
[req-ebbf749f-359f-4282-94c5-0913cffe97a3
66adb965bef64eaaab2af93ade87e2ca
68feefdf7c5847388534e9214dc13366 - default default] Original
exception
being dropped: ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line
5655,
in
_swap_volume\n mountpoint, resize_to)\n', ' File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line
1647,
in swap_volume\n self._disconnect_volume(context,
new_connection_info,
instance)\n', ' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line
220, in
__exit__\n self.force_reraise()\n', ' File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line
196, in
force_reraise\n six.reraise(self.type_, self.value, self.tb)\n',
'
File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line
1644,
in swap_volume\n self._swap_volume(guest, disk_dev, conf,
resize_to)\n',
' File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py",
line 1593, in _swap_volume\n raise exception.VolumeRebaseFailed(reason=six.text_type(exc))\n', "VolumeRebaseFailed: Volume rebase failed: Requested operation is
not
valid: pivot of disk 'vda' requires an active copy job\n"]: VolumeAttachmentNotFound: Volume attachment 2cd820e0-85e8-498d-a62a-800260d0cf31 could not be found
Must I try to change some parameters in nova.conf on compute nodes
about
volume attachment ?
Thanks Ignazio