[kolla][wallaby][cinder] netapp driver failed do create volume from images
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101 Any help please ? Ignazio
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide. On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
You can try a few options to see if it helps. It might be a question of NFSv3 or V4 or the Netapp driver changes themselves. https://forum.opennebula.io/t/nfs-v3-datastore-and-failed-to-lock-byte-100/7... On Fri, Oct 8, 2021 at 8:51 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio
Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
I will try next week and write test results. Thanks Il giorno ven 8 ott 2021 alle ore 15:07 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
You can try a few options to see if it helps. It might be a question of NFSv3 or V4 or the Netapp driver changes themselves.
https://forum.opennebula.io/t/nfs-v3-datastore-and-failed-to-lock-byte-100/7...
On Fri, Oct 8, 2021 at 8:51 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio
Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
Pingback from other thread Does it work with a Netapp/Ontap NFS volume now? On Fri, Oct 8, 2021 at 10:18 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
I will try next week and write test results. Thanks
Il giorno ven 8 ott 2021 alle ore 15:07 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
You can try a few options to see if it helps. It might be a question of NFSv3 or V4 or the Netapp driver changes themselves.
https://forum.opennebula.io/t/nfs-v3-datastore-and-failed-to-lock-byte-100/7...
On Fri, Oct 8, 2021 at 8:51 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio
Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano < ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
Yes, I removed cinder containers. I deployed cinder again. After the above procedure, netapp volume changed the error: cannot find valid backend. So after the reinstallarion I was not able to create netapp volume at all. I inserted the parameter I mentioned in my previous email, and now I am able to create both empty and from image netapp volumes. Ignazio Il Ven 8 Ott 2021, 23:00 Laurent Dumont <laurentfdumont@gmail.com> ha scritto:
Pingback from other thread
Does it work with a Netapp/Ontap NFS volume now?
On Fri, Oct 8, 2021 at 10:18 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
I will try next week and write test results. Thanks
Il giorno ven 8 ott 2021 alle ore 15:07 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
You can try a few options to see if it helps. It might be a question of NFSv3 or V4 or the Netapp driver changes themselves.
https://forum.opennebula.io/t/nfs-v3-datastore-and-failed-to-lock-byte-100/7...
On Fri, Oct 8, 2021 at 8:51 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio
Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano < ignaziocassano@gmail.com> wrote:
Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101
Any help please ? Ignazio
Sorry, I sent the solution in a wrong thread. I solved inserting default_filter=Driver_Filters in cinder.conf. Probably it solved because I have both netapp and ceph backends. Ignazio Il Sab 9 Ott 2021, 10:07 Ignazio Cassano <ignaziocassano@gmail.com> ha scritto:
Yes, I removed cinder containers. I deployed cinder again. After the above procedure, netapp volume changed the error: cannot find valid backend. So after the reinstallarion I was not able to create netapp volume at all. I inserted the parameter I mentioned in my previous email, and now I am able to create both empty and from image netapp volumes. Ignazio
Il Ven 8 Ott 2021, 23:00 Laurent Dumont <laurentfdumont@gmail.com> ha scritto:
Pingback from other thread
Does it work with a Netapp/Ontap NFS volume now?
On Fri, Oct 8, 2021 at 10:18 AM Ignazio Cassano <ignaziocassano@gmail.com> wrote:
I will try next week and write test results. Thanks
Il giorno ven 8 ott 2021 alle ore 15:07 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
You can try a few options to see if it helps. It might be a question of NFSv3 or V4 or the Netapp driver changes themselves.
https://forum.opennebula.io/t/nfs-v3-datastore-and-failed-to-lock-byte-100/7...
On Fri, Oct 8, 2021 at 8:51 AM Ignazio Cassano < ignaziocassano@gmail.com> wrote:
Hello Laurent, I am using nfs_mount_options = nfsvers=3,lookupcache=pos I always use the above options. I have this issue only with the last cinder images of wallaby Thanks Ignazio
Il giorno ven 8 ott 2021 alle ore 14:46 Laurent Dumont < laurentfdumont@gmail.com> ha scritto:
What options are you using for the NFS client on the controllers side? There are some recommended settings that Netapp can provide.
On Fri, Oct 8, 2021 at 7:20 AM Ignazio Cassano < ignaziocassano@gmail.com> wrote:
> Hello, > I've just updated my kolla wallaby with latest images. When I create > volume from image on ceph it works. > When I create volume from image on nfs netapp ontap, it does not > work. > The following is reported in cinder-volume.log: > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", > line 950, in _create_from_image_cache_or_download > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server > model_update = self._create_from_image_download( > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", > line 766, in _create_from_image_download > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server > volume_utils.copy_image_to_volume(self.driver, context, volume, > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File > "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", > line 1158, in copy_image_to_volume > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise > exception.ImageCopyFailure(reason=ex.stderr) > 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server > cinder.exception.ImageCopyFailure: Failed to copy image to volume: > qemu-img: > /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: > error while converting raw: Failed to lock byte 101 > > Any help please ? > Ignazio >
participants (2)
-
Ignazio Cassano
-
Laurent Dumont