[nova] Is It Possible to Attach a Cinder Volume as device='lun' ?
Hi, I'm running an OpenStack setup with Fibre Channel (FC) storage using Cinder volumes. I'm currently trying to configure a Windows Failover Cluster in guest VMs and have encountered an issue. Windows is querying the SCSI device for the Device Identification VPD (Page 0x83). The LUN is attached as a SCSI device using a virtio-scsi controller. In the domain XML (via libvirt), I see that the disk is defined as <disk type='block' device='disk'> also has a <serial>cinder-vol-uuid</serial> When Page 0x83 is queried, it returns the serial value (i.e., the cinder-vol-uuid), which Windows fails to recognize or accept. However, if I modify the XML to use: <disk type='block' device='lun'> and remove the <serial> tag, the guest is able to retrieve the expected information (I had to stop the compute service and restart the libvirt domain). Unfortunately, any changes made directly to the domain XML do not persist. Is there a supported way to specify device='lun' instead of device='disk' for volumes attached ? Any guidance would be greatly appreciated. Thank you With Best regards, Janaka
G'day Janaka, Disclaimer that our case was not identical to yours, but maybe you'll find this information useful: I went through a similar process recently - our cinder backend is ceph-rbd, rather than iscsi, but the limitation we ran into was that the ISCSI drive for virtio-scsi block driver does not support SCSI-3 PR, which is required for WFC to work/allow it to be set up/detected as a "valid" disk type. We were technically able to work around the disk-type issue and mount the device as a multipath lun, but due to the virtio driver not supporting SCSI-3 PR, we were not able to proceed further. There is a virtio-scsi driver for paravirtualised devices in KVM, but I do not believe it ships default Openstack. I have not yet found an open bug or patch or discussion about this, nor I have not gotten a POC working, but if you are keen to pursue it, there is an oracle blog about getting virtio-scsi working with persistent reservation in QEMU, and getting that working for WFC, which you might be able to apply that learning to your Openstack. https://blogs.oracle.com/linux/post/configuring-windows-failover-cluster-for... Kind Regards, Joel McLean – Micron21 Pty Ltd -----Original Message----- From: Janaka Wickramasinghe <janakawicks@gmail.com> Sent: Monday, 16 June 2025 1:36 PM To: openstack-discuss@lists.openstack.org Subject: [nova] Is It Possible to Attach a Cinder Volume as device='lun' ? Hi, I'm running an OpenStack setup with Fibre Channel (FC) storage using Cinder volumes. I'm currently trying to configure a Windows Failover Cluster in guest VMs and have encountered an issue. Windows is querying the SCSI device for the Device Identification VPD (Page 0x83). The LUN is attached as a SCSI device using a virtio-scsi controller. In the domain XML (via libvirt), I see that the disk is defined as <disk type='block' device='disk'> also has a <serial>cinder-vol-uuid</serial> When Page 0x83 is queried, it returns the serial value (i.e., the cinder-vol-uuid), which Windows fails to recognize or accept. However, if I modify the XML to use: <disk type='block' device='lun'> and remove the <serial> tag, the guest is able to retrieve the expected information (I had to stop the compute service and restart the libvirt domain). Unfortunately, any changes made directly to the domain XML do not persist. Is there a supported way to specify device='lun' instead of device='disk' for volumes attached ? Any guidance would be greatly appreciated. Thank you With Best regards, Janaka
Hi Joel, Thank you very much for your detailed and thoughtful response. I really appreciate you taking the time to share your experience and the references. I'll definitely look into it further to see possibility of getting this sorted. Thanks again for your support and for pointing me in the right direction. Cheers, Janaka On Mon, Jun 16, 2025 at 10:09 AM Joel McLean <joel.mclean@micron21.com> wrote:
G'day Janaka,
Disclaimer that our case was not identical to yours, but maybe you'll find this information useful: I went through a similar process recently - our cinder backend is ceph-rbd, rather than iscsi, but the limitation we ran into was that the ISCSI drive for virtio-scsi block driver does not support SCSI-3 PR, which is required for WFC to work/allow it to be set up/detected as a "valid" disk type. We were technically able to work around the disk-type issue and mount the device as a multipath lun, but due to the virtio driver not supporting SCSI-3 PR, we were not able to proceed further.
There is a virtio-scsi driver for paravirtualised devices in KVM, but I do not believe it ships default Openstack. I have not yet found an open bug or patch or discussion about this, nor I have not gotten a POC working, but if you are keen to pursue it, there is an oracle blog about getting virtio-scsi working with persistent reservation in QEMU, and getting that working for WFC, which you might be able to apply that learning to your Openstack.
https://blogs.oracle.com/linux/post/configuring-windows-failover-cluster-for...
Kind Regards,
Joel McLean – Micron21 Pty Ltd
-----Original Message----- From: Janaka Wickramasinghe <janakawicks@gmail.com> Sent: Monday, 16 June 2025 1:36 PM To: openstack-discuss@lists.openstack.org Subject: [nova] Is It Possible to Attach a Cinder Volume as device='lun' ?
Hi,
I'm running an OpenStack setup with Fibre Channel (FC) storage using Cinder volumes. I'm currently trying to configure a Windows Failover Cluster in guest VMs and have encountered an issue.
Windows is querying the SCSI device for the Device Identification VPD (Page 0x83). The LUN is attached as a SCSI device using a virtio-scsi controller. In the domain XML (via libvirt), I see that the disk is defined as <disk type='block' device='disk'> also has a <serial>cinder-vol-uuid</serial> When Page 0x83 is queried, it returns the serial value (i.e., the cinder-vol-uuid), which Windows fails to recognize or accept.
However, if I modify the XML to use: <disk type='block' device='lun'> and remove the <serial> tag, the guest is able to retrieve the expected information (I had to stop the compute service and restart the libvirt domain). Unfortunately, any changes made directly to the domain XML do not persist.
Is there a supported way to specify device='lun' instead of device='disk' for volumes attached ? Any guidance would be greatly appreciated.
Thank you With Best regards, Janaka
On 16/06/2025 06:27, Janaka Wickramasinghe wrote:
Hi Joel,
Thank you very much for your detailed and thoughtful response.
I really appreciate you taking the time to share your experience and the references. I'll definitely look into it further to see possibility of getting this sorted.
this has been partly fixed so without going into a lot of detail this used to work for a long time then qemu and livbirt changed from ignoring the serial of the device to makign it hard error for it to be in the xml/command line. that required nova to reqite how we mangage and identify block device to use libvirt alaias instead of serial number. https://review.opendev.org/q/topic:%22lun-device-serial%22 fixes type=lun for cinder volumes however this can only be set when booting a vm with a cinder volume you cannot use type=lun for voluems attached after the fact as the volume attachment api https://docs.openstack.org/api-ref/compute/#attach-a-volume-to-an-instance does not supprot setign that field. the device type can only be set via the block device mappigns when creating a new instnace. https://docs.openstack.org/nova/latest/user/block-device-mapping.html#block-... if your usign rbd storage provisioned by nova automaticaly im not sure if ti would work but you would also have to enable it in the bdm on instace create.
Thanks again for your support and for pointing me in the right direction.
Cheers, Janaka
On Mon, Jun 16, 2025 at 10:09 AM Joel McLean <joel.mclean@micron21.com> wrote:
G'day Janaka,
Disclaimer that our case was not identical to yours, but maybe you'll find this information useful: I went through a similar process recently - our cinder backend is ceph-rbd, rather than iscsi, but the limitation we ran into was that the ISCSI drive for virtio-scsi block driver does not support SCSI-3 PR, which is required for WFC to work/allow it to be set up/detected as a "valid" disk type. We were technically able to work around the disk-type issue and mount the device as a multipath lun, but due to the virtio driver not supporting SCSI-3 PR, we were not able to proceed further.
so it not actully that the virtio-scsi cant supprot iscsi v3 command it that it cnad do it without lun mode. although for ceph im not sure that lun mode actully is an option we have never tested that.
There is a virtio-scsi driver for paravirtualised devices in KVM, but I do not believe it ships default Openstack. I have not yet found an open bug or patch or discussion about this, nor I have not gotten a POC working, but if you are keen to pursue it, there is an oracle blog about getting virtio-scsi working with persistent reservation in QEMU, and getting that working for WFC, which you might be able to apply that learning to your Openstack.
https://blogs.oracle.com/linux/post/configuring-windows-failover-cluster-for...
so we have got this workign with the lvm backend for basic test using device_type=lun https://review.opendev.org/c/openstack/tempest/+/918457 we do not directly use the v3 capablites in that test but it was manually checked. allowing funcitonalty to be used more boradly iin openstack woudl be anew feature and require api changes. i.e. to allow ti to work when attaching a cidner volume after the fact woudl require exposign the device type in the attach request or adding new image properteis. neither is really backportable although it woudl be useful to have device_type on volume attach so that you could properly attach iso's to support driver disk or install isos exposign the boot priporty would also help with that use case but its not need for scsiv3 of lun mode works with ceph it may be worth considrign a config option for this for nova provisioned volumes but ther would be upgrade imapcts to that. in otherwords supporting this more broadly would need a nova spec.
Kind Regards,
Joel McLean – Micron21 Pty Ltd
-----Original Message----- From: Janaka Wickramasinghe <janakawicks@gmail.com> Sent: Monday, 16 June 2025 1:36 PM To: openstack-discuss@lists.openstack.org Subject: [nova] Is It Possible to Attach a Cinder Volume as device='lun' ?
Hi,
I'm running an OpenStack setup with Fibre Channel (FC) storage using Cinder volumes. I'm currently trying to configure a Windows Failover Cluster in guest VMs and have encountered an issue.
Windows is querying the SCSI device for the Device Identification VPD (Page 0x83). The LUN is attached as a SCSI device using a virtio-scsi controller. In the domain XML (via libvirt), I see that the disk is defined as <disk type='block' device='disk'> also has a <serial>cinder-vol-uuid</serial> When Page 0x83 is queried, it returns the serial value (i.e., the cinder-vol-uuid), which Windows fails to recognize or accept.
However, if I modify the XML to use: <disk type='block' device='lun'> and remove the <serial> tag, the guest is able to retrieve the expected information (I had to stop the compute service and restart the libvirt domain). Unfortunately, any changes made directly to the domain XML do not persist.
Is there a supported way to specify device='lun' instead of device='disk' for volumes attached ? Any guidance would be greatly appreciated.
Thank you With Best regards, Janaka
participants (3)
-
Janaka Wickramasinghe
-
Joel McLean
-
Sean Mooney