Open Stack

Hello Openstack,

We have Ussuri deployed on a few clouds, and they're all plugged to 
PureStorage Arrays. We allow users to only use volumes for their 
servers. It means that each server disk is a LUN attached over ISCSI 
(with multipath) on the compute node hosting the server. Everything 
works quite fine, but we have a weird issue when extending volumes 
attached to running instances. The guests notice the new disk size .. of 
the last extent.

Say I have a server with a 10gb disk. I add 5gb. On the guest, still 
10gb. I add another 5gb, and on the guest I get 15, and so on. I've 
turned the debug mode on and I could see no error in the log. Looking 
closer at the log I could catch the culprit:

2022-12-08 17:35:13.998 46195 DEBUG os_brick.initiator.linuxscsi [] 
Starting size: *76235669504*
2022-12-08 17:35:14.028 46195 DEBUG os_brick.initiator.linuxscsi [] 
volume size after scsi device rescan *80530636800* extend_volume
2022-12-08 17:35:14.035 46195 DEBUG os_brick.initiator.linuxscsi [] 
Volume device info = {'device': 
'/dev/disk/by-path/ip-1...1:3260-iscsi-iqn.2010-06.com.purestorage:flasharray.x-lun-10', 
'host': '5', 'channel': '0', 'id': '0', 'lun': '10'} extend_volume
2022-12-08 17:35:14.348 46195 INFO os_brick.initiator.linuxscsi [] Find 
Multipath device file for volume WWN 3624...
2022-12-08 17:35:14.349 46195 DEBUG os_brick.initiator.linuxscsi [] 
Checking to see if /dev/disk/by-id/dm-uuid-mpath-3624.. exists yet. 
wait_for_path
2022-12-08 17:35:14.349 46195 DEBUG os_brick.initiator.linuxscsi [] 
/dev/disk/by-id/dm-uuid-mpath-3624... has shown up. wait_for_path
2022-12-08 17:35:14.382 46195 INFO os_brick.initiator.linuxscsi [] 
mpath(/dev/disk/by-id/dm-uuid-mpath-3624) *current size 76235669504*
2022-12-08 17:35:14.412 46195 INFO os_brick.initiator.linuxscsi [] 
mpath(/dev/disk/by-id/dm-uuid-mpath-3624) *new size 76235669504*
2022-12-08 17:35:14.413 46195 DEBUG oslo_concurrency.lockutils [] Lock 
"extend_volume" released by 
"os_brick.initiator.connectors.iscsi.ISCSIConnector.extend_volume" :: 
held 2.062s inner 2022-12-08 17:35:14.459 46195 DEBUG 
os_brick.initiator.connectors.iscsi [] <== extend_volume: return 
(2217ms) *76235669504* trace_logging_wrapper
2022-12-08 17:35:14.461 46195 DEBUG nova.virt.libvirt.volume.iscsi [] 
Extend iSCSI Volume /dev/dm-28; new_size=*76235669504* extend_volume
2022-12-08 17:35:14.462 46195 DEBUG nova.virt.libvirt.driver [] Resizing 
target device /dev/dm-28 to *76235669504* _resize_attached_volume

The logs clearly shows that the rescan confirm the new size but when 
interrogating multipath, it does not. But requesting multipath few 
seconds after on the command line shows the new size as well. It 
explains the behaviour.

I'm running Ubuntu 18.04 with multipath 0.7.4-2ubuntu3.2. The os-brick 
code for multipath is far more basic than the one in master branch. 
Maybe the multipath version installed is too recent for os-brick.

Thanks for the help

Jerome

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20221208/3b4c6199/attachment.htm>