Hello all,
On 16/02/2021 22:32, Christian Rohmann wrote:
I have just been trying to reproduce the issue, but in all my new attempts it just worked as expected:
[162262.926512] sd 0:0:0:1: Capacity data has changed [162262.932868] sd 0:0:0:1: [sdb] 6291456 512-byte logical blocks: (3.22 GB/3.00 GiB) [162262.933061] sdb: detected capacity change from 2147483648 to 3221225472
On 17/02/2021 09:43, Tobias Urdin wrote:
Regarding the actual extending of in-use volumes, we had an issue where cinder could not talk to os-server-external-events endpoint for nova because it used the wrong endpoint when looking up in keystone. We saw the error in cinder-volume.log except for that I can't remember we did anything special.
Had to use newer microversion for cinder when using CLI. cinder --os-volume-api-version 3.42 extend <volume ID or name> <new size in GB>
I am very sorry for the delay, but I was now finally able to reproduce the issue and with a VERY strange finding:
1)
* When using the cinder client with a Volume API version >= 3.42 like you suggested Tobias, it works just fine using cloud admin credentials. * The volume is attached / in-use, but it is resized just fine, including the notification of the kernel on the VM.
2)
* When attempting the same thing using the project users credentials the resize also works just fine, volume still attached and in-use, but then the VM is NOT notified * Also this does not seems to be related to nova or QEMU, but rather it appears there is no extend_volume event triggered or at least logged:
--- cut --- Request ID -- Action -- Start Time -- User ID -- Message
req-a4065b2d-1b77-4c5a-bf53-a2967b574fa0 extend_volume May 5, 2021, 1:22 p.m. 784bd2a5b82c3b31eb56ee - req-5965910b-874f-4c7a-ab61-32d1a080d1b2 attach_volume May 5, 2021, 1:09 p.m. 4b2abc14e511a7c0b10c - req-75ef5bd3-75d3-4146-84eb-1809789b6586 Create May 5, 2021, 1:09 p.m. 4b2abc14e511a7c0b10c - --- cut ---
UserID "784bd2a5b82c3b31eb56ee" is the regular user creating and attaching the volume. But that user also did an extend_volume, which is not logged as an event. There also was no API errors reported back to the client, the resize did happen - just not propagated to the VM - so a stop and restart was required.
But the admin user with id "4b2abc14e511a7c0b10c" doing a resize attempt caused an extend_volume and consequently did trigger a notification of the VM, just as expected and documented in regards to this feature.
Does anybody have any idea what could cause this or where to look for more details? Regards
Christian