Hi,

have you compared the affected rbd images with working images? Maybe 
the mirroring failed for those images? Were they promoted correctly? 
Which mirror mode are you using, journal or snapshot? I would check 
the 'rbd info pool/image' output and compare to see if there's a 
difference.


Zitat von Satish Patel <satish.txt@gmail.com>:

> Folks,
>
> I have a ceph cluster and recently I configured rbd-mirror to replicate all
> data to remove ceph cluster for disaster recovery.
>
> Yesterday for POC we did a hard cutover on ceph and point openstack to the
> new cluster. All other vms came back up fine but 2 vms stuck in this error
> state in libvirt logs
>
> 2024-01-31 22:44:37.591+0000: 474597: error :
> qemuMonitorJSONCheckErrorFull:399 : internal error: unable to execute QEMU
> command 'query-named-block-nodes': cannot read image start for probe:
> Permission denied
>
> If it's a wider issue than it should impact all the vms but why only two VM
> stuck and not starting and libvirt giving me this error in logs
>
> nova-compute logs also showing same error
>
> 2024-01-31 22:44:40.925 7 INFO nova.compute.manager [None
> req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00
> ca5c652478c7429e964257990800e9cb - - default default] [instance:
> 2de0f880-77c7-4d2c-9e01-898c57ad3693] Successfully reverted task state from
> powering-on on failure for instance.
> 2024-01-31 22:44:40.944 7 ERROR oslo_messaging.rpc.server [None
> req-b33485b5-8740-48ae-8b5b-a440de3f11a4 c48fcfb9347f413f92fcece065644b00
> ca5c652478c7429e964257990800e9cb - - default default] Exception during
> message handling: libvirt.libvirtError: internal error: unable to execute
> QEMU command 'query-named-block-nodes': cannot read image start for probe:
> Permission denied