Open Stack

On Wed, 2023-08-09 at 13:59 +0200, Sven Kieske wrote:
> Hi,
> 
> I can't cover everything here, because performance is a huge topic, but
> here are some questions which I didn't find the answer to:
> 
> which nvme is this? is this a consumer device by chance?
> 
> which openstack release are you running, which hypervisor os and which
> guest os and kernel versions?
> 
> which deployment method do you use?
> 
> > *[5] image properties:
> > https://docs.ceph.com/en/pacific/rbd/rbd-openstack/#image-properties
> > <https://docs.ceph.com/en/pacific/rbd/rbd-openstack/#image-properties
> 
> at least the ceph recommendations for virtio-scsi are somewhat outdated
> as virtio-blk is, depending on the benchmark you look at - faster and
> also supports discard (it's a well hidden secret, but it's true).
discard support depend on your distro and machine type

on rhel-9/centos 9 stream this is only true if using q35 machine type
the only pc-i440fx machine type in centos 9 is form centso 7 and that predates
that feature landing in qemu so if you use the pc/pc-i440fx manchie type
virtio-blk does not support discard. on debian based distos since they ship
the upstream qemu machine  types this is not an issue. it will work with pc
too.

in general the reason to use virtio-scsi for ceph is that it allow more volues to be
used as each one does not consume a pci devce unlike virtio-blk. so if you need many
disks use virtio-scsi if you need less then 30 ish then use virtio-blk.

nova does not curretly supprrt multiqueue or iothread which are likely required to fully
use nvme ssds. we did start workign on a proposal for that but the person working on it move
to a diffent role so that is likely something we will look at again for the next deveolpment cycle.


> 
> I'd test with virtio-blk first and make sure deadline scheduler is used
> inside the vm and caching should be none.
> 
> HTH
>