Open Stack

On Thu, 2022-01-06 at 00:12 -0600, Eric K. Miller wrote:
> Hi,
> 
>  
> 
> I haven't found anything that indicates Nova supports adding iothreads
> parameters to the Libvirt XML file.  I had asked various performance
> related questions a couple years back, including asking if iothreads
> were available, but I didn't get any response (so assumed the answer was
> no).  So I'm just checking again to see if this has been a consideration
> to help improve a VM's storage performance - specifically with extremely
> high-speed storage in the host.
hi up until recently the advice from our virt team was that iotread where not really needed
for openstack howver in the last 6 weeks they have actully asked us to consider enabling them

so work will be happening in qemu/libvirt to always create at least one iothread going forward and
affinies it to the same set of cores as the emulator threads by default.

we dont have a downstream rfe currently filed for ioithread specifically but we do virtio scsi multi queue support
https://bugzilla.redhat.com/show_bug.cgi?id=1880273
i was proposing that we also enable iotread support as part of that work but we have not currently internaly
piroited it for any upstream release. enable support for iotrhead and virtio multiqueue i think makes a lot of sense
to do together. my understanding is that without iothread multi queue virtio scsi does not provide as much of
a perfromace boost as with io threads.

if you our other have capasity to work on this i would be happy to work on a spec with ye to enable it.
effectivly what i was plannign to propose if we got around to it is adding a new config option
cpu_iothread_set which would default to the same value as cpu_share_set.
this effectivly will ensure that witout any config updates all existing deployment will start benifiting
form iothreads and allow you to still dedicate a set of cores to running the iothread seperate form the cpu_share_set 
if you wasnt this to also benifit floating vms not just pinned vms.

in addtion to that a new flavor extra spec/image property woudl be added similar to cpu_emultor_threads.

im not quite sure how that extra spec should work but either 
hw:cpu_iotread_policy woudl either support the same vales as hw:cpu_emulator_threads where
hw:cpu_iotread_policy=shared woudl allocate an iotread that floats over the cpu_iothread_set (which is the same as cpu_shared_set by default)
and hw:cpu_iotread_policy=isolate would allocate an addtional iothread form the cpu_dedicated_set.
hw:cpu_iotread_policy=share woudl be the default behavior if cpu_shared_set or cpu_iothread_set was defined in the config and not flavor extra
spec or image property was defiend. basically all vms woudl have at least 1 iothread that floated over teh shared pool if a share pool was configured
on the host.

that is option a
option b woudl be to allso support

hw:cpu_iotread_count so you could ask for n iothread eitehr form the shared/iothread set or dedicated set depending on the value of
hw:cpu_iotread_policy

im not really sure if there is a need for more the 1 io thread. my understanding is that once you have at least 1 there is demising retruns.
it will improve your perfoamce if you have more propvided you have multiple disks/volumes attached but not as much as having the initall iotread.


is this something you wold be willing to work on and implement?
i woudl be happy to review any spec in this areay and i can bring it up downstream again but i cant commit to working on this in the z release.
this would require some minor rpc chagnes to ensure live migrate work properly as the iothread set or cpu share set could be different on different
hosts. but beyond that the feature is actully pretty simple to enable.
> 
>  
> 
> Or is there a way to add iothread-related parameters without Nova being
> involved (such as modifying a template)?
no there is no way to enable them out of band of nova today.
you technially could wrap the qemu binary wiht a script that inject parmaters but that obviously would not be supported upstream.
but that would be a workaround if you really needed it 

https://review.opendev.org/c/openstack/devstack/+/817075 is an exmaple of such a script
that break apparmor and selinx but you could proably make it work with enough effort.
although i woudl sugess just implemeting the feature upstream and downing a downstream backport instead.
> 
>  
> 
> Thanks!
> 
>  
> 
> Eric
>