Hello Stephen, Yes, OpenStack does support USB passthrough. You can pass through USB devices to instances running in OpenStack by using the nova.virt.libvirt.vif.LibvirtGenericVIFDriver driver and the qemu:commandline option in the nova.conf file. Best, Kerem Çeliker keremceliker.medium.com Sent from my iPhone
On 4 Jan 2023, at 02:25, openstack-discuss-request@lists.openstack.org wrote:
Send openstack-discuss mailing list submissions to openstack-discuss@lists.openstack.org
To subscribe or unsubscribe via the World Wide Web, visit https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss
or, via email, send a message with subject or body 'help' to openstack-discuss-request@lists.openstack.org
You can reach the person managing the list at openstack-discuss-owner@lists.openstack.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of openstack-discuss digest..."
Today's Topics:
1. Re: [nova] Do openstack support USB passthrough (Sean Mooney) 2. Re: [ops] QOS on flavor breaking live migration from CentOS 7 to 8 (Sean Mooney) 3. Re: [cloudkitty] Instances billing based on tags (Satish Patel) 4. [nova][keystone] workload identity? (Mohammed Naser) 5. Re: [cloudkitty] Instances billing based on tags (Rafael Weing?rtner)
----------------------------------------------------------------------
Message: 1 Date: Tue, 03 Jan 2023 19:03:23 +0000 From: Sean Mooney <smooney@redhat.com> To: Stephen Finucane <stephenfin@redhat.com>, ??? <hanguangyu2@gmail.com>, openstack-discuss <openstack-discuss@lists.openstack.org> Subject: Re: [nova] Do openstack support USB passthrough Message-ID: <765850d1c170a510d34dafb4253ab97528829351.camel@redhat.com> Content-Type: text/plain; charset="UTF-8"
On Tue, 2023-01-03 at 14:54 +0000, Stephen Finucane wrote:
On Mon, 2022-12-26 at 10:04 +0000, ??? wrote: Hi, all
I want to ask if openstack support USB passthrough now?
Or if I want the instance to recognize the USB flash drive on the host, do you have any suggestions?
Thanks, Han
This isn't supported in nova and probably never will be. The closest you can get is to passthrough an entire USB controller as suggested by this blog [1], but that's really a hack and I 100% would not use it in production.
ya so we have discussed usb passthough supprot a few times and its somethign nova could add but there has neither been the demand or desire to add it stongly enough in the core team to actully do it.
the shorted path to enableing usb passthoug would likely be to add support to cyborg and then add support for that ot nova. i am perhaps the most open of the nova cores to supporting usb passthough since i have wanted to add it in the past but if we were to support it it would have to be similar to howe we support pci passhtough. static provisioning and likely only of stateless devices which rules out usb falsh drives.
usb gps recivers for precision time stamping was one of the usecause raised in the past which we were somewhat open too the other was usb programmers/debuggers forh cases when vms where used in industral test and automation workflows.
as stephen said the only way to do it today is to piggyback on pci passthough and passhtough a usb contoller not a single device.
if we were to ever support this in nova directly i would proably extend the pci tracker or support other buses like usb or use the generic resouce table created for persistent memory to model the devices. in eitehr case we would want this capablity to be placement native from the start if we added this capablity so it would be more and less work then you might imagine to do this right. less work if we maintain the requirement for statless devices only (ie no usb flash drives) more if you also need to handel multi tenancy and move operation include data copying, erasure and or encypetion.
i would not expect this to change in the next few release unless multiple operators provide feedback that this is a broadly deired capablity. with out a top level generic device api for mutple type of devices (vgpu, usb, pci) that was decoupled form the flaovr or an abstraction like the cyborg device-profile or pci alias it is hard to see a clean way to model this in our api. that is why enabling it in cyborg and then extneding nova ot support device profiles with a device type of usb is the simplar solution form a nova perspecitve but that is non trivial from an operational perspective as you requrie cyborg to utilise the feature. doing it via a usb_alias in the flavor has all the draw backs of the pci_alias, static configuration that must match on all compute nodes and futher proliferation of flavor explostion. this is one of the reasons we have not added this in the past. the work to do it in the libvirt driver would not be hard but the maintaince and operational overhead of using it for operators is non trivial.
Stephen
------------------------------
Message: 2 Date: Tue, 03 Jan 2023 19:20:50 +0000 From: Sean Mooney <smooney@redhat.com> To: Jahson Babel <jahson.babel@cc.in2p3.fr>, openstack-discuss@lists.openstack.org Subject: Re: [ops] QOS on flavor breaking live migration from CentOS 7 to 8 Message-ID: <b0f13cde177cd8b5f7e6e5b20445decab7dbc8b8.camel@redhat.com> Content-Type: text/plain; charset="UTF-8"
hi yes this is a know issue.
so the simple answer is resize all affected vms instead of live migrating them the longer answer is we have been dissing this internally at redhat on and off for some time now. https://bugs.launchpad.net/nova/+bug/1960840 is one example where this happens.
there is another case for the cpu based quotas that happens when going form rhel/centos 8->9 basically in the 8->9 change the cgroups implemantion changes form v1 to v2 https://bugzilla.redhat.com/show_bug.cgi?id=2035518
when adressing that we did not have a good universal solution for instnace that hardcoded a value that was incompatible with the cgroups_v2 api in the kernel except resize.
in https://review.opendev.org/c/openstack/nova/+/824048/ we removed automatically adding the cpu_shares cgroup option to enable booting vms with more then 8 cpus
we did not come up with any option other then resize for the other quotas that were in a similar situation. the one option that we considerd possibel to do was extend nova-mange to allow the embeded flaour to be updated this would be similar to what we did to enable the image property to be modifed for chaing machine types.
https://docs.openstack.org/nova/latest/cli/nova-manage.html#image-property-c...
we didcussed at the time that while we did not want to allow falvor extra specs to be modifed we might recondier that if the quota issue forced our hand or we had a similar need due to foces beyond our contol. i.e. we needed to provide a way beyond resize e.g. due ot operating system changes. what make image properties and flavor extra spec different is that image proerties can only be updated by rebuild which is a destructive operation. extra specs are upsted by resize which is not a destructive operation. that is one of the reasons we have special considertion to image properties and did not do the same for extra specs.
if we allow the same for flavor extra specs you would still have to stop the instance, make the change and then migrate the instnace resize automates that so it is generall a better fit. we were also conceren that adding it to nova manage would result in it being abused to modify instnace in ways that were either invalid for the host(changing the numa toplogy, adding traits/resouce request not trackedcxd in placemnt) or otherwise break the instnace in weird ways. that could happen via image properites too but its less likely.
On Tue, 2023-01-03 at 17:25 +0100, Jahson Babel wrote: Hello,
I'm trying to live migrate some VMs from CentOS 7 to Rocky 8. Everything run smoothly when there is no extra specs on flavors but things getting more complicated when those are fixed. Especially when using quota:vif_burst for QOS. I know that we aren't supposed to use this for QOS now but it's an old cluster and it was done that way at the time. So VMs kinda have all those specs tied to them.
When live migrate a VM this show up in the nova's logs : driver.py _live_migration_operation nova.virt.libvirt.driver? Live Migration failure: internal error: Child process (tc class add dev tapxxxxxxxx-xx parent 1: classid 1:1 htb rate 250000kbps ceil 2000000kbps burst 60000000kb quantum 21333) unexpected exit status 1: Illegal "burst" This bug cover the problem : https://bugs.launchpad.net/nova/+bug/1960840 So it's seems to be a normal behavior. Plus I forgot to mention that I'm on OpenStack Train version and the file mentioned in the launchpad is not present for this version. By using Rocky 8 I have to use an updated libvirt that won't accept the burst parameter we used to set. All available versions of libvirt on Rocky 8 have changed behavior concerning the burst parameter.
I've done some testing to make things works including removing the extra_specs on flavors and in the DB, removing it through libvirt and trying to modify tc rules used by a VM but it didn't worked. I have not tried yet to patch Nova or Libvirt but I don't really know where to look for. The only thing that did work was to resize the VM to an identical flavor without the extra_specs. But this induce a complete reboot of the VM. I would like, if possible, to be able to live migrate the VMs which is quite easier.
Is it possible to remove the extra_specs on the VMs and then live migrate ? Or should I just plan to resize/reboot all VMs without those extra_specs ? Any advise will be appreciated.
Thank you for any help, Best regards.
Jahson
------------------------------
Message: 3 Date: Tue, 3 Jan 2023 15:23:19 -0500 From: Satish Patel <satish.txt@gmail.com> To: Rafael Weing?rtner <rafaelweingartner@gmail.com> Cc: OpenStack Discuss <openstack-discuss@lists.openstack.org> Subject: Re: [cloudkitty] Instances billing based on tags Message-ID: <CAPgF-fpGyO-Z8bt7KMdP7CAhi9kne3136ptoE+8coURyFwi7yw@mail.gmail.com> Content-Type: text/plain; charset="utf-8"
Wow! very interesting, I will poke around and see how it's feasible. Very curious how it will represent that data in horizon UI. Currently I am seeing rates based on project_id so assuming it will show based on customer_id. correct?
On Tue, Jan 3, 2023 at 8:13 AM Rafael Weing?rtner < rafaelweingartner@gmail.com> wrote:
You can do that. Basically, you can start collecting those attributes you want for billing (e.g. tags) via Ceilometer dynamic pollster (that is the easiest way to achieve this). Then, you need to configure the resource type in Gnocchi to store this extra attribute, and of course, configure CloudKitty to collect/use it. Both in the metrics.yml and then in the hashmap or Pyscript rules.
On Mon, Jan 2, 2023 at 11:01 PM Satish Patel <satish.txt@gmail.com> wrote:
Folks,
We have an AWS project and in a single project we run multiple customers so for billing we use tags. In short every vm instance has a tag (customer name) and that way we can produce bills for each customer.
Recently I am playing with openstack cloudkitty and it works with standard cases like project based billing. But does it support tags based billing just similar to what i have explained in the above aws example?
-- Rafael Weing?rtner