Open Stack

Hello Stephen,

Yes, OpenStack does support USB passthrough. You can pass through USB devices to instances running in OpenStack by using the nova.virt.libvirt.vif.LibvirtGenericVIFDriver driver and the qemu:commandline option in the nova.conf file.

Best,
Kerem Çeliker
keremceliker.medium.com
Sent from my iPhone

> On 4 Jan 2023, at 02:25, openstack-discuss-request at lists.openstack.org wrote:
> 
> Send openstack-discuss mailing list submissions to
>    openstack-discuss at lists.openstack.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
>    https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss
> 
> or, via email, send a message with subject or body 'help' to
>    openstack-discuss-request at lists.openstack.org
> 
> You can reach the person managing the list at
>    openstack-discuss-owner at lists.openstack.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of openstack-discuss digest..."
> 
> 
> Today's Topics:
> 
>   1. Re: [nova] Do openstack support USB passthrough (Sean Mooney)
>   2. Re: [ops] QOS on flavor breaking live migration from CentOS 7
>      to 8 (Sean Mooney)
>   3. Re: [cloudkitty] Instances billing based on tags (Satish Patel)
>   4. [nova][keystone] workload identity? (Mohammed Naser)
>   5. Re: [cloudkitty] Instances billing based on tags
>      (Rafael Weing?rtner)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Tue, 03 Jan 2023 19:03:23 +0000
> From: Sean Mooney <smooney at redhat.com>
> To: Stephen Finucane <stephenfin at redhat.com>,  ???
>    <hanguangyu2 at gmail.com>, openstack-discuss
>    <openstack-discuss at lists.openstack.org>
> Subject: Re: [nova] Do openstack support USB passthrough
> Message-ID:
>    <765850d1c170a510d34dafb4253ab97528829351.camel at redhat.com>
> Content-Type: text/plain; charset="UTF-8"
> 
>> On Tue, 2023-01-03 at 14:54 +0000, Stephen Finucane wrote:
>>> On Mon, 2022-12-26 at 10:04 +0000, ??? wrote:
>>> Hi, all
>>> 
>>> I want to ask if openstack support USB passthrough now?
>>> 
>>> Or if I want the instance to recognize the USB flash drive on the
>>> host, do you have any suggestions?
>>> 
>>> Thanks,
>>> Han
>> 
>> This isn't supported in nova and probably never will be. The closest you can get
>> is to passthrough an entire USB controller as suggested by this blog [1], but
>> that's really a hack and I 100% would not use it in production.
> 
> ya so we have discussed usb passthough supprot a few times and its somethign nova could add but there has
> neither been the demand or desire to add it stongly enough in the core team to actully do it.
> 
> the shorted path to enableing usb passthoug would likely be to add support to cyborg and then add support for that ot nova.
> i am perhaps the most open of the nova cores to supporting usb passthough since i have wanted to add it in the past but
> if we were to support it it would have to be similar to howe we support pci passhtough. static provisioning and likely only of
> stateless devices which rules out usb falsh drives.
> 
> usb gps recivers for precision time stamping was one of the usecause raised in the past which we were somewhat open too
> the other was usb programmers/debuggers forh cases when vms where used in industral test and automation workflows.
> 
> as stephen said the only way to do it today is to piggyback on pci passthough and passhtough a usb contoller not a single device.
> 
> if we were to ever support this in nova directly i would proably extend the pci tracker or support other buses like usb or use the generic
> resouce table created for persistent memory to model the devices. in eitehr case we would want this capablity to be placement native from the
> start if we added this capablity so it would be more and less work then you might imagine to do this right.
> less work if we maintain the requirement for statless devices only (ie no usb flash drives) more if you also need to handel multi tenancy and
> move operation include data copying, erasure and or encypetion.
> 
> i would not expect this to change in the next few release unless multiple operators provide feedback that this is a broadly deired capablity.
> with out a top level generic device api for mutple type of devices (vgpu, usb, pci) that was decoupled form the flaovr or an abstraction like
> the cyborg device-profile or pci alias it is hard to see a clean way to model this in our api. that is why enabling it in cyborg and then extneding
> nova ot support device profiles with a device type of usb is the simplar solution  form a nova perspecitve but that is non trivial from an operational
> perspective as you requrie cyborg to utilise the feature. doing it via a usb_alias in the flavor has all the draw backs of the pci_alias, static
> configuration that must match on all compute nodes and futher proliferation of flavor explostion. this is one of the reasons we have not added this in
> the past.  the work to do it in the libvirt driver would not be hard but the maintaince and operational overhead of using it for operators is non
> trivial.
> 
>> 
>> Stephen
>> 
>> [1] https://egallen.com/openstack-usb-passthrough/
>> 
>> 
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Tue, 03 Jan 2023 19:20:50 +0000
> From: Sean Mooney <smooney at redhat.com>
> To: Jahson Babel <jahson.babel at cc.in2p3.fr>,
>    openstack-discuss at lists.openstack.org
> Subject: Re: [ops] QOS on flavor breaking live migration from CentOS 7
>    to 8
> Message-ID:
>    <b0f13cde177cd8b5f7e6e5b20445decab7dbc8b8.camel at redhat.com>
> Content-Type: text/plain; charset="UTF-8"
> 
> hi yes this is a know issue.
> 
> so the simple answer is resize all affected vms instead of live migrating them
> the longer answer is we have been dissing this internally at redhat on and off for
> some time now.
> https://bugs.launchpad.net/nova/+bug/1960840 is one example where this happens.
> 
> there is another case for the cpu based quotas that happens when going form rhel/centos 8->9
> basically in the 8->9 change the cgroups implemantion changes form v1 to v2
> https://bugzilla.redhat.com/show_bug.cgi?id=2035518
> 
> when adressing that we did not have a good universal solution for instnace that hardcoded a value that
> was incompatible with the cgroups_v2 api in the kernel except resize.
> 
> in https://review.opendev.org/c/openstack/nova/+/824048/ we removed automatically adding the
> cpu_shares cgroup option to enable booting vms with more then 8 cpus
> 
> we did not come up with any option other then resize for the other quotas that were in a similar situation.
> the one option that we considerd possibel to do was extend nova-mange to allow the embeded flaour to be updated
> this would be similar to what we did to enable the image property to be modifed for chaing machine types.
> 
> https://docs.openstack.org/nova/latest/cli/nova-manage.html#image-property-commands
> 
> we didcussed at the time that while we did not want to allow falvor extra specs to be modifed we might recondier that
> if the quota issue forced our hand or  we had a similar need due to foces beyond our contol. i.e. we needed to provide a way beyond
> resize e.g. due ot operating system changes. what make image properties and flavor extra spec different is that image proerties can
> only be updated by rebuild which is a destructive operation. extra specs are upsted by resize which is not a destructive operation.
> that is one of the reasons we have special considertion to image properties and did not do the same for extra specs.
> 
> if we allow the same for flavor extra specs you would still have to stop the instance, make the change and then migrate the instnace
> resize automates that so it is generall a better fit. we were also conceren that adding it to nova manage would result in it being abused
> to modify instnace in ways that were either invalid for the host(changing the numa toplogy, adding traits/resouce request not trackedcxd in placemnt)
> or otherwise break the instnace in weird ways. that could happen via image properites too but its less likely. 
> 
> 
> 
>> On Tue, 2023-01-03 at 17:25 +0100, Jahson Babel wrote:
>> Hello,
>> 
>> I'm trying to live migrate some VMs from CentOS 7 to Rocky 8.
>> Everything run smoothly when there is no extra specs on flavors but 
>> things getting more complicated when those are fixed. Especially when 
>> using quota:vif_burst for QOS.
>> I know that we aren't supposed to use this for QOS now but it's an old 
>> cluster and it was done that way at the time. So VMs kinda have all 
>> those specs tied to them.
>> 
>> When live migrate a VM this show up in the nova's logs :
>> driver.py _live_migration_operation nova.virt.libvirt.driver? Live 
>> Migration failure: internal error: Child process (tc class add dev 
>> tapxxxxxxxx-xx parent 1: classid 1:1 htb rate 250000kbps ceil 
>> 2000000kbps burst 60000000kb quantum 21333) unexpected exit status 1: 
>> Illegal "burst"
>> This bug cover the problem : https://bugs.launchpad.net/nova/+bug/1960840
>> So it's seems to be a normal behavior. Plus I forgot to mention that I'm 
>> on OpenStack Train version and the file mentioned in the launchpad is 
>> not present for this version.
>> By using Rocky 8 I have to use an updated libvirt that won't accept the 
>> burst parameter we used to set. All available versions of libvirt on 
>> Rocky 8 have changed behavior concerning the burst parameter.
>> 
>> I've done some testing to make things works including removing the 
>> extra_specs on flavors and in the DB, removing it through libvirt and 
>> trying to modify tc rules used by a VM but it didn't worked.
>> I have not tried yet to patch Nova or Libvirt but I don't really know 
>> where to look for.
>> The only thing that did work was to resize the VM to an identical flavor 
>> without the extra_specs. But this induce a complete reboot of the VM. I 
>> would like, if possible, to be able to live migrate the VMs which is 
>> quite easier.
>> 
>> Is it possible to remove the extra_specs on the VMs and then live 
>> migrate ? Or should I just plan to resize/reboot all VMs without those 
>> extra_specs ?
>> Any advise will be appreciated.
>> 
>> Thank you for any help,
>> Best regards.
>> 
>> Jahson
> 
> 
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Tue, 3 Jan 2023 15:23:19 -0500
> From: Satish Patel <satish.txt at gmail.com>
> To: Rafael Weing?rtner <rafaelweingartner at gmail.com>
> Cc: OpenStack Discuss <openstack-discuss at lists.openstack.org>
> Subject: Re: [cloudkitty] Instances billing based on tags
> Message-ID:
>    <CAPgF-fpGyO-Z8bt7KMdP7CAhi9kne3136ptoE+8coURyFwi7yw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Wow! very interesting, I will poke around and see how it's feasible. Very
> curious how it will represent that data in horizon UI. Currently I am
> seeing rates based on project_id so assuming it will show based on
> customer_id. correct?
> 
> 
> On Tue, Jan 3, 2023 at 8:13 AM Rafael Weing?rtner <
> rafaelweingartner at gmail.com> wrote:
> 
>> You can do that. Basically, you can start collecting those attributes you
>> want for billing (e.g. tags) via Ceilometer dynamic pollster (that is the
>> easiest way to achieve this). Then, you need to configure the resource type
>> in Gnocchi to store this extra attribute, and of course, configure
>> CloudKitty to collect/use it. Both in the metrics.yml and then in the
>> hashmap or Pyscript rules.
>> 
>>> On Mon, Jan 2, 2023 at 11:01 PM Satish Patel <satish.txt at gmail.com> wrote:
>>> 
>>> Folks,
>>> 
>>> We have an AWS project and in a single project we run multiple customers
>>> so for billing we use tags. In short every vm instance has a tag (customer
>>> name) and that way we can produce bills for each customer.
>>> 
>>> Recently I am playing with openstack cloudkitty and it works with
>>> standard cases like project based billing. But does it support tags based
>>> billing just similar to what i have explained in the above aws example?
>>> 
>>> 
>>> 
>> 
>> --
>> Rafael Weing?rtner
>> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230103/55a6b39b/attachment-0001.htm>
> 
> ------------------------------
> 
> Message: 4
> Date: Tue, 3 Jan 2023 15:27:03 -0500
> From: Mohammed Naser <mnaser at vexxhost.com>
> To: OpenStack Discuss <openstack-discuss at lists.openstack.org>
> Subject: [nova][keystone] workload identity?
> Message-ID:
>    <CAEs876hYr_mkhkgJ4S8cc2Cij4ZZ5oK20CiZpsWvpif8oitKFA at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> Hi folks:
> 
> I?m wondering if there?s anyone who?s had some thought or perhaps some
> work/progress/thoughts on workload identity (example service accounts for
> VMs) ? is that something that?s really far away for us?  Has someone
> thought about outlining what?s needed?
> 
> Thanks
> Mohammed
> -- 
> Mohammed Naser
> VEXXHOST, Inc.
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230103/8d2e1b21/attachment-0001.htm>
> 
> ------------------------------
> 
> Message: 5
> Date: Tue, 3 Jan 2023 17:39:18 -0300
> From: Rafael Weing?rtner <rafaelweingartner at gmail.com>
> To: Satish Patel <satish.txt at gmail.com>
> Cc: OpenStack Discuss <openstack-discuss at lists.openstack.org>
> Subject: Re: [cloudkitty] Instances billing based on tags
> Message-ID:
>    <CAG97rac6+PE+scD3j=hty20d9w_VVUbqdHG1B1a4pktrpH8ANw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> It is showing metrics based on the scope configured. I guess you have set
> the scopes to be mapped as project IDs. If you want other attribute to
> represent the scope, you need to change that in CloudKitty.
> 
>> On Tue, Jan 3, 2023 at 5:23 PM Satish Patel <satish.txt at gmail.com> wrote:
>> 
>> Wow! very interesting, I will poke around and see how it's feasible. Very
>> curious how it will represent that data in horizon UI. Currently I am
>> seeing rates based on project_id so assuming it will show based on
>> customer_id. correct?
>> 
>> 
>> On Tue, Jan 3, 2023 at 8:13 AM Rafael Weing?rtner <
>> rafaelweingartner at gmail.com> wrote:
>> 
>>> You can do that. Basically, you can start collecting those attributes you
>>> want for billing (e.g. tags) via Ceilometer dynamic pollster (that is the
>>> easiest way to achieve this). Then, you need to configure the resource type
>>> in Gnocchi to store this extra attribute, and of course, configure
>>> CloudKitty to collect/use it. Both in the metrics.yml and then in the
>>> hashmap or Pyscript rules.
>>> 
>>> On Mon, Jan 2, 2023 at 11:01 PM Satish Patel <satish.txt at gmail.com>
>>> wrote:
>>> 
>>>> Folks,
>>>> 
>>>> We have an AWS project and in a single project we run multiple customers
>>>> so for billing we use tags. In short every vm instance has a tag (customer
>>>> name) and that way we can produce bills for each customer.
>>>> 
>>>> Recently I am playing with openstack cloudkitty and it works with
>>>> standard cases like project based billing. But does it support tags based
>>>> billing just similar to what i have explained in the above aws example?
>>>> 
>>>> 
>>>> 
>>> 
>>> --
>>> Rafael Weing?rtner
>>> 
>> 
> 
> -- 
> Rafael Weing?rtner
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230103/cca3f116/attachment.htm>
> 
> ------------------------------
> 
> Subject: Digest Footer
> 
> _______________________________________________
> openstack-discuss mailing list
> openstack-discuss at lists.openstack.org
> 
> 
> ------------------------------
> 
> End of openstack-discuss Digest, Vol 51, Issue 6
> ************************************************
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230104/6d660f04/attachment-0001.htm>