June 2023 - openstack-discuss - lists.openstack.org

Experience with VGPUs
by Oliver Weinmann 22 Jun '23

22 Jun '23

Dear All, we are planning to have a POC on VGPUs in our Openstack cluster. Therefore I have a few questions and generally wanted to ask how well VGPUs are supported in Openstack. The docs, in particular: https://docs.openstack.org/nova/zed/admin/virtual-gpu.html explain quite well the general implementation. But I am more interested in general experience with using VGPUs in Openstack. We currently have a small YOGA cluster, planning to upgrade to Zed soon, with a couple of compute nodes. Currently our users use consumer cards like RTX 3050/3060 on their laptops and the idea would be to provide VGPUs to these users. For this I would like to make a very small POC where we first equip one compute node with an Nvidia GPU. Gladly also a few tips on which card would be a good starting point are highly appreciated. I know this heavily depends on the server hardware but this is something I can figure out later. Also do we need additional software licenses to run this? I saw this very nice presentation from CERN on VGPUs: https://indico.cern.ch/event/776411/contributions/3345183/attachments/18516… In the table they are listing Quadro vDWS licenses. I assume we need these in order to use the cards? Also do we need something like Cyborg for this or is VGPU fully implemented in Nova? Best Regards, Oliver

17 36

[nova][tempest] Hold your rechecks
by Sylvain Bauza 22 Jun '23

22 Jun '23

As far as I can tell, all nova-live-migration job runs are getting a FAILURE [1] due to [2] so please hold your rechecks. Thanks to Rodolfo, we have a workaround for it https://review.opendev.org/c/openstack/tempest/+/886496* so please wait until this change is merged.* Tempest cores, it would be nice if you can look at the above change quickly :) Thanks, -Sylvain [1] https://zuul.openstack.org/builds?job_name=nova-live-migration&skip=0 [2] https://bugs.launchpad.net/nova/+bug/2024160

2 1

[keystone] Updates to core reviewers
by Nikolla, Kristi 21 Jun '23

21 Jun '23

Hi all, Some updates to the core reviewers in Keystone repos. Dave Wilde (d34dh0r53) has been Keystone PTL for the Antelope and Bobcat cycle and has done an amazing work. Dave is now a core reviewer in Keystone. Artem Goncharov (gtema) is the PTL of OpenStack SDK, he is willing to help out with the Keystoneauth library, that being a core part of the SDK and CLI. Thank you Artem, you are now a core reviewer of Keystoneauth. Lance Bragstad (lbragstad) has been inactive for a bit over a year now and I believe doesn't work on OpenStack anymore. Due to that I am removing as a core reviewer from Keystone. Lance has been a contributor to Keystone and OpenStack for a really really long time and was an essential part of the Keystone team, serving as its PTL too. Best, Kristi Nikolla

3 2

[ceilometer][cinder] How to properly set up Block Storage meters / cinder-volume-usage-audit
by Christian Rohmann 21 Jun '23

21 Jun '23

Hello OpenStack-Discuss, when setting up telemetry (ceilometer, gnocci, aodh) I followed the documentation at https://docs.openstack.org/ceilometer/latest/install/cinder/install-cinder-… to set up sending metrics for cinder to ceilomter. 1) The instruction to run cinder-volume-usage-audit as a timer / cron every 5 minutes > */5 * * * * /path/to/cinder-volume-usage-audit --send_actions has me confused. This would cause the usage data for the last full month, to be sent to the notifications queue every 5 minutes. There is no progress tracker or pointer limiting this to only sent data that has not yet been sent. Looking at how openstack-ansible configures this (https://github.com/openstack/openstack-ansible-os_cinder/blob/1af3003e163e0…) it seems they were aware that running this without bounds and every 5 minutes is wrong. I also opened a bug (https://bugs.launchpad.net/ceilometer/+bug/2024475) about this. 2) Extending on 1) I believe setting the > volume_usage_audit_period in the cinder configuration, to e.g. month or hour, will already set the interval for which the data is emitted. Then the important thing is to only run cinder-volume-usage-audit in the same frequency. 3) Looking at the "--send_actions" options, I am wondering if the "create/delete" actions for volumes, snapshosts and backups are not sent by cinder anyways and why they would need to be sent again in batch and by cinder-volume-usage-audit? I am referring to > [oslo_messaging_notifications] > driver = messagingv2 > being enabled. This might also be the reasoning behind this bug, https://bugs.launchpad.net/openstack-ansible/+bug/1968734, asking for this parameter to be behind a config switch to avoid duplicate actions to be sent. 4) Thinking about the whole setup more questions arise * What happens when I run this more than once in the same interval? What will ceilometer do with the (redundant) data? * I am also wondering how well this single piece of software scales and how this could be split up more? * Is there no way to implement a pointer to allow running this command more often and only send "new data"? Just like the timestamp up to which data was sent. * What about any locking / coordination to enable multiple instances? Regards Christian

1 0

[ceilometer][gnocchi][telemetry] (User) frontend or tool to create reports on telemetry data (GPU, RAM, disk, network, ...)
by Christian Rohmann 21 Jun '23

21 Jun '23

Hello OpenStack-Discuss, I was wondering if there are any simple frontends or similar solutions to provide ceilometer's telemetry data, such as CPU, RAM, volume throughput and IOPs, network throughput, ... to users? Something like a basic horizon dashboard? Or does anybody have written something using the API to create some nice graphs? I know there is Cloudkitty, but that looks more targeted towards doing rating and billing with this kind of data? Thanks and with regards Christian

3 2

HOT template help
by Antony P 21 Jun '23

21 Jun '23

Hi team, This is user data jenkins_password=$(sudo cat /var/lib/jenkins/secrets/initialAdminPassword) i want to pass this as output like Output: value: password How can i resolve this? *--Warm Regards* *Antony P* *Jr. DevOps Engineer* antony(a)edgebricks.com Mob No: +919498079898 www.edgebricks.com

1 0

Cinder Bug Report 2023-06-21
by Sofia Enriquez 21 Jun '23

21 Jun '23

Hello Argonauts, Cinder Bug Meeting Etherpad <https://etherpad.opendev.org/p/cinder-bug-squad-meeting> *Wishlist* - Restore RBD Incremental Backup Fails When Using the --container Argument. <https://bugs.launchpad.net/cinder/+bug/2024484> - *Status*: Fix proposed to master. <https://review.opendev.org/c/openstack/cinder/+/788319> *Low* - Hitachi: wrong exception when deleted volume is busy. <https://bugs.launchpad.net/cinder/+bug/2024418> - *Status*: Fix proposed to master <https://review.opendev.org/c/openstack/cinder/+/885986>. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

versioning hidden container accounting
by Geert Geurts 21 Jun '23

21 Jun '23

Hello I've implemented the newer type of versioning (x-versions-enabled: True) for a container. The documentation (https://docs.openstack.org/swift/latest/middleware.html#object-versioning) states that actual data is written to a hidden container. Now my question is would it be possible to set the account for the data of the current ("is_latest": true) objects to a different account? Would it be possible to set headers on the event that 'current' changes? Can I see/change the properties of this "hidden" container? Thanks allot for any help/pointers! Best regards, Geert

1 0

[ironic] [networking-generic-switch] NGS performance and moving to an agent-based design
by Baptiste Jonglez 21 Jun '23

21 Jun '23

Hello, We have been hitting performance issues with networking-generic-switch (NGS), the Neutron ML2 plugin that performs dynamic reconfiguration of physical switches, most notably used by Ironic. In our quest to improve NGS performance, we have designed and implemented an agent-based system for NGS, and we would now like to upstream it. Since this is a fairly big change, and some parts of the design such as HA are not completely clear yet, feel free to join the discussion on the RFE: https://bugs.launchpad.net/networking-generic-switch/+bug/2024385 Regards, Baptiste -- Baptiste Jonglez Research Engineer, Inria <https://www.inria.fr/> STACK team <https://stack-research-group.gitlabpages.inria.fr/web/>

2 1

[kolla] nova-compute image build failed with invalid signature
by Satish Patel 20 Jun '23

20 Jun '23

Folks, Just wanted to check with folks if I did anything wrong here. Building 2023.1 image of nova-compute and encounter errors. root@docker-reg:/opt/kolla/etc/kolla# kolla-build --registry docker-reg:4000 --config-file kolla-build.conf --debug --cache --threads 1 --skip-existing --push --format none nova-compute INFO:kolla.common.utils:Using engine: docker INFO:kolla.common.utils:Found the container image folder at /usr/local/share/kolla/docker INFO:kolla.common.utils:Added image nova-compute-ironic to queue INFO:kolla.common.utils:Added image nova-compute to queue INFO:kolla.common.utils:Attempt number: 1 to run task: BuildTask(nova-compute-ironic) DEBUG:kolla.common.utils.nova-compute-ironic:Processing INFO:kolla.common.utils.nova-compute-ironic:Building started at 2023-06-20 17:36:33.771585 DEBUG:kolla.common.utils.nova-compute-ironic:Turned 0 plugins into plugins archive DEBUG:kolla.common.utils.nova-compute-ironic:Turned 0 additions into additions archive INFO:kolla.common.utils.nova-compute-ironic:Step 1/4 : FROM docker-reg:4000/kolla/nova-base:2023.1 INFO:kolla.common.utils.nova-compute-ironic: ---> 0ebe71c922b4 INFO:kolla.common.utils.nova-compute-ironic:Step 2/4 : LABEL maintainer="Kolla Project (https://launchpad.net/kolla)" name="nova-compute-ironic" build-date="20230620" INFO:kolla.common.utils.nova-compute-ironic: ---> Using cache INFO:kolla.common.utils.nova-compute-ironic: ---> 5209e0cc6403 INFO:kolla.common.utils.nova-compute-ironic:Step 3/4 : RUN apt-get --error-on=any update && apt-get -y install --no-install-recommends genisoimage nvme-cli && apt-get clean && rm -rf /var/lib/apt/lists/* INFO:kolla.common.utils.nova-compute-ironic: ---> Running in 196bc3715e45 INFO:kolla.common.utils.nova-compute-ironic:Get:1 http://ubuntu-cloud.archive.canonical.com/ubuntu jammy-updates/antelope InRelease [5,463 B] INFO:kolla.common.utils.nova-compute-ironic:Get:2 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [108 kB] INFO:kolla.common.utils.nova-compute-ironic:Err:1 http://ubuntu-cloud.archive.canonical.com/ubuntu jammy-updates/antelope InRelease INFO:kolla.common.utils.nova-compute-ironic: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:Get:3 http://mirrors.ubuntu.com/mirrors.txt Mirrorlist [3,463 B] INFO:kolla.common.utils.nova-compute-ironic:Get:6 http://archive.ubuntu.com/ubuntu jammy-security InRelease [110 kB] INFO:kolla.common.utils.nova-compute-ironic:Err:2 http://archive.ubuntu.com/ubuntu jammy-backports InRelease INFO:kolla.common.utils.nova-compute-ironic: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:Get:5 https://mirror.dal.nexril.net/ubuntu jammy-updates InRelease [119 kB] INFO:kolla.common.utils.nova-compute-ironic:Err:6 http://archive.ubuntu.com/ubuntu jammy-security InRelease INFO:kolla.common.utils.nova-compute-ironic: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:Get:4 http://ftp.usf.edu/pub/ubuntu jammy InRelease [270 kB] INFO:kolla.common.utils.nova-compute-ironic:Err:5 https://mirror.dal.nexril.net/ubuntu jammy-updates InRelease INFO:kolla.common.utils.nova-compute-ironic: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:Err:4 http://ftp.usf.edu/pub/ubuntu jammy InRelease INFO:kolla.common.utils.nova-compute-ironic: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:Reading package lists... INFO:kolla.common.utils.nova-compute-ironic:W: GPG error: http://ubuntu-cloud.archive.canonical.com/ubuntu jammy-updates/antelope InRelease: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:E: The repository 'http://ubuntu-cloud.archive.canonical.com/ubuntu jammy-updates/antelope InRelease' is not signed. INFO:kolla.common.utils.nova-compute-ironic:W: GPG error: http://archive.ubuntu.com/ubuntu jammy-backports InRelease: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:E: The repository 'http://archive.ubuntu.com/ubuntu jammy-backports InRelease' is not signed. INFO:kolla.common.utils.nova-compute-ironic:W: GPG error: http://archive.ubuntu.com/ubuntu jammy-security InRelease: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt jammy-security InRelease' is not signed. INFO:kolla.common.utils.nova-compute-ironic:W: GPG error: https://mirror.dal.nexril.net/ubuntu jammy-updates InRelease: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt jammy-updates InRelease' is not signed. INFO:kolla.common.utils.nova-compute-ironic:W: GPG error: http://ftp.usf.edu/pub/ubuntu jammy InRelease: At least one invalid signature was encountered. INFO:kolla.common.utils.nova-compute-ironic:E: The repository 'mirror://mirrors.ubuntu.com/mirrors.txt jammy InRelease' is not signed. INFO:kolla.common.utils.nova-compute-ironic: INFO:kolla.common.utils.nova-compute-ironic:Removing intermediate container 196bc3715e45 ERROR:kolla.common.utils.nova-compute-ironic:Error'd with the following message ERROR:kolla.common.utils.nova-compute-ironic:The command '/bin/sh -c apt-get --error-on=any update && apt-get -y install --no-install-recommends genisoimage nvme-cli && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 INFO:kolla.common.utils:Attempt number: 2 to run task: BuildTask(nova-compute-ironic) DEBUG:kolla.common.utils.nova-compute-ironic:Processing Full trace - https://paste.opendev.org/show/b3IYgcUXsvg0kxVpEaLM/

2 2