December 2022 - openstack-discuss

Fwd: DPDK+OVS with OpenStack
by Mark Wittling 23 Apr '25

23 Apr '25

Looking for someone who knows OpenStack with OpenVSwitch, and in addition to that, DPDK with OpenStack and OVS.I am using OpenStack Queens, with OpenVSwitch. The architecture I am using is documented here: https://docs.openstack.org/neutron/queens/admin/deploy-ovs-provider.htmlThe OVS I am using on the Compute Node, is compiled with DPDK, and I have enabled the datapath to netdev (DPDK) on br-prv (provider network bridge), and br-tun (tunneling bridge). But these two bridges, br-tun and br-prv, are patched into another OpenStack bridge, called br-int. I wasn’t actually sure about whether to tinker with this bridge, and wondered what datapath it was using.Then, I realized there is a parameter in the openvswitch_agent.ini file, which I will list here: # OVS datapath to use. 'system' is the default value and corresponds to the # kernel datapath. To enable the userspace datapath set this value to 'netdev'. # (string value) # Possible values: # system - <No description provided> # netdev - <No description provided> #datapath_type = system datapath_type = netdev So in tinkering with this, what I realized, is that when you set this datapath_type to system or netdev, it will adjust the br-int bridge to that datapath type.So here is my question. How can I launch a non-DPDK VM, if all of the bridges are using the netdev datapath type?Here is another question. What if one of the flavors don’t have the largepages property set on them? I assumed OpenStack would revert to a system datapath and not use DPDK for those VM interfaces. Well, I found out in testing, that is not the case. If you set all your bridges up for netdev, and you don’t set the property on the Flavor of the VM (largepages), the VM will launch, but it simply won’t work.Is there no way, on a specific Compute Host, to support both DPDK (netdev datapaths) and non-DPDK (system datapaths)?Either on a VM interface level (VM has one interface that is netdev DPDK and another that is system datapath non-DPDK)?Or on a VM by VM basis (VM 1 has 1 or more netdev datapath interfaces and VM 2 has 1 or more system datapath interfaces)?Am I right here? Once you set up a Compute Host for DPDK, it’s DPDK or nothing on that Compute Host? (edited)

3 2

How to config DPDK compute in OVN
by dangerzone ar 22 Apr '25

22 Apr '25

Hi....as far I understand ovs does support dpdk compute to be deployed. I'm referring to the guideline ' *https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html <https://docs.openstack.org/networking-ovn/queens/admin/dpdk.html>* ' ... It's too brief.. May I know how to config dpdk compute in OVN networking openstack? Hope someone could help what is the config to be done.... Currently compute is run without dpdk over openstack train ovn based. Please need some info on how I can proceed. Thank you

2 1

ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat.| Openstack tripleo wallaby version
by Swogat Pradhan 24 Dec '24

24 Dec '24

Hi, I am currently trying to deploy openstack wallaby using tripleo arch. I created the network jinja templates, ran the following commands also: #openstack overcloud network provision --stack overcloud --output networks-deployed-environment.yaml custom_network_data.yaml # openstack overcloud network vip provision --stack overcloud --output vip-deployed-environment.yaml custom_vip_data.yaml # openstack overcloud node provision --stack overcloud --overcloud-ssh-key /home/stack/sshkey/id_rsa overcloud-baremetal-deploy.yaml and used the environment files in the openstack overcloud deploy command: (undercloud) [stack@hkg2director ~]$ cat deploy.sh #!/bin/bash THT=/usr/share/openstack-tripleo-heat-templates/ CNF=/home/stack/ openstack overcloud deploy --templates $THT \ -r $CNF/templates/roles_data.yaml \ -n $CNF/workplace/custom_network_data.yaml \ -e ~/containers-prepare-parameter.yaml \ -e $CNF/templates/node-info.yaml \ -e $CNF/templates/scheduler-hints.yaml \ -e $CNF/workplace/networks-deployed-environment.yaml \ -e $CNF/workplace/vip-deployed-environment.yaml \ -e $CNF/workplace/overcloud-baremetal-deployed.yaml \ -e $CNF/workplace/custom-net-bond-with-vlans.yaml Now when i run the ./deploy.sh script i encounter an error stating: ERROR openstack [-] Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command.: tripleoclient.exceptions.InvalidConfiguration: Resource OS::TripleO::Network::Ports::ControlPlaneVipPort maps to type OS::Neutron::Port and the Neutron service is not available when using ephemeral Heat. The generated environments from 'openstack overcloud baremetal provision' and 'openstack overcloud network provision' must be included with the deployment command. 2022-04-19 13:47:16.582 735924 INFO osc_lib.shell [-] END return value: 1 Can someone tell me where the mistake is? With regards, Swogat Pradhan

4 4

Kolla Ansible on Ubuntu 20.04 - cloud-init & other network issues
by Tobias McNulty 10 Oct '24

10 Oct '24

Hi, I'm attempting to use Kolla Ansible 14.6.0 to deploy OpenStack Yoga on a small 3-node Ubuntu 20.04 cluster. The nodes have 128 GB RAM each, dual Xeon processors, and dual 10G Intel NICs. The NICs are connected to access ports on a 10G switch with separate VLANs for the local and external networks. All the playbooks run cleanly, but cloud-init is failing in the Ubuntu 20.04 and 22.04 VMs I attempt to boot. The VM images are unmodified from https://cloud-images.ubuntu.com/, and cloud-init works fine if I mount a second volume with user-data. The error is a timeout attempting to reach 169.254.169.254. This occurs both when booting a VM in an internal routed network and directly in an external network. I tried various neutron plugin agents (ovn, linuxbridge, and openvswitch both with and without firewall_driver = openvswitch <https://docs.openstack.org/kolla-ansible/latest/reference/networking/neutro…>) first with a clean install of the entire OS each time, all with the same result. Running tcpdump looking for 169.254.169.254 shows nothing. As a possible clue, the virtual NICs are unable to pass any traffic (e.g., to reach an external DHCP server) unless I completely disable port security on the interface (even if the associated security group is wide open). But disabling port security does not fix cloud-init (not to mention I don't really want to disable port security). Are there any additional requirements related to deploying OpenStack with Kolla on Ubuntu 20.04? This is a fairly vanilla configuration using the multinode inventory as a starting point. I tried to follow the Quick Start <https://docs.openstack.org/kolla-ansible/yoga/user/quickstart.html> as closely as possible; the only material difference I see is that I'm using the same 3 nodes for control + compute. I am using MAAS so it's easy to get a clean OS install on all three nodes ahead of each attempt. I plan to try again with the standard (non-HWE) kernel just in case, but otherwise I am running out of ideas. In case of any additional clues, here are my globals.yml and inventory file, along with the playbook I'm using to configure the network, images, VMs, etc., after bootstrapping the cluster: https://gist.github.com/tobiasmcnulty/7dbbdbc67abc08cbb013bf5983852ed6 Thank you in advance for any advice! Cheers, Tobias

5 9

Live migration fails
by Szabo, Istvan (Agoda) 07 Aug '24

07 Aug '24

Hi, I have couple of compute nodes where the live migration fails with existing vms. When I quickly spawn a vm and try live migration it works so I assume shouldn't be a big problem with the compute node. However I have many existing vms where it fails with a servername not found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 ERROR nova.conductor.tasks.migrate [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Unable to find record for source node servername on servername: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Failed to compute_task_migrate_server: Compute host servername could not be found.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.605 227612 WARNING nova.scheduler.utils [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] [instance: 1517a2ac-3b51-4d8d-80b3-89a5614d1ae0] Setting instance to ACTIVE state.: ComputeHostNotFound: Compute host servername could not be found. /var/log/nova/nova-conductor.log:2021-04-21 14:47:12.672 227612 ERROR oslo_messaging.rpc.server [req-f4067a26-a233-4673-8c07-9a8a290980b0 dce35e6eceea4312bb0baa0510cef363 ca7e35079f4440c78bd9870724b9638b - default default] Exception during message handling: ComputeHostNotFound: Compute host am-osfecn-4025 Tried with this command: nova live-migration --block-migrate id. Any idea? Thank you. ________________________________ This message is confidential and is for the sole use of the intended recipient(s). It may also be privileged or otherwise protected by copyright or other legal rules. If you have received it by mistake please let us know by reply email and delete it from your system. It is prohibited to copy this message or disclose its content to anyone. Any confidentiality or privilege is not waived or lost by any mistaken delivery or unauthorized disclosure of the message. All messages sent to and from Agoda may be monitored to ensure compliance with company policies, to protect the company's interests and to remove potential malware. Electronic messages may be intercepted, amended, lost or deleted, or contain viruses.

5 16

[puppet] Proposing core team clean up
by Takashi Kajinami 23 Aug '23

23 Aug '23

Hello, I went through the current core reviewers and found out that the current list includes some people no longer working in the OpenStack community. I'd like to propose removing the following persons from the core reviewer team as we haven't seen these people making any activity in the community recently. * Pradeep Kilambi - puppet-aodh, puppet-ceilometer, puppet-gnocchi, puppet-panko * Richard Megginson - puppet-keystone * Denis Egorenko - puppet-murano * Serg Melikyan - puppet-murano * Sergey Kolekonov - puppet-neutron * David Moreau Simard - puppet-openstack-integration * Yu xingchao - all modules Also, I confirmed that Wes Hayutin is no longer working for OpenStack and he agreed to retire from the core reviewer position. Thus I'll remove him as well. I'll be open for any feedback for one week to hear any thoughts/feedback about this removal. I'd like to take this moment to thank all people for their past work. We always welcome them back if they resume their work in this project. Thank you, Takashi KAJINAMI

3 4

Re: [all] [oslo.messaging] Interest in collaboration on a NATS driver
by Tobias Urdin 22 Jul '23

22 Jul '23

Hello Kai, Great to hear that you’re interested! I’m currently just testing this as a POC directly in devstack so there is real-world testing (not should it yet I think, needs more development first). Best regards Tobias > On 30 Aug 2022, at 18:32, Kai Bojens <kb(a)kbojens.de> wrote: > > Am 29.08.22 um 15:46 schrieb Tobias Urdin: > >> If anybody, or any company, out there would be interested in collaborating in a project to bring this support and maintain it feel free to >> reach out. I’m hoping somebody will bite but atleast I’ve put it out there for all of you. > > Hi, > I am very much interested to take a closer look at your work and maybe contribute to it. Although I'm working with OpenStack for my employer at the moment, I'd do this in my spare time as I'm not sure that I can convince him to add another staging system with a full OpenStack installation just for developing a RabbitMQ replacement. That's not on our agenda as we are mostly users and not developers of OpenStack. > > As I'm pretty new to the messaging topic and had just heard of NATS and your idea, I'll first would have to dive into NATS and then I would take a closer look at your code and maybe try to create some documentation or tests. So, I can't promise anything but as I said I'm very interested in your approach as I also see the massive load from RabbitMQ. > > This brings me to the most important question: Where do you run and test your code? > > Greetings, > Kai

8 30

[ironic][stable] Proposing EOL of ironic project branches older than Wallaby
by Jay Faulkner 02 Jun '23

02 Jun '23

Hi all, Ironic has a large amount of stable branches still in EM. We need to take action to ensure those branches are either retired or have CI repaired to the point of being usable. Specifically, I'm looking at these branches across all Ironic projects: - stable/queens - stable/rocky - stable/stein - stable/train - stable/ussuri - stable/victoria In lieu of any volunteers to maintain the CI, my recommendation for all the branches listed above is that they be marked EOL. If someone wants to volunteer to maintain CI for those branches, they can propose one of the below paths be taken instead: 1 - Someone volunteers to maintain these branches, and also report the status of CI of these older branches periodically on the Ironic whiteboard and in Ironic meetings. If you feel strongly that one of these branches needs to continue to be in service; volunteering in this way is how to save them. 2 - We seriously reduce CI. Basically removing all tempest tests to ensure that CI remains reliable and able to merge emergency or security fixes when needed. In some cases; this still requires CI fixes as some older inspector branches are failing *installing packages* in unit tests. I would still like, in this case, that someone volunteers to ensure the minimalist CI remains happy. My intention is to let this message serve as notice and a waiting period; and if I've not heard any response here or in Monday's Ironic meeting (in 6 days), I will begin taking action on retiring these branches. This is simply a start; other branches (including bugfix branches) are also in bad shape in CI, but getting these retired will significantly reduce the surface area of projects and branches to evaluate. I know it's painful to drop support for these branches; but we've provided good EM support for these branches for a long time and by pruning them away, we'll be able to save time to dedicate to other items. Thanks, Jay Faulkner

3 5

[Openstack] Lack of Balance solution such as Watcher.
by Nguyễn Hữu Khôi 16 Mar '23

16 Mar '23

I am building a private cloud.Everything is ok. But I cannot find a way to balance vm when It have heavy load which cause impact other vm on the compute node. I found solutions such as Watcher and Leveller. But they need to be done manually. Watcher is not good because It need cpu metric such as cpu load in Ceilometer which is removed so we cannot use it. Leveller is good, but It is obsolete. Nguyen Huu Khoi

4 7

live resize
by Parsa Aminian 15 Mar '23

15 Mar '23

hello on openstack with ceph backend is it possible to live resize instances ? I want to change flavor without any down time .

5 6

[kolla-ansible] [yoga] [magnum] [k8s] cannot attach persistent volume to pod
by Stuart Whitman 24 Feb '23

24 Feb '23

Hello, When I try to attach a persistent cinder volume to a pod, I get FailedMount and FailedAttachVolume timeout events. I also get these errors in the log of the csi-cinder-controllerplugin-0 pod: E1020 13:38:41.747511 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.VolumeAttachment: the server could not find the requested resource E1020 13:38:41.748187 1 reflector.go:126] k8s.io/client-go/informers/factory.go:133: Failed to list *v1beta1.CSINode: the server could not find the requested resource I fixed a CrashLoopBackoff error with the csi-snapshotter container in the csi-cinder-controllerplugin-0 pod by providing the label "csi_snapshotter_tag=v4.0.0" when I created the cluster template. I found that suggestion in an issue on the GitHub cloud-provider-openstack project. I'm not finding any help with this error on Google. Thanks, -Stu _____________________________________ The information contained in this e-mail and any attachments from Group W may contain confidential and/or proprietary information and is intended only for the named recipient to whom it was originally addressed. If you are not the intended recipient, be aware that any disclosure, distribution, or copying of this e-mail or its attachments is strictly prohibited. If you have received this e-mail in error, please notify the sender immediately of that fact by return e-mail and permanently delete the e-mail and any attachments to it.

3 5

[charms] Team Delegation proposal
by Chris MacNaughton 08 Feb '23

08 Feb '23

Hello All, <tl;dr> I would like to propose some new ACLs in Gerrit for the openstack-charms project: - openstack-core-charms - ceph-charms - network-charms - stable-maintenance </tl;dr> With an increasing focus split among the openstack-charmers team, I'm observing that people are focused on more specific subsets of the charms, and would like to propose that new ACLs are created to allow us to recognize that officially. I've chosen the breakdown above as it aligns neatly with where the focus lines are at this point, letting developers work on their specific focus areas. This proposal would not reduce permissions for anybody who is currently a core on the openstack-charms project and, in fact, future subteam core members could aspire to full openstack-charmers core as well. Ideally, this approach will let us escalate developers to "core" developers for the subteam(s) where they have demonstrated the expertise we expect in a core-charmer. It also allows a more gradual escalation to being a core in the openstack-charms project, making it a progression rather than a single destination. As a related addition, I'm appending to this proposal the creation of a stable-maintenance ACL which would allow members to manage backports without a full core-charmer grant. --- Chris MacNaughton

5 7

[Openstack Tacker] - Issue while creating NS
by Lokendra Rathour 16 Jan '23

16 Jan '23

Hi Team, Was trying to create NS using the document : https://docs.openstack.org/tacker/yoga/user/nsd_usage_guide.html But getting an error, the bug has been opened, requesting your kind assistance. https://bugs.launchpad.net/tacker/+bug/1999502 thanks once again. -- ~ Lokendra skype: lokendrarathour

2 1

[ironic] Bugfix branches being EOL'd first week of Jan, 2023
by Jay Faulkner 12 Jan '23

12 Jan '23

OpenStack Community and Operators, As documented in https://specs.openstack.org/openstack/ironic-specs/specs/approved/new-relea…, Ironic performs bugfix releases in the middle of a cycle to permit downstream packagers to more rapidly deliver features to standalone Ironic users. However, we've neglected as a project to cleanup or EOL any of these branches -- until now. Please take notice that during the first week in January, we will be EOL-ing all old, unsupported Ironic bugfix branches. This will be handled similarly to an EOL of a stable branch; we will create a tag -- e.g. for bugfix/x.y branch, we would tag bugfix-x.y-eol -- then remove the branch. These branches have been out of support for months and should not be in use in your Ironic clusters. If you are using any branches slated for retirement, please immediately upgrade to a supported Ironic version. A full listing of projects and branches impacted: ironic branches being retired bugfix/15.1 bugfix/15.2 bugfix/16.1 bugfix/16.2 bugfix/18.0 bugfix/20.0 ironic-python-agent branches being retired bugfix/6.2 bugfix/6.3 bugfix/6.5 bugfix/6.6 bugfix/8.0 bugfix/8.4 ironic-inspector branches being retired bugfix/10.2 bugfix/10.3 bugfix/10.5 bugfix/10.10 Thank you, Jay Faulkner Ironic PTL

1 4

Issue resizing volumes attached to running volume backed instances
by Jérôme BECOT 12 Jan '23

12 Jan '23

Hello Openstack, We have Ussuri deployed on a few clouds, and they're all plugged to PureStorage Arrays. We allow users to only use volumes for their servers. It means that each server disk is a LUN attached over ISCSI (with multipath) on the compute node hosting the server. Everything works quite fine, but we have a weird issue when extending volumes attached to running instances. The guests notice the new disk size .. of the last extent. Say I have a server with a 10gb disk. I add 5gb. On the guest, still 10gb. I add another 5gb, and on the guest I get 15, and so on. I've turned the debug mode on and I could see no error in the log. Looking closer at the log I could catch the culprit: 2022-12-08 17:35:13.998 46195 DEBUG os_brick.initiator.linuxscsi [] Starting size: *76235669504* 2022-12-08 17:35:14.028 46195 DEBUG os_brick.initiator.linuxscsi [] volume size after scsi device rescan *80530636800* extend_volume 2022-12-08 17:35:14.035 46195 DEBUG os_brick.initiator.linuxscsi [] Volume device info = {'device': '/dev/disk/by-path/ip-1...1:3260-iscsi-iqn.2010-06.com.purestorage:flasharray.x-lun-10', 'host': '5', 'channel': '0', 'id': '0', 'lun': '10'} extend_volume 2022-12-08 17:35:14.348 46195 INFO os_brick.initiator.linuxscsi [] Find Multipath device file for volume WWN 3624... 2022-12-08 17:35:14.349 46195 DEBUG os_brick.initiator.linuxscsi [] Checking to see if /dev/disk/by-id/dm-uuid-mpath-3624.. exists yet. wait_for_path 2022-12-08 17:35:14.349 46195 DEBUG os_brick.initiator.linuxscsi [] /dev/disk/by-id/dm-uuid-mpath-3624... has shown up. wait_for_path 2022-12-08 17:35:14.382 46195 INFO os_brick.initiator.linuxscsi [] mpath(/dev/disk/by-id/dm-uuid-mpath-3624) *current size 76235669504* 2022-12-08 17:35:14.412 46195 INFO os_brick.initiator.linuxscsi [] mpath(/dev/disk/by-id/dm-uuid-mpath-3624) *new size 76235669504* 2022-12-08 17:35:14.413 46195 DEBUG oslo_concurrency.lockutils [] Lock "extend_volume" released by "os_brick.initiator.connectors.iscsi.ISCSIConnector.extend_volume" :: held 2.062s inner 2022-12-08 17:35:14.459 46195 DEBUG os_brick.initiator.connectors.iscsi [] <== extend_volume: return (2217ms) *76235669504* trace_logging_wrapper 2022-12-08 17:35:14.461 46195 DEBUG nova.virt.libvirt.volume.iscsi [] Extend iSCSI Volume /dev/dm-28; new_size=*76235669504* extend_volume 2022-12-08 17:35:14.462 46195 DEBUG nova.virt.libvirt.driver [] Resizing target device /dev/dm-28 to *76235669504* _resize_attached_volume The logs clearly shows that the rescan confirm the new size but when interrogating multipath, it does not. But requesting multipath few seconds after on the command line shows the new size as well. It explains the behaviour. I'm running Ubuntu 18.04 with multipath 0.7.4-2ubuntu3.2. The os-brick code for multipath is far more basic than the one in master branch. Maybe the multipath version installed is too recent for os-brick. Thanks for the help Jerome

2 2

[tc][mistral][release] Propose to deprecate Mistral
by Elõd Illés 06 Jan '23

06 Jan '23

Hi, Mistral projects are unfortunately not actively maintained and caused hard times in latest official series releases for release management team. Thus we discussed this and decided to propose to deprecate Mistral [1] to avoid broken releases and last minute debugging of gate issues (which usually fall to release management team), and remove mistral projects from 2023.1 Antelope release [2]. We would like to ask @TC to evaluate the situation and review our patches (deprecation [1] + removal from the official release [2]). Thanks in advance, Előd Illés irc: elodilles @ #openstack-release [1] https://review.opendev.org/c/openstack/governance/+/866562 [2] https://review.opendev.org/c/openstack/releases/+/865577

6 7

[kolla] Propose Bartosz Bezak for core reviewer
by Michał Nasiadka 06 Jan '23

06 Jan '23

Hello Koalas, I’d like to propose Bartosz Bezak as a core reviewer for Kolla, Kolla-Ansible, Kayobe and ansible-collection-kolla. Bartosz has recently went through release preparations and release process itself for all mentioned repositories, has been a great deal of help in meeting the cycle trailing projects deadline. In addition to that, he’s been the main author of Ubuntu Jammy and EL9 (Rocky Linux 9 to be precise) support in Kayobe for Zed release, as well as fixing various bugs amongst all four repositories. Bartosz also brings OVN knowledge, which will make the review process for those patches better (and improve our overall review velocity, which hasn’t been great recently). Kind regards, Michal Nasiadka

6 6

Nova libvirt/kvm sound device
by Zakhar Kirpichenko 06 Jan '23

06 Jan '23

Hi! I'd like to have the following configuration added to every guest on a specific host managed by Nova and libvirt/kvm: <sound model='bla'> <address type='pci' domain='0x0000' bus='0x00' slot='0x1b' function='0x0'/> </sound> When I add the device manually to instance xml, it works as intended but the instance configuration gets overwritten on instance stop/start or hard reboot via Nova. What is the currently supported / proper way to add a virtual sound device without having to modify libvirt or Nova code? I would appreciate any advice. Best regards, Zakhar

2 2

[designate] Implemeting PTR record restrictions
by Thomas Goirand 06 Jan '23

06 Jan '23

Hi, We implemented this scenario for our public cloud: https://docs.openstack.org/neutron/latest/admin/config-dns-int-ext-serv.htm… This is currently in production in beta-mode at Infomaniak's public cloud. We did that, because we want our customers to be able to set any domain name or PTR for the IPs they own. However, we discovered that there's no restriction on what zone customers can set. For example, if customer A owns the IP 203.0.113.9, customer B can do "openstack zone create 9.113.0.203.in-addr.arpa.", preventing customer A to set their PTR record. Is there currently a way to fix this? Or maybe a spec to implement the correct restrictions? What is the way to fix this problem in a public cloud env? Cheers, Thomas Goirand (zigo)

2 1

[designate] New project: designate-tlds
by Thomas Goirand 06 Jan '23

06 Jan '23

Hi, We wrote this: https://salsa.debian.org/openstack-team/services/designate-tlds The interesting code bits are in: https://salsa.debian.org/openstack-team/services/designate-tlds/-/blob/debi… What it does is download the TLD list from https://publicsuffix.org/list/public_suffix_list.dat using requests (with an optional proxy), compare it to the list of TLDs in Designate, and fix the difference. It's by default setup in a cron every week. Basically, it's just apt-get install designate-tlds, configure keystone_authtoken in /etc/designate-tlds/designate-tlds.conf and set dry_run=false, and you're done! Note I also wrote a patch for puppet-designate [1] to support it. Moving forward I see 2 solutions: 1- we continue to maintain this separately from Designate 2- our code gets integrated into Designate itself. Designate team: are you interested for option 2? Cheers, Thomas Goirand (zigo) [1] https://salsa.debian.org/openstack-team/puppet/puppet-module-designate/-/bl…

2 1

[cinder][drivers] third-party CI for os-brick changes
by Brian Rosmaita 04 Jan '23

04 Jan '23

To all third-party CI maintainers, As you are aware, cinder third-party CI systems are required to run on all cinder changes. However, the os-brick library used in cinder CI testing is the latest appropriate *released* version of os-brick. Thus, it is possible for changes to be happening in os-brick development that might impact the functionality of your driver. If you aren't testing os-brick changes, you won't find out about these until *after* the next os-brick release, which is bad news all around. Therefore, at last week's cinder midcycle [0], the cinder project team agreed to require that cinder third-party CI systems run on all os-brick changes in addition to all cinder changes. This is a nice-to-have for the current (Zed) development cycle, but will be required in order for a driver to be considered 'supported' in the 2023.1 (Antelope) release [1]. If you have comments or concerns about this policy, please reply on the list to this email or put an item on the agenda [2] for the cinder weekly meeting. [0] https://etherpad.opendev.org/p/cinder-zed-midcycles [1] https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers#What_changes… [2] https://etherpad.opendev.org/p/cinder-zed-meetings

2 1

[nova] Do openstack support USB passthrough
by 韩光宇 04 Jan '23

04 Jan '23

Hi, all I want to ask if openstack support USB passthrough now? Or if I want the instance to recognize the USB flash drive on the host, do you have any suggestions? Thanks, Han

6 5

2023 PTGs + Virtual Team Signup
by Kendall Nelson 04 Jan '23

04 Jan '23

Hello Everyone! The next PTG on the calendar will be our usual virtual format and it will take place March 27 -31 right after the 2023.1 release. The project signup survey is already ready to go! If your team is interested in participating, sign them up here[1]! Registration is also open now[2] We have also heard many requests from the contributor community to bring back in person events and also heard that with global travel reduction, it would be better travel wise for the community at large to combine it with the OpenInfra Summit[3]. So, an in-person PTG will take place Wednesday and Thursday! June 14 -15th. OpenInfra Summit registration will be required to participate, and contributor pricing and travel support[4] will be available. We will also be planning another virtual PTG in the latter half of the year. The exact date is still being finalized. This will give you three opportunities to get your team together in 2023: two virtual and one in-person. - Virtual: March 27-31 - In- Person: June 14-15 collocated with the OpenInfra Summit - Virtual: TBD, but second half of the year As any project in our community, you are free to take advantage of all or some of the PTGs this coming year. I, for one, hope to see you and your project at as many as make sense for your team! To make sure we are as inclusive as possible I will continue to encourage team moderators to write summaries of their discussions to gather and promote after the events as well. Stay tuned for more info as we get closer, but the team signup survey[1] has opened for the first virtual PTG and is ready for you! -Kendall (diablo_rojo) [1] https://openinfrafoundation.formstack.com/forms/march2023_vptg_survey [2] https://openinfra-ptg.eventbrite.com [3]https://openinfra.dev/summit/vancouver-2023 [4] https://openinfrafoundation.formstack.com/forms/openinfra_tsp

1 1

[cloudkitty][horizon] change service name list in GUI
by Satish Patel 03 Jan '23

03 Jan '23

Folks, I am trying to configure rating for network data bytes in/out but what I have seen is that the ceilometer uses a different name than what cloudkitty GUI drop down-menu has. Ceilometer using "network.outgoing.bytes" and "network.incoming.bytes" Cloudkitty GUI drop-down list has "network.outgoing.bytes.rate" & "network.incoming.bytes.rate" I have tried to change it in cloudkitty metrics.yml file to match it with the ceilometer but the rating is not calculating until I change that in cloudkitty. Please advise how to change it in cloudkitty.

2 1

[ansible][ACO] - Contribution questions
by Gaël THEROND 31 Dec '22

31 Dec '22

Hi ansible collections openstack team! I finally had time to list all my issues met with the project, created few bug reports and even contributed to a patch today (minor, mainly copy/paste) however I’ve few questions regarding the CI process! Overall, what’s the rule with the CI code testing? I’ve read the contributing guide and had an eye on previous patches to see how it’s used but I’m having a hard time to find a real unified method. For instance, it seems that some module miss CI tasks (such as compute_service_info) or did I missed something? Thanks a lot for all the good job!

4 7

[release] Release countdown for week R-11, Jan 2 - 6
by Thierry Carrez 30 Dec '22

30 Dec '22

Development Focus ----------------- The Antelope-2 milestone is next week, on January 5, 2023! Antelope-related specs should now be finalized so that teams can move to implementation ASAP. Some teams observe specific deadlines on the second milestone (mostly spec freezes): please refer to https://releases.openstack.org/antelope/schedule.html for details. General Information ------------------- Libraries need to be released at least once per milestone period. Next week, the release team will propose releases for any library that has not been otherwise released since milestone 1. PTL's and release liaisons, please watch for these and give a +1 to acknowledge them. If there is some reason to hold off on a release, let us know that as well. A +1 would be appreciated, but if we do not hear anything at all by the end of the week, we will assume things are OK to proceed. Remember that non-library deliverables that follow the cycle-with-intermediary release model should have an intermediary release before milestone-2. Those who haven't will be proposed to switch to the cycle-with-rc model, which is more suited to deliverables that are released only once per cycle. Next week is also the deadline to freeze the contents of the final release. All new 'Antelope' deliverables need to have a deliverable file in https://opendev.org/openstack/releases/src/branch/master/deliverables and need to have done a release by milestone-2. Changes proposing those deliverables for inclusion in Antelope have been posted, please update them with an actual release request before the milestone-2 deadline if you plan on including that deliverable in Antelope, or -1 if you need one more cycle to be ready. Upcoming Deadlines & Dates -------------------------- Antelope-2 Milestone: January 5th, 2023 OpenInfra Summit Vancouver CFP deadline: January 10th, 2023 Final 2023.1 Antelope release: March 22nd, 2023 -- Thierry Carrez (ttx)

2 1

Re: [ops][nova] RBD IOPS bottleneck on client-side
by Ümit Seren 30 Dec '22

30 Dec '22

We had a similar performance issue with networking (via openswitch) instead of I/O. Our hypervisor and VM configuration were like yours (VCPU pinning + isolcpus). We saw a 50% drop in virtualized networking throughput (measure via iperf). This was because the vhost_net kthreads which are responsible for the virtualized networking were pinned to 2 cores per socket and this quickly became the bottleneck. This was with OpenStack Queens and RHEL 7.6. We ended up keeping the VCPU pinning but removing the isolcpus kernel setting. This fixed the performance regression. Unfortunately, we didn’t further investigate this, so I don’t know why a newer kernel and/or newer Openstack release improves it. Hope this still helps Best Ümit On 27.12.22, 13:33, "Can Özyurt" <acozyurt(a)gmail.com> wrote: Hi everyone, I hope you are all doing well. We are trying to pinpoint an IOPS problem with RBD and decided to ask you for your take on it. 1 control plane 1 compute node 5 storage with 8 SSD disks each Openstack Stein/Ceph Mimic deployed with kolla-ansible on ubuntu-1804 (kernel 5.4) isolcpus 4-127 on compute vcpu_pin_set 4-127 in nova.conf image_metadatas: hw_scsi_model: virtio-scsi hw_disk_bus: scsi flavor_metadatas: hw:cpu_policy: dedicated What we have tested: fio --directory=. --ioengine=libaio --direct=1 --name=benchmark_random_read_write --filename=test_rand --bs=4k --iodepth=32 --size=1G --readwrite=randrw --rwmixread=50 --time_based --runtime=300s --numjobs=16 1. First we run the fio test above on a guest VM, we see average 5K/5K read/write IOPS consistently. What we realize is that during the test, one single core on compute host is used at max, which is the first of the pinned cpus of the guest. 'top -Hp $qemupid' shows that some threads (notably tp_librbd) share the very same core throughout the test. (also emulatorpin set = vcpupin set as expected) 2. We remove isolcpus and every other configuration stays the same. Now fio tests now show 11K/11K read/write IOPS. No bottlenecked single cpu on the host, observed threads seem to visit all emulatorpins. 3. We bring isolcpus back and redeploy the cluster with Train/Nautilus on ubuntu-1804. Observations are identical to #1. 4. We tried replacing vcpu_pin_set to cpu_shared_set and cpu_dedicated_set to be able to pin emulator cpuset to 0-4 to no avail. Multiple guests on a host can easily deplete resources and IOPS drops. 5. Isolcpus are still in place and we deploy Ussuri with kolla-ansible and Train (to limit the moving parts) with ceph-ansible both on ubuntu-1804. Now we see 7K/7K read/write IOPS. 6. We destroy only the compute node and boot it with ubuntu-2004 with isolcpus set. Add it back to the existing cluster and fio shows slightly above 10K/10K read/write IOPS. What we think happens: 1. Since isolcpus disables scheduling between given cpus, qemu process and its threads are stuck at the same cpu which created the bottleneck. They should be runnable on any given emulatorpin cpus. 2. Ussuri is more performant despite isolcpus, with the improvements made over time. 3. Ubuntu-2004 is more performant despite isolcpus, with the improvements made over time in the kernel. Now the questions are: 1. What else are we missing here? 2. Are any of those assumptions false? 3. If all true, what can we do to solve this issue given that we cannot upgrade openstack nor ceph on production overnight? 4. Has anyone dealt with this issue before? We welcome any opinion and suggestions at this point as we need to make sure that we are on the right path regarding the problem and upgrade is not the only solution. Thanks in advance.

3 2

Glance api deployed only on a single controller on multi-controller deployment [kolla]
by A Monster 30 Dec '22

30 Dec '22

I've deployed openstack xena using kolla ansible on a centos 8 stream cluster, using two controller nodes, however I found out after the deployment that glance api is not available in one node, I tried redeploying but I got the same behavior, although the deployment finished without displaying any error. Thank you. Regards

4 4

[all][requirements][qa]: requirement-checks job broken for <=stable/yoga because it run on ubuntu-jammy on stable branch
by Ghanshyam Mann 29 Dec '22

29 Dec '22

Hello Everyone, While base jobs in opendev moved to run on ubuntu-jammy, requirement-checks job start running on ubuntu-jammy which is ok for the master but not for stable branches. As result, requirement-checks on stable branch running on ubuntu-jammy and started failing for stable/yoga and less: Error: No package matching 'python-dev' is available - https://zuul.opendev.org/t/openstack/build/811ca9311b894b97bfae4438070e5e22 Fixing it by setting the nodeset explicitly in that job so that it does not get changed in future when we move to ubuntu new version for testing. - https://review.opendev.org/q/Ie676079c0b798335ab2a82c03fe4715e0eee4d57 If you see the requirement-checks job failing (if there is a requirement change in the stable branch backport), please hold the recheck until the above fixes are merged. As tox4 fails the requirement gate, pushing the above fixes on top of tox4 fixes - https://review.opendev.org/q/I7bc7c98954395765f16cc943e5c826983db5dba0 -gmann

1 0

kolla openstack the freezer end point is not reachable
by K Santhosh 29 Dec '22

29 Dec '22

Hai , In kolla openstack the freezer end point is not reachable and the freezer api couldn't reach in web browser if any document is there make it to me

1 0

Container trying to access mounted device file
by vincent lee 29 Dec '22

29 Dec '22

Hi all, I am trying to access a mounted device file from my node to my newly created container. I am currently using Kolla-ansible for the deployment, and it is in the yoga version. For example, when I mount a device to my node, such as a radio receiver, the device file (/dev/mounted device name) is accessible on my node directly. However, I am looking for an option to enable the newly created container to access that device file. Best, Vincent

1 0

[Glance] Extending Specs Freeze Deadline by 2 weeks
by Pranali Deore 28 Dec '22

28 Dec '22

Hello All, As discussed in last glance meeting [1] we are extending the glance specs freeze deadline by 2 weeks due to unavailability of core members and less review bandwidth because of year end holidays. Current Spec Freeze date (for Glance) : 5th Jan 2023 New Spec Freeze date (for Glance) : 19th Jan 2023 Please make sure your specs are updated before the new deadline. [1]: https://meetings.opendev.org/meetings/glance/2022/glance.2022-12-22-14.00.l… Thanks, Pranali deore

1 0

Re: openstack-discuss Digest, Vol 50, Issue 63
by Kerem Çeliker 28 Dec '22

28 Dec '22

Hello Can, To fix an RBD (Rados Block Device) IOPS bottleneck on the client side in OpenStack, you can try the following: Monitor the CPU and memory usage on the client machine to ensure that it has sufficient resources. You can use tools like top or htop to view real-time resource usage. Check the network bandwidth between the client and the storage system to ensure that it is not a bottleneck. You can use tools like iperf or tcpdump to measure network performance. Review the configuration of the storage system to ensure that it is optimized for the workload. This may include adjusting the number and type of disks used, as well as the RAID level and chunk size. Consider using a storage system with a higher IOPS rating to see if it can improve performance. This may involve upgrading to faster disks or using a storage solution with more disks or SSDs. Try using a different client machine with more resources (e.g., a machine with a faster CPU and more memory) to see if it can issue more I/O requests. Consider using a different network connection between the client and the storage system, such as a faster network card or a direct connection rather than a network switch. If you are using Ceph as the underlying storage system, you can try adjusting the Ceph configuration to improve performance. This may include adjusting the number of placement groups, the object size, or the number of OSDs (object storage devices). It's also worth noting that an IOPS bottleneck can also occur on the server side (i.e., within the storage system itself). In this case, you may need to adjust the configuration of the storage system or add more resources (e.g., disks or SSDs) to improve performance. BR, Kerem Çeliker keremceliker.medium.com IBM | Red Hat Champion Sent from my iPhone > On 28 Dec 2022, at 08:13, openstack-discuss-request(a)lists.openstack.org wrote: > > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss > > or, via email, send a message with subject or body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > > Today's Topics: > > 1. [ops][nova] RBD IOPS bottleneck on client-side (Can ?zyurt) > 2. [keystone][Meeting] Reminder Keystone meeting is cancelled > today (Dave Wilde) > 3. Nova libvirt/kvm sound device (Zakhar Kirpichenko) > 4. Re: [Tacker][SRBAC] Update regarding implementation of > project personas in Tacker (Yasufumi Ogawa) > 5. Re: [Tacker][SRBAC] Update regarding implementation of > project personas in Tacker (manpreet kaur) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 27 Dec 2022 15:33:56 +0300 > From: Can ?zyurt <acozyurt(a)gmail.com> > To: OpenStack Discuss <openstack-discuss(a)lists.openstack.org> > Subject: [ops][nova] RBD IOPS bottleneck on client-side > Message-ID: > <CAMf4N71awOfNyBk4FfpM6UCjH4AZWz+QuJwUnOv+itvqvPbTjw(a)mail.gmail.com> > Content-Type: text/plain; charset="UTF-8" > > Hi everyone, > > I hope you are all doing well. We are trying to pinpoint an IOPS > problem with RBD and decided to ask you for your take on it. > > 1 control plane > 1 compute node > 5 storage with 8 SSD disks each > Openstack Stein/Ceph Mimic deployed with kolla-ansible on ubuntu-1804 > (kernel 5.4) > isolcpus 4-127 on compute > vcpu_pin_set 4-127 in nova.conf > > image_metadatas: > hw_scsi_model: virtio-scsi > hw_disk_bus: scsi > flavor_metadatas: > hw:cpu_policy: dedicated > > What we have tested: > fio --directory=. --ioengine=libaio --direct=1 > --name=benchmark_random_read_write --filename=test_rand --bs=4k > --iodepth=32 --size=1G --readwrite=randrw --rwmixread=50 --time_based > --runtime=300s --numjobs=16 > > 1. First we run the fio test above on a guest VM, we see average 5K/5K > read/write IOPS consistently. What we realize is that during the test, > one single core on compute host is used at max, which is the first of > the pinned cpus of the guest. 'top -Hp $qemupid' shows that some > threads (notably tp_librbd) share the very same core throughout the > test. (also emulatorpin set = vcpupin set as expected) > 2. We remove isolcpus and every other configuration stays the same. > Now fio tests now show 11K/11K read/write IOPS. No bottlenecked single > cpu on the host, observed threads seem to visit all emulatorpins. > 3. We bring isolcpus back and redeploy the cluster with Train/Nautilus > on ubuntu-1804. Observations are identical to #1. > 4. We tried replacing vcpu_pin_set to cpu_shared_set and > cpu_dedicated_set to be able to pin emulator cpuset to 0-4 to no > avail. Multiple guests on a host can easily deplete resources and IOPS > drops. > 5. Isolcpus are still in place and we deploy Ussuri with kolla-ansible > and Train (to limit the moving parts) with ceph-ansible both on > ubuntu-1804. Now we see 7K/7K read/write IOPS. > 6. We destroy only the compute node and boot it with ubuntu-2004 with > isolcpus set. Add it back to the existing cluster and fio shows > slightly above 10K/10K read/write IOPS. > > > What we think happens: > > 1. Since isolcpus disables scheduling between given cpus, qemu process > and its threads are stuck at the same cpu which created the > bottleneck. They should be runnable on any given emulatorpin cpus. > 2. Ussuri is more performant despite isolcpus, with the improvements > made over time. > 3. Ubuntu-2004 is more performant despite isolcpus, with the > improvements made over time in the kernel. > > Now the questions are: > > 1. What else are we missing here? > 2. Are any of those assumptions false? > 3. If all true, what can we do to solve this issue given that we > cannot upgrade openstack nor ceph on production overnight? > 4. Has anyone dealt with this issue before? > > We welcome any opinion and suggestions at this point as we need to > make sure that we are on the right path regarding the problem and > upgrade is not the only solution. Thanks in advance. > > > > ------------------------------ > > Message: 2 > Date: Tue, 27 Dec 2022 07:00:30 -0800 > From: Dave Wilde <dwilde(a)redhat.com> > To: openstack-discuss(a)lists.openstack.org > Subject: [keystone][Meeting] Reminder Keystone meeting is cancelled > today > Message-ID: > <CAJEkJ+qYCHZayR-t9w4e=ZB846CZSrggcqdYKE85fWkj2XAo5w(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Just a quick reminder that there won?t be the keystone weekly meeting > today. We?ll resume our regulararly scheduled programming on 03-Jan-2023. > Please update the agenda if you have anything you?d like to discuess. The > reviewathon is also cancelled this week, to be resumed on 06-Jan-2023. > > /Dave >

1 0

[Tacker][SRBAC] Update regarding implementation of project personas in Tacker
by manpreet kaur 28 Dec '22

28 Dec '22

Hi Ogawa san and Tacker team, This mailer is regarding the SRBAC implementation happening in Tacker. In the Tacker release 2023.1 virtual PTG [1], it was decided by the Tacker community to partially implement the project personas (project-reader role) in the current release. And in upcoming releases, we will implement the remaining project-member role. To address the above requirement, I have prepared a specification [2] and pushed the same in Gerrit for community review. Ghanshyam san reviewed the specification and shared TC's opinion and suggestion to implement both the project-reader and project-member roles. The complete persona implementation will depreciate the 'owner' rule, and help in restricting any other role to accessing project-based resources. Additionally, intact legacy admin (current admin), works in the same way so that we do not break things and introduce the project personas which should be additional things to be available for operators to adopt. Current Status: Incorporated the new requirement and uploaded a new patch set to address the review comment. Note: The Tacker spec freeze date is 28th Dec 2022, there might be some delay in merging the specification in shared timelines. [1] https://etherpad.opendev.org/p/tacker-antelope-ptg#L186 [2] https://review.opendev.org/c/openstack/tacker-specs/+/866956 Thanks & Regards, Manpreet Kaur

3 3

Re: openstack-discuss Digest, Vol 50, Issue 62
by Kerem Çeliker 27 Dec '22

27 Dec '22

Hello Vincent, To allow a user from a container to access files on OpenStack, you can use the --volume or -v option when creating the container to mount a host file or directory as a volume inside the container. For example: Copy code docker run -it --volume /path/on/host:/path/in/container image_name This will mount the host file or directory located at /path/on/host as a volume inside the container at /path/in/container. The user inside the container will be able to access the files in the volume at /path/in/container. You can also use the --mount option to mount a volume. This option provides more control over the volume, such as specifying the volume driver and options. For example: Copy code docker run -it --mount type=bind,source=/path/on/host,target=/path/in/container image_name This will also mount the host file or directory located at /path/on/host as a volume inside the container at /path/in/container, and the user inside the container will be able to access the files in the volume at /path/in/container. Note that the /path/on/host must be an absolute path and must exist on the host system. The /path/in/container can be any path inside the container and does not need to exist prior to the mount. It's also important to note that the user inside the container must have the appropriate permissions to access the mounted files. You can use the --user option to specify the user that the container should run as, or you can set the ownership and permissions of the mounted files on the host system to allow the user inside the container to access them. Hope it works for you.. — Best, Regards, Kerem Çeliker IBM | Red Hat Champion keremceliker.medium.com Sent from my iPhone > On 27 Dec 2022, at 15:07, openstack-discuss-request(a)lists.openstack.org wrote: > Send openstack-discuss mailing list submissions to > openstack-discuss(a)lists.openstack.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss > > or, via email, send a message with subject or body 'help' to > openstack-discuss-request(a)lists.openstack.org > > You can reach the person managing the list at > openstack-discuss-owner(a)lists.openstack.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of openstack-discuss digest..." > > > Today's Topics: > > 1. Container access mounted device (vincent lee) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 27 Dec 2022 19:01:21 +0800 > From: vincent lee <vincentlee676(a)gmail.com> > To: openstack-discuss(a)lists.openstack.org > Subject: Container access mounted device > Message-ID: > > > Hi, I have deployed a working OpenStack and would like to know if there are > any possible options I can enable so that when creating a new container, > files can be mounted to it, and a user from the container can access it. > For instance, having a mounted device from the compute host and allowing > the /dev/(mounted device name) file to be accessible in the container. > > Best regards, > Vincent >

1 0

[keystone][Meeting] Reminder Keystone meeting is cancelled today
by Dave Wilde 27 Dec '22

27 Dec '22

Just a quick reminder that there won’t be the keystone weekly meeting today. We’ll resume our regulararly scheduled programming on 03-Jan-2023. Please update the agenda if you have anything you’d like to discuess. The reviewathon is also cancelled this week, to be resumed on 06-Jan-2023. /Dave

1 0

[ops][nova] RBD IOPS bottleneck on client-side
by Can Özyurt 27 Dec '22

27 Dec '22

Hi everyone, I hope you are all doing well. We are trying to pinpoint an IOPS problem with RBD and decided to ask you for your take on it. 1 control plane 1 compute node 5 storage with 8 SSD disks each Openstack Stein/Ceph Mimic deployed with kolla-ansible on ubuntu-1804 (kernel 5.4) isolcpus 4-127 on compute vcpu_pin_set 4-127 in nova.conf image_metadatas: hw_scsi_model: virtio-scsi hw_disk_bus: scsi flavor_metadatas: hw:cpu_policy: dedicated What we have tested: fio --directory=. --ioengine=libaio --direct=1 --name=benchmark_random_read_write --filename=test_rand --bs=4k --iodepth=32 --size=1G --readwrite=randrw --rwmixread=50 --time_based --runtime=300s --numjobs=16 1. First we run the fio test above on a guest VM, we see average 5K/5K read/write IOPS consistently. What we realize is that during the test, one single core on compute host is used at max, which is the first of the pinned cpus of the guest. 'top -Hp $qemupid' shows that some threads (notably tp_librbd) share the very same core throughout the test. (also emulatorpin set = vcpupin set as expected) 2. We remove isolcpus and every other configuration stays the same. Now fio tests now show 11K/11K read/write IOPS. No bottlenecked single cpu on the host, observed threads seem to visit all emulatorpins. 3. We bring isolcpus back and redeploy the cluster with Train/Nautilus on ubuntu-1804. Observations are identical to #1. 4. We tried replacing vcpu_pin_set to cpu_shared_set and cpu_dedicated_set to be able to pin emulator cpuset to 0-4 to no avail. Multiple guests on a host can easily deplete resources and IOPS drops. 5. Isolcpus are still in place and we deploy Ussuri with kolla-ansible and Train (to limit the moving parts) with ceph-ansible both on ubuntu-1804. Now we see 7K/7K read/write IOPS. 6. We destroy only the compute node and boot it with ubuntu-2004 with isolcpus set. Add it back to the existing cluster and fio shows slightly above 10K/10K read/write IOPS. What we think happens: 1. Since isolcpus disables scheduling between given cpus, qemu process and its threads are stuck at the same cpu which created the bottleneck. They should be runnable on any given emulatorpin cpus. 2. Ussuri is more performant despite isolcpus, with the improvements made over time. 3. Ubuntu-2004 is more performant despite isolcpus, with the improvements made over time in the kernel. Now the questions are: 1. What else are we missing here? 2. Are any of those assumptions false? 3. If all true, what can we do to solve this issue given that we cannot upgrade openstack nor ceph on production overnight? 4. Has anyone dealt with this issue before? We welcome any opinion and suggestions at this point as we need to make sure that we are on the right path regarding the problem and upgrade is not the only solution. Thanks in advance.

1 0

Container access mounted device
by vincent lee 27 Dec '22

27 Dec '22

Hi, I have deployed a working OpenStack and would like to know if there are any possible options I can enable so that when creating a new container, files can be mounted to it, and a user from the container can access it. For instance, having a mounted device from the compute host and allowing the /dev/(mounted device name) file to be accessible in the container. Best regards, Vincent

1 0

[openstacksdk][cinder] openstacksdk-functional-devstack job is failing
by Rajat Dhasmana 27 Dec '22

27 Dec '22

Hi, Currently the *openstacksdk-functional-devstack* job is failing on the cinder gate[1] due to which gate is blocked. This seems to be due to the new tox 4 enforcement and could be an issue for other projects as well. We've a patch in OpenstackSDK that fixes this issue[2] but that is also failing on the* osc-tox-py38-tips* job. The patch has no update since 21st December which is concerning since it's blocking other projects (at least cinder). I've proposed a patch[3] to make the *openstacksdk-functional-devstack *job non-voting until the situation is resolved on the OpenStackSDK side. [1] https://zuul.opendev.org/t/openstack/build/29c0eb15abd144a8a4f167e70fe61530 [2] https://review.opendev.org/c/openstack/openstacksdk/+/867827 [3] https://review.opendev.org/c/openstack/cinder/+/868596 Thanks Rajat Dhasmana

2 2

[openstack][cinder] Cinder volume id wont change when retype volume
by Nguyễn Hữu Khôi 27 Dec '22

27 Dec '22

Hello guys, I have a problem with cinder: I use cinder volume retype to move my volume to another backend. It worked but when I check on new backend, volume id doest match with id which on Horizon dash board.It expects to have same volume id on Horizon and new backend. I use DellSC and DellPowerstore. Thank you. Nguyen Huu Khoi

2 2

Help overriding Nova policies (Ussuri)
by Jérôme BECOT 26 Dec '22

26 Dec '22

Hello, I should miss something, and I need some help. I've updated some rules that I placed in the /etc/nova/policy.yaml file. I use all defaults for the olso policies (ie scope is not enabled). When I generate the fulle policy with oslopolicy-policy-generator, the rules are applied in the generated output. If I test the policy with oslopolicy-checker under the user's token, the results matches the permissions, but Nova API continue to refuse the operation Policy check for os_compute_api:os-flavor-manage:create failed with credentials I'm using the openstack cli to make the request (openstack flavor create) Thanks for your help J

2 2

[tacker] Cancelling next irc meetings
by Yasufumi Ogawa 26 Dec '22

26 Dec '22

Hi team, Due to the holidays, we are cancelling next two meetings on 27th Dec and 3rd Jun. Cheers, Yasufumi

1 0

[kolla-ansible] telemetry data
by Satish Patel 26 Dec '22

26 Dec '22

Folks, I am deployed kolla-ansible and trying to set up telemetry for billing etc. But not able to get it working may be broken or something. In global.yml I turned on the following options and deployed kolla and it did install all the components but I am not seeing it deploy a ceilometer agent on compute nodes. How does it collect data from compute nodes if the ceilometer agent is not running on compute nodes? Did I miss something here? enable_ceilometer: "yes" enable_gnocchi: "yes" enable_cloudkitty: "yes"

2 2

The CFP for the OpenInfra Summit 2023 is open!
by Erin Disney 26 Dec '22

26 Dec '22

Hi Everyone! The CFP for the 2023 OpenInfra Summit (June 13-15, 2023) is NOW LIVE [1]! Check out the full list of tracks and submit a talk on your topic of expertise [2]. The CFP closes January 10, 2023, at 11:59 p.m. PT We are also now accepting submissions for Forum sessions [3]! What should you submit to the forum vs. the traditional CFP? For the Forum, submit discussion-oriented sessions, including challenges around different software components, working group progress or best practices to tackle common issues. Looking for other resources? Registration [4], sponsorships [5], travel support [6] and visa requests [7] are also all open! Find all the information on the OpenInfra Summit 2023 in one place [8]! Cheers, Erin [1] https://cfp.openinfra.dev/app/vancouver-2023/19/presentations <https://cfp.openinfra.dev/app/vancouver-2023/19/presentationshttps://cfp.op…> [2] https://openinfra.dev/summit/vancouver-2023/summit-tracks/ <https://openinfra.dev/summit/vancouver-2023/summit-tracks/> [3] https://cfp.openinfra.dev/app/vancouver-2023/20/ <https://cfp.openinfra.dev/app/vancouver-2023/20/> [4] http://openinfra.dev/summit/registration <http://openinfra.dev/summit/registration> [5] http://openinfra.dev/summit/vancouver-2023/summit-sponsor/ <http://openinfra.dev/summit/vancouver-2023/summit-sponsor/> [6] https://openinfrafoundation.formstack.com/forms/openinfra_tsp <https://openinfrafoundation.formstack.com/forms/openinfra_tsp> [7] https://openinfrafoundation.formstack.com/forms/visa_yvrsummit2023 <https://openinfrafoundation.formstack.com/forms/visa_yvrsummit2023> [8] https://openinfra.dev/summit/vancouver-2023 <https://openinfra.dev/summit/vancouver-2023>

2 1

the zun service run is OK , but at dashboard web no view zun serivce!
by 无数的星球 24 Dec '22

24 Dec '22

After I successfully installed the zun service, I logged in to the dashboard, but I could not see the zun entry I've rebooted all the machines，but invalid. thanks you !

2 1

Container access mounted device
by vincent lee 24 Dec '22

24 Dec '22

Hi, I have deployed a working OpenStack and would like to know if there are any possible options I can enable so that when creating a new container, files can be mounted to it, and a user from the container can access it. For instance, having a mounted device from the compute host and allowing the /dev/(mounted device name) file to be accessible in the container. Best regards, Vincent

2 1

the zun server (zun-api) in controller host error(s)
by 无数的星球 24 Dec '22

24 Dec '22

iAfter I followed the official documentation and fixed a 'pip3' error, it still failed to run successfully. This is an error reported by the zun api. Other components work normally. If you need my profile, it is attached root@controller:~# systemctl status zun-api ● zun-api.service - OpenStack Container Service API      Loaded: loaded (/etc/systemd/system/zun-api.service; enabled; vendor preset: enabled)      Active: failed (Result: exit-code) since Sun 2022-12-11 11:43:04 CST; 1min 38s ago     Process: 1017 ExecStart=/usr/local/bin/zun-api (code=exited, status=1/FAILURE)    Main PID: 1017 (code=exited, status=1/FAILURE) Dec 11 11:43:04 controller zun-api[1017]:     from zun import objects Dec 11 11:43:04 controller zun-api[1017]:   File "/usr/local/lib/python3.8/dist-packages/zun/objects/__init__.py", line 13, in <module> Dec 11 11:43:04 controller zun-api[1017]:     from zun.objects import compute_node Dec 11 11:43:04 controller zun-api[1017]:   File "/usr/local/lib/python3.8/dist-packages/zun/objects/compute_node.py", line 16, in <module> Dec 11 11:43:04 controller zun-api[1017]:     from zun.db import api as dbapi Dec 11 11:43:04 controller zun-api[1017]:   File "/usr/local/lib/python3.8/dist-packages/zun/db/api.py", line 32, in <module> Dec 11 11:43:04 controller zun-api[1017]:     (a)profiler.trace("db") Dec 11 11:43:04 controller zun-api[1017]: AttributeError: partially initialized module 'zun.common.profiler' has no attribute 'trace' (most likely due to a circular import) Dec 11 11:43:04 controller systemd[1]: zun-api.service: Main process exited, code=exited, status=1/FAILURE Dec 11 11:43:04 controller systemd[1]: zun-api.service: Failed with result 'exit-code'.

2 1

[all][tc] What's happening in Technical Committee: summary 2022 Dec 23: Reading: 5 min
by Ghanshyam Mann 24 Dec '22

24 Dec '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on Dec 21. Most of the meeting discussions are summarized in this email. Meeting logs are available @ https://meetings.opendev.org/meetings/tc/2022/tc.2022-12-21-16.00.log.html * The TC meeting on Dec 28 is cancelled [1]. The next TC weekly meeting will be on Jan 4 Wed at 16:00 UTC, Feel free to add the topic to the agenda[2] by Jan 3. 2. What we completed this week: ========================= * Made inactive projects timeline as release milestone-2 of the cycle[3] 3. Activities In progress: ================== TC Tracker for the 2023.1 cycle ------------------------------------- * Current cycle working items and their progress are present in the 2023.1 tracker etherpad[4]. Open Reviews ----------------- * Three open reviews for ongoing activities[5]. IMPORTANT: Tox 4 failure and time to fix them -------------------------------------------------------- tox is uncapped in ensure-tox [6]. With that tox4 is used in unit/functional tests job (or any job using ensure-role. If you are seeing failure in your project's jobs, please fix those on priority and if it needs time then you can cap tox it in your project zuul.yaml or for the particular job via 'ensure_tox_version' var. Mistral release and more maintainers ------------------------------------------- avanzagh who is the new maintainer of Mistral is in discussion with the release team for the required work for the Mistral release. We will keep monitoring the situation until the proposal for Mistral release deprecation is closed[7]. Project updates ------------------- * Add Cinder Huawei charm[8] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[9]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15:00 UTC [10] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://lists.openstack.org/pipermail/openstack-discuss/2022-December/03156… [2] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda_Suggesti… [3] https://review.opendev.org/c/openstack/governance/+/867062 [4] https://etherpad.opendev.org/p/tc-2023.1-tracker [5] https://review.opendev.org/q/projects:openstack/governance+status:open [6] https://review.opendev.org/c/zuul/zuul-jobs/+/866943 [7] https://review.opendev.org/c/openstack/governance/+/866562 [8] https://review.opendev.org/c/openstack/governance/+/867588 [9] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [10] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

[manila] Cancelling Dec 29th weekly meeting
by Carlos Silva 23 Dec '22

23 Dec '22

Hello, Zorillas! As discussed in yesterday's weekly meeting, next week's IRC meeting (Dec 29th) is cancelled due to most of us being offline. The next weekly meeting is on January 5th. Happy holidays to those celebrating! Cheers, carloss

1 0

[neutron] bug deputy report December 19th -23th
by Miguel Lavalle 23 Dec '22

23 Dec '22

Hi, Here's this week's bugs deputy report: Under investigation at the moment of this report ==================================== https://bugs.launchpad.net/neutron/+bug/2000378 [OVN] orphaned virtual parent ports break new ports. Under investigation by ralonsoh Medium ====== https://bugs.launchpad.net/neutron/+bug/2000163 [FT] Error in "test_get_datapath_id". Proposed fix: https://review.opendev.org/c/openstack/neutron/+/868311 https://bugs.launchpad.net/neutron/+bug/2000164 [FT] Error in "test_dvr_update_gateway_port_with_no_gw_port_in_namespace". Needs owner https://bugs.launchpad.net/neutron/+bug/2000252 [OVN] "DBInconsistenciesPeriodics.check_for_inconsistencies" failing during port deletion. Needs owner https://bugs.launchpad.net/neutron/+bug/2000314 [OVS] Call to "driver.unregister" failing. Needs owner Low === https://bugs.launchpad.net/neutron/+bug/2000238 [FT] Remove the "Running command:" messages in the logs. Proposed fix: https://review.opendev.org/c/openstack/neutron/+/868304

1 0

[nova] Isolating scheduler
by Dominik Danelski 23 Dec '22

23 Dec '22

Hello, I'd like to create and test some new filters and weights for the scheduler. Ideally, I would mimic an existing OpenStack deployment I have access to. I already installed placement component, to which the scheduler would connect and now I'd like to isolate it. I'm not sure how to approach it, as scheduler is part of Nova and not a stand-alone component as placement. My intention is to initially fill placement's database with information about the infrastructure, set up the scheduler and then query it for consecutive requests (linear, no need for locks if that changes anything) to log the decisions it would make with modified weights/filters and the state of the system in-between requests. What is, in your opinion, the best way to detach the scheduler as much as possible from the rest of Nova, in order to test it under configuration most resembling the real-world setup? Should I use the test framework and mimic how it is set up for the unit tests or is there a better way to go? Thank you for your time. Regards Dominik Danelski

1 0

[neutron] Neutron drivers meeting cancelled
by Rodolfo Alonso Hernandez 23 Dec '22

23 Dec '22

Hello Neutrinos: Due to the lack of agenda, today's drivers meeting is cancelled. Please spend some time reviewing the active specs from the RFEs approved last week: https://review.opendev.org/q/project:openstack/neutron-specs+status:open+-a… . See you next year!

1 0

[all][tc] Canceling next week TC meetings
by Ghanshyam Mann 22 Dec '22

22 Dec '22

Hello Everyone, Due to the holidays, we are cancelling next week's (Dec 28) TC meeting. -gmann

1 0

[Glance] Cancelling Weekly meeting for Next 2 weeks
by Pranali Deore 22 Dec '22

22 Dec '22

Hello, Upstream glance weekly meeting for next 2 weeks is cancelled as most of the team members will be on New Year Holidays. The Next weekly meeting will be on Jan 12th ! Wishing everyone a Wonderful Holiday Season & a Joyful New Year. Thanks, Pranali

1 0

[Octavia] cancelling meeting Dec 28
by Gregory Thiemonge 22 Dec '22

22 Dec '22

Hi, Next week's meeting (Dec 28th) is cancelled, The next weekly meeting will be on Jan 4th! Gregory

1 0

Kolla ansible, add a service to a running openstack deployment
by A Monster 21 Dec '22

21 Dec '22

is there a way to add a service such as elasticsearch to an already done openstack deployment using kolla-ansible?

2 1

[cinder] Cancelling meeting on 28th December
by Rajat Dhasmana 21 Dec '22

21 Dec '22

Hello Argonauts, As discussed in today's meeting[1], we will be cancelling the upstream cinder meeting on 28st December, 2022 and will meet in the meeting on 4th January, 2023. Wishing everyone Merry Christmas and a happy new year! [1] https://meetings.opendev.org/meetings/cinder/2022/cinder.2022-12-21-14.00.l… Thanks Rajat Dhasmana

1 0

[Openstack Heat] - Problems to find right Heat syntax to declare static route in a router template
by ddorra＠t-online.de 21 Dec '22

21 Dec '22

Hi Lajos, many thanks for your hints! Dieter > Hi, > > For adding extra/static routes for your routers you have 2 ways in Heat > (actually Neutron API has it, and you can use them in hot also): > > OS::Neutron::ExtraRoute https://docs.openstack.org/api-ref/network/v2/index.html#update-router > > OS::Neutron::ExtraRouteSethttps://docs.openstack.org/api-ref/network/v2/ind… > > The 1st one is the "old" set router attribute, and the 2nd is add_extraroutes & remove_extraroutes is optimized for concurrent > updates (see the API ref's > description for add_extraroutes). > So in hot template it will look something like this: > type: OS::Neutron::ExtraRouteSet > properties: > router: { get_resource: myrouter0 } > routes: > - destination: 179.24.2.0/24 > nexthop: 192.168.222.221 > > Lajos > > > ddorra(a)t-online.de <ddorra(a)t-online.de> ezt írta (időpont: 2022. dec. 19., H, 21:43): > > Hi, > > > > I want extend my router HOT template by adding a static route. > > > > In the Openstack cli this would be > > > > openstack router add route --route destination='1.2.3.4/17',gateway='10.0.0.66' myrouter > > > > From the OS::Neutron::Router resource description I guess this belongs into the value_specs > > section as a map, but all my attempts failed, e.g. > > > > router1: > > type: 'OS::Neutron::Router' > > properties: > > external_gateway_info: > > network: provider > > name: myrouter > > value_specs: > > route: > > destination: '0.0.0.0/0' > > gateway: '10.0.0.62' > > > > Can somebody help me with the correct syntax? > > > > > > Thanks & best regards, > > Dieter

1 0

[cinder] Bug Report from 12-21-2022
by Sofia Enriquez 21 Dec '22

21 Dec '22

This is a bug report from 12-14-2022 to 12-21-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1999706 "Cinder fails to automatically map availability zones to volume types." Unassigned. Medium - https://bugs.launchpad.net/cinder/+bug/1999766 "Retype VM root Volume Cause VM failed to boot." Unassigned. Cheers, Sofia -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

Magnum private docker registry (insecure_registry) not working?
by Oliver Weinmann 21 Dec '22

21 Dec '22

Hi,I can't seem to get magnum (k8s) to accept my private docker registry. I wanted to have a central registry so not all hosts pull the images during deployment.For this I configured a registry:v2 docker container, pulled the images and pushed them to the local registry and added the following label to my k8s template:container_infra_prefix=172.28.7.140:4000/At first this seems to be working fine and when deploying a new k8s cluster using magnum I can see that it pulls the heat-container-agent image from my local registry:[core@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]$ sudo podman ps -aCONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES2d08559b9cdc 172.28.7.140:4000/heat-container-agent:wallaby-stable-1 /usr/bin/start-he... 1 second ago Up 1 second ago heat-container-agentBut then it fails to pull the next container:tail -f /var/log/heat-config/heat-config-script/64d35aad-5453-4da4-97c7-45abb640fc90-k8s-admin-test-local-reg-6c4hx7gxbdhr-kube_masters-h3wbcqgm6qv4-0-sfagopiu52se-master_config-2f5lhvr32z7j.logWARNING Attempt 8: Trying to install kubectl. Sleeping 5s+ ssh -F /srv/magnum/.ssh/config root@localhost '/usr/bin/podman run --entrypoint /bin/bash --name install-kubectl --net host --privileged --rm --user root --volume /srv/magnum/bin:/host/srv/magnum/bin 172.28.7.140:4000/hyperkube:v1.23.3-rancher1 -c '\''cp /usr/local/bin/kubectl /host/srv/magnum/bin/kubectl'\'''Trying to pull 172.28.7.140:4000/hyperkube:v1.23.3-rancher1...Error: initializing source docker://172.28.7.140:4000/hyperkube:v1.23.3-rancher1: pinging container registry 172.28.7.140:4000: Get "https://172.28.7.140:4000/v2/": http: server gave HTTP response to HTTPS clientI don't know why but there is no /etc/docker/daemon.json and the /etc/sysconfig/docker also doesn'T contain the line for my insecure registry:root@k8s-admin-test-local-reg-6c4hx7gxbdhr-master-0 ~]# cat /etc/sysconfig/docker# /etc/sysconfig/docker# Modify these options if you want to change the way the docker daemon runsOPTIONS="--selinux-enabled \ --log-driver=journald \ --live-restore \ --default-ulimit nofile=1024:1024 \ --init-path /usr/libexec/docker/docker-init \ --userland-proxy-path /usr/libexec/docker/docker-proxy \"As soon as I manually add my insecure registry here it works just fine. I looked at the magnum code and there is indeed some lines that should actually handle this, but it doesn't seem to be working. What is also weird is that while there is the Option in the Horizon WebUI to set an insecure registry, the openstack coe command doesn't offer this.Best Regards,Oliver

1 1

[kolla] Cancelling meetings on 21 and 28 Dec
by Michał Nasiadka 21 Dec '22

21 Dec '22

Hello Koalas, Cancelling this and next weekly meeting due to Christmas/New Year period. See you next year! Best regards, Michal

1 0

horizon policy
by Parsa Aminian 21 Dec '22

21 Dec '22

Hello is it possible to prevent access to delete instance on horizon for some users ? I want to access only admin user to delete instances from dashboard

1 0

Instance console works only if opened from one of the deployment nodes
by A Monster 21 Dec '22

21 Dec '22

I've deployed openstack yoga using kolla ansible, in three controller nodes, a storage cluster and multiple compute nodes. The deployment included using haproxy, keepalived and hacluster. everything seems to work fine, I can launch instances and networking is doing great, however I found out a problem that I didn't get in all the deployment tests that i've run before. I've combien openstack networks under the same network 10.10.10.0/24 and used the same ip address for both : *kolla_internal_vip_address* and *kolla_external_vip_interface *which is 10.10.10.254, and used an ip route to access to the openstack cluster from an external pc > ip route add 10.10.10.0/24 via 192.168.129.29 dev eth0 where *192.168.129.29* is the address of one of my controllers, however when I try to login to the horizon dashboard using *10.10.10.254 *i don't get a response from the web page, and when I log to the dashboard using the ip address of one of my controller nodes, I cannot use the instance console since it is under the ip address of 10.10.10.254 which is assigned to both *kolla_internal_vip_address* and *kolla_external_vip_interface.* although i can ping 10.10.10.254 from the external network using ip route.

1 0

[all][tc] Technical Committee next weekly meeting on 2022 Dec 21 at 1600 UTC
by Ghanshyam Mann 21 Dec '22

21 Dec '22

Hello Everyone, The technical Committee's next weekly meeting is scheduled for 2022 Dec 21, at 1600 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Tuesday, Dec 20 at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[horizon] Cancelling next three weekly meetings
by vishal manchanda 20 Dec '22

20 Dec '22

Hello Team, As agreed, during the last weekly meeting, we are canceling our weekly meeting on 21st, 28th December and 4th January. The next weekly meeting will be on 11th January. Happy holidays and a happy new year in advance! Thanks & Regards, Vishal Manchanda

1 0

[neutron] Bug deputy report for week of December 12th
by Rodolfo Alonso Hernandez 20 Dec '22

20 Dec '22

Hello Neutrinos: This is the bug report from last week. * Critical: ** https://bugs.launchpad.net/neutron/+bug/2000150: "Fullstack dhcp test test_multiple_agents_for_network is failing intermittently". *Not assigned* * High: ** https://bugs.launchpad.net/neutron/+bug/1999558: "tox4 is breaking Neutron CI (most of the jobs)". To be discussed during the weekly Neutron meeting. ** https://bugs.launchpad.net/neutron/+bug/2000071: "[ovn-octavia-provider] Do not make the status of a newly HM conditional on the status of existing members". Patch: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/868092 * Medium: ** https://bugs.launchpad.net/neutron/+bug/1999392: "Some events which are supposed to be run AFTER changes are committed are performed BEFORE commit". Assigned ** https://bugs.launchpad.net/neutron/+bug/1999678: "Static route can get stuck in the router snat namespace". Patch: https://review.opendev.org/c/openstack/neutron/+/867678 ** https://bugs.launchpad.net/neutron/+bug/1999774: "SDK: Neutron stadiums use python bindings from python-neutronclient which will be deprecated". This bug is a container for any patch on any project deprecating "python-neutronclient". ** https://bugs.launchpad.net/neutron/+bug/1999813: "[ovn-octavia-provider] when a HM is created/deleted the listener remains in PENDING_UPDATE". Patch: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/867974. ** https://bugs.launchpad.net/neutron/+bug/2000078: "neutron-remove-duplicated-port-bindings doesn't remove binding_levels". Assigned/ * Low ** https://bugs.launchpad.net/neutron/+bug/1999390: "TypeError raised in neutron-dhcp-agent due to missing argument in clean_devices method". *Not assigned.* ** https://bugs.launchpad.net/neutron/+bug/1999391: "Attribute error in neutron-ovs-agent".* Not assigned.* * Incomplete: ** https://bugs.launchpad.net/neutron/+bug/1999400: "neutron-metadata-agent does not sometimes provide instance-id". The issue seems to be an error during the cloud-init route assignment. Must be confirmed first. ** https://bugs.launchpad.net/neutron/+bug/1999540: "Tempest test test_two_vms_fips failed due to port binding on instance failing". Missing logs (not accessible). ** https://bugs.launchpad.net/neutron/+bug/1999677: "Defunct nodes are reported as happy in network agent list". This issue is already solved in newer versions (latest Ussuri release 16.4.2). Also newer releases allow the removal of the OVN agents using the CLI. Regards.

1 0

[Openstack Heat] - Problems to find right Heat syntax to declare static route in a router template
by ddorra＠t-online.de 20 Dec '22

20 Dec '22

Hi, I want extend my router HOT template by adding a static route. In the Openstack cli this would be openstack router add route --route destination='1.2.3.4/17',gateway='10.0.0.66' myrouter >From the OS::Neutron::Router resource description I guess this belongs into the value_specs section as a map, but all my attempts failed, e.g. router1: type: 'OS::Neutron::Router' properties: external_gateway_info: network: provider name: myrouter value_specs: route: destination: '0.0.0.0/0' gateway: '10.0.0.62' Can somebody help me with the correct syntax? Thanks & best regards, Dieter

2 1

OpenStack Release Name Voting Now Open!
by Erin Disney 20 Dec '22

20 Dec '22

Hey everyone- It’s that time again to vote for the next OpenStack release name! Here are the finalists: 1. Bandicoot 2. Basalt 3. Bobcat 4. Boomerang Please submit your vote by January 3rd at 11:59pm PT (January 4th at 7:59 UTC). https://civs1.civs.us/cgi-bin/vote.pl?id=E_9c7f7a20a8871b09&akey=5b974247f9… <https://civs1.civs.us/cgi-bin/vote.pl?id=E_9c7f7a20a8871b09&akey=5b974247f9…> Happy voting! Thanks, Erin

2 1

Kolla-ansible openstack deployment with High Availability
by A Monster 18 Dec '22

18 Dec '22

Do I need to deploy and configure PaceMaker and HAproxy or something similar in order to get a HA kolla-ansible openstack deployment? or is it something done automatically? If so, what are the steps I should be taking?

3 2

[cinder] Extending specs deadline to 23rd December
by Rajat Dhasmana 17 Dec '22

17 Dec '22

Hello Argonauts, As per the schedule[1], cinder spec deadline was 16th December, 2022. However, due to unavailability of some core members, less review bandwidth throughout the week (including me) and other factors like bringing cinderlib to a release-able state, we couldn't get the sufficient reviews done and hence will be extending the deadline for all the open specs proposed for 2023.1 (Antelope) release. The new deadline for cinder specs will be 23rd December, 2022. Please make sure your specs are updated and in a reviewable state. [1] https://releases.openstack.org/antelope/schedule.html#a-cinder-spec-freeze Thanks Rajat Dhasmana

1 0

[all][TC] Bare rechecks stats week of 25.07
by Slawek Kaplonski 17 Dec '22

17 Dec '22

Hi, Here are fresh data about bare rechecks. +--------------------+---------------+--------------+-------------------+ | Team | Bare rechecks | All Rechecks | Bare rechecks [%] | +--------------------+---------------+--------------+-------------------+ | requirements | 2 | 2 | 100.0 | | keystone | 1 | 1 | 100.0 | | OpenStackSDK | 3 | 3 | 100.0 | | tacker | 3 | 3 | 100.0 | | freezer | 1 | 1 | 100.0 | | rally | 1 | 1 | 100.0 | | OpenStack-Helm | 14 | 15 | 93.33 | | cinder | 6 | 7 | 85.71 | | kolla | 20 | 25 | 80.0 | | neutron | 4 | 5 | 80.0 | | Puppet OpenStack | 10 | 13 | 76.92 | | OpenStack Charms | 3 | 6 | 50.0 | | nova | 11 | 22 | 50.0 | | tripleo | 8 | 19 | 42.11 | | ironic | 1 | 3 | 33.33 | | Quality Assurance | 1 | 3 | 33.33 | | octavia | 0 | 1 | 0.0 | | OpenStackAnsible | 0 | 1 | 0.0 | | designate | 0 | 1 | 0.0 | | Release Management | 0 | 1 | 0.0 | | horizon | 0 | 1 | 0.0 | +--------------------+---------------+--------------+-------------------+ Those data should be more accurate as my script now counts only comments from last 7 days, not all comments from patches updated in last 7 days. Reminder: "bare rechecks" are recheck comments without any reason given. If You need to do recheck for patch due to failed job(s), please first check such failed job and try to identify what was the issue there. Maybe there is already opened bug for that or maybe You can open new one and add it as explanation in the recheck comment. Or maybe it was some infra issue, in such case short explanation in the comment would also be enough. -- Slawek Kaplonski Principal Software Engineer Red Hat

6 10

[all][tc] What's happening in Technical Committee: summary 2022 Dec 16: Reading: 5 min
by Ghanshyam Mann 17 Dec '22

17 Dec '22

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * We had this week's meeting on Dec 14. Most of the meeting discussions are summarized in this email. Meeting logs are available @ https://meetings.opendev.org/meetings/tc/2022/tc.2022-12-14-16.00.log.html * Next TC weekly meeting will be on Dec 21 Wed at 16:00 UTC, Feel free to add the topic to the agenda[1] by Dec 20. 2. What we completed this week: ========================= * The technical elections dates for next cycle is finalized[2], please plan your nominations for open leaderships roles accordingly. 3. Activities In progress: ================== TC Tracker for the 2023.1 cycle ------------------------------------- * Current cycle working items and their progress are present in the 2023.1 tracker etherpad[3]. Open Reviews ----------------- * Three open reviews for ongoing activities[4]. IMPORTANT: Tox 4 failure and time to fix them -------------------------------------------------------- You might know about Tox 4 failure and currently it is capped as <4 in ensure-tox role. This is going to be uncapped on Dec 21[5] and you might see tox based unit/functional tests job (or any job using ensure-role except devstack based) will start failing. Please plan the testing and fix accordingly. Calling out here as holiday seasons or fixing it in early cycle will help to plan the smooth development cycle and release. Change in Inactive project timeline ----------------------------------------- The proposal for this is under review[6] and positive response so far. Mistral release and more maintainers ------------------------------------------- avanzagh joined TC meeting and updates us that new maintainers from OVHCloud have been added as Core members to help maintaining Mistral. And we also discussed and agreed to continue Mistral in DPL model for this cycle. As next step, avanzagh will discuss the required work to release Mistral or what are the expectations from the release team to consider it as release item. This seems like good progress. We will keep monitor the situation until proposal for Mistral release deprecation is closed[7]. Renovate translation SIG i18 ---------------------------------- * No updates this week and we will track it in TC tracker and update on ML if there is any update. Project updates ------------------- * Add Cinder Huawei charm[8] 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[9]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15:00 UTC [10] 3. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://lists.openstack.org/pipermail/openstack-discuss/2022-November/03124… [2] https://governance.openstack.org/election/ [3] https://etherpad.opendev.org/p/tc-2023.1-tracker [4] https://review.opendev.org/q/projects:openstack/governance+status:open [5] https://lists.openstack.org/pipermail/openstack-discuss/2022-December/03144… [6] https://review.opendev.org/c/openstack/governance/+/867062 [7] https://review.opendev.org/c/openstack/governance/+/866562 [8] https://review.opendev.org/c/openstack/governance/+/867588 [9] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [10] http://eavesdrop.openstack.org/#Technical_Committee_Meeting -gmann

1 0

[keystone] Holiday Meeting Schedule
by Dave Wilde 16 Dec '22

16 Dec '22

Hello, For the upcoming holidays I’m setting the Keystone meeting schedule as follows: 2022: - 20-Dec: Keystone weekly meeting on IRC - 23-Dec: Keystone Reviewathon on Google Meet - 27-Dec: Cancelled - Keystone weekly meeting - 30-Dec: Cancelled - Keystone Reviewathon on Google Meet 2023: - 03-Jan: Keystone weekly meeting on IRC as usual - 06-Jan: Keystone Reviewathon on Google Meet As always please let me know if you need anything. I hope you have a nice holiday and safe and happy new year! Thanks, /Dave

1 0

Could I deploy openstack in openstack instance
by 韩光宇 15 Dec '22

15 Dec '22

Hello, Could I deploy openstack in openstack instance? When I use a instance just with one network interface. After ip move to brq(using linuxbridge) or br-ex(uring openvswitch), the instance lost connection.  Does anyone know how to set it up? Or suggest me some documentation. I would appreciate any kind of guidance or help. Thank you.

2 1

Re: auto switch of glance-api containers to other controllers
by pradeep 15 Dec '22

15 Dec '22

Hello All Any help is highly appreciated. On Thu, 15 Dec 2022 at 01:41, pradeep <pradeep8985(a)gmail.com> wrote: > Hi Danny, > > Yes, i did, All of them are mounted in all the 3 controllers as well. > > # cat /etc/kolla/globals.yml | egrep > "glance_backend_file|glance_file_datadir_volume" > #glance_backend_file: "yes" > glance_file_datadir_volume: "/glanceimages" > > root@ctrl-1:~# df -h /glanceimages/ > Filesystem Size Used Avail Use% Mounted on > <IPaddress>:/Openstack_fileshare1 150G 888M 150G 1% /glanceimages > > root@ctrl-2:~# df -h /glanceimages/ > Filesystem Size Used Avail Use% Mounted on > <IPaddress>:/Openstack_fileshare1 150G 888M 150G 1% /glanceimages > > root@ctrl-3:~# df -h /glanceimages/ > Filesystem Size Used Avail Use% Mounted on > <IPaddress>:/Openstack_fileshare1 150G 888M 150G 1% /glanceimages > > > On Wed, 14 Dec 2022 at 17:54, Danny Webb <Danny.Webb(a)thehutgroup.com> > wrote: > >> did you change the glance_file_datadir_volume variable? that is >> required for kolla to deploy multiples: >> >> >> https://github.com/openstack/kolla-ansible/blob/ae3de342e48bc4293564a3b532a… >> >> ------------------------------ >> *From:* pradeep <pradeep8985(a)gmail.com> >> *Sent:* 14 December 2022 10:19 >> *To:* Danny Webb <Danny.Webb(a)thehutgroup.com> >> *Subject:* Re: auto switch of glance-api containers to other controllers >> >> >> * CAUTION: This email originates from outside THG * >> ------------------------------ >> Thanks Danny, All >> >> I have found the glance-api containers not deployed in control2 and >> control3 hosts although I have used NFS as my backend and share is enabled >> in all my controllers. >> >> I have tried to deploy the containers in control 2 and 3 as below but see >> some failures. I hope I am not using the wrong command. Am i missing >> something? Please let me know. >> >> *Command used* >> kolla-ansible -i multinode deploy --tags "glance" --limit >> control02.lab,control03.lab -vv >> >> *Errors in deployment* >> >> RUNNING HANDLER [glance : Restart glance-api container] >> ********************************************************************************* >> task path: >> /root/kolla-ansible/venv/share/kolla-ansible/ansible/roles/glance/handlers/main.yml:2 >> >> fatal: [control02.lab]: FAILED! => {"changed": true, "msg": "'Traceback >> (most recent call last):\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 268, >> in _raise_for_status\\n response.raise_for_status()\\n File >> \"/usr/lib/python3/dist-packages/requests/models.py\", line 940, in >> raise_for_status\\n raise HTTPError(http_error_msg, >> response=self)\\nrequests.exceptions.HTTPError: 500 Server Error: Internal >> Server Error for url: >> http+docker://localhost/v1.41/images/create?tag=yoga&fromImage=quay.io%2Fopenstack.kolla%2Fubuntu-source-glance-api\\n\\nDuring >> handling of the above exception, another exception occurred:\\n\\nTraceback >> (most recent call last):\\n File >> \"/tmp/ansible_kolla_docker_payload_iltg71bi/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", >> line 381, in main\\n File >> \"/tmp/ansible_kolla_docker_payload_iltg71bi/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 651, in recreate_or_restart_container\\n self.start_container()\\n >> File >> \"/tmp/ansible_kolla_docker_payload_iltg71bi/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 669, in start_container\\n self.pull_image()\\n File >> \"/tmp/ansible_kolla_docker_payload_iltg71bi/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 450, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) >> for line in self.dc.pull(\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/image.py\", line 430, >> in pull\\n self._raise_for_status(response)\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 270, >> in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n >> File \"/usr/local/lib/python3.8/dist-packages/docker/errors.py\", line 31, >> in create_api_error_from_http_exception\\n raise cls(e, response=response, >> explanation=explanation)\\ndocker.errors.APIError: 500 Server Error for >> http+docker://localhost/v1.41/images/create?tag=yoga&fromImage=quay.io%2Fopenstack.kolla%2Fubuntu-source-glance-api: >> Internal Server Error (\"Get \"https://quay.io/v2/\ >> <https://quay.io/v2//>": net/http: request canceled while waiting for >> connection (Client.Timeout exceeded while awaiting headers)\")\\n'"} >> fatal: [control03.lab]: FAILED! => {"changed": true, "msg": "'Traceback >> (most recent call last):\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 268, >> in _raise_for_status\\n response.raise_for_status()\\n File >> \"/usr/lib/python3/dist-packages/requests/models.py\", line 940, in >> raise_for_status\\n raise HTTPError(http_error_msg, >> response=self)\\nrequests.exceptions.HTTPError: 500 Server Error: Internal >> Server Error for url: >> http+docker://localhost/v1.41/images/create?tag=yoga&fromImage=quay.io%2Fopenstack.kolla%2Fubuntu-source-glance-api\\n\\nDuring >> handling of the above exception, another exception occurred:\\n\\nTraceback >> (most recent call last):\\n File >> \"/tmp/ansible_kolla_docker_payload_kkez0k31/ansible_kolla_docker_payload.zip/ansible/modules/kolla_docker.py\", >> line 381, in main\\n File >> \"/tmp/ansible_kolla_docker_payload_kkez0k31/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 651, in recreate_or_restart_container\\n self.start_container()\\n >> File >> \"/tmp/ansible_kolla_docker_payload_kkez0k31/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 669, in start_container\\n self.pull_image()\\n File >> \"/tmp/ansible_kolla_docker_payload_kkez0k31/ansible_kolla_docker_payload.zip/ansible/module_utils/kolla_docker_worker.py\", >> line 450, in pull_image\\n json.loads(line.strip().decode(\\'utf-8\\')) >> for line in self.dc.pull(\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/image.py\", line 430, >> in pull\\n self._raise_for_status(response)\\n File >> \"/usr/local/lib/python3.8/dist-packages/docker/api/client.py\", line 270, >> in _raise_for_status\\n raise create_api_error_from_http_exception(e)\\n >> File \"/usr/local/lib/python3.8/dist-packages/docker/errors.py\", line 31, >> in create_api_error_from_http_exception\\n raise cls(e, response=response, >> explanation=explanation)\\ndocker.errors.APIError: 500 Server Error for >> http+docker://localhost/v1.41/images/create?tag=yoga&fromImage=quay.io%2Fopenstack.kolla%2Fubuntu-source-glance-api: >> Internal Server Error (\"Get \"https://quay.io/v2/\ >> <https://quay.io/v2//>": net/http: request canceled while waiting for >> connection (Client.Timeout exceeded while awaiting headers)\")\\n'"} >> META: ran handlers >> >> Regards >> Pradeep >> >> >> On Mon, 12 Dec 2022 at 14:54, Danny Webb <Danny.Webb(a)thehutgroup.com> >> wrote: >> >> Kolla only uses a single controller if you use file based backend by >> default. If you want to continue using the file backend, you can use >> something like an NFS share to share images between controllers and then >> change the glance_file_datadir_volume parameter which will result in the >> api being deployed on all controllers >> ------------------------------ >> *From:* pradeep <pradeep8985(a)gmail.com> >> *Sent:* 12 December 2022 04:53 >> *To:* Dmitriy Rabotyagov <noonedeadpunk(a)gmail.com> >> *Cc:* openstack-discuss <openstack-discuss(a)lists.openstack.org> >> *Subject:* Re: auto switch of glance-api containers to other controllers >> >> >> * CAUTION: This email originates from outside THG * >> ------------------------------ >> Any help is highly appreciated. >> >> Thank you! >> >> On Fri, 9 Dec 2022 at 21:46, pradeep <pradeep8985(a)gmail.com> wrote: >> >> Hi Dmitriy, Thanks for the response. Sorry, I forgot to mention that i >> used kolla ansible for deployment. As per kolla docs, glance-api container >> runs only in one controller. So i am looking from kolla perspective if it >> can automatically switch it over other available controllers. >> >> Regards >> Pradeep >> >> On Fri, 9 Dec 2022 at 14:51, Dmitriy Rabotyagov <noonedeadpunk(a)gmail.com> >> wrote: >> >> Hi, >> >> I'm not sure if you used any deployment tool for your environment, but >> usually for such failover there's a load balancer in conjunction with VRRP >> or Anycast, that able to detect when controller1 is down and forward >> traffic to another controller. >> >> As example, OpenStack-Ansible as default option installs haproxy to each >> controller and keepalived to implement VRRP and Virtual IPs. So when glance >> is down on controller1, haproxy will detect that and forward traffic to >> other controllers. If controller1 is down as a whole, keepalived will >> detect that and raise VIP on controller2, so all client traffic will go >> there. Since controller2 also have haproxy, it will pass traffic to >> available backends based on the source IP. >> >> >> пт, 9 дек. 2022 г., 09:06 pradeep <pradeep8985(a)gmail.com>: >> >> Hello All, >> >> I understand that glance-api container run only one controller always. I >> have tried to check if the glance-api container to switch over to the next >> available controller by means of rebooting and stopping the containers, but >> nothing happened. Is there a way to make sure that glance-api container >> switches to other controllers if controller 1 is not available. >> >> Thanks for you help >> >> Regards >> Pradeep >> >> >> >> >> -- >> ----------------------- >> Regards >> Pradeep Kumar >> >> >> >> -- >> ----------------------- >> Regards >> Pradeep Kumar >> >> *Danny Webb* >> Principal OpenStack Engineer >> Danny.Webb(a)thehutgroup.com >> [image: THG Ingenuity Logo] >> www.thg.com >> <https://www.linkedin.com/company/thg-ingenuity/?originalSubdomain=uk> >> <https://twitter.com/thgingenuity?lang=en> >> >> Danny Webb >> Principal OpenStack Engineer >> The Hut Group <http://www.thehutgroup.com/> >> >> Tel: >> Email: Danny.Webb(a)thehutgroup.com >> >> >> For the purposes of this email, the "company" means The Hut Group >> Limited, a company registered in England and Wales (company number 6539496) >> whose registered office is at Fifth Floor, Voyager House, Chicago Avenue, >> Manchester Airport, M90 3DQ and/or any of its respective subsidiaries. >> >> *Confidentiality Notice* >> This e-mail is confidential and intended for the use of the named >> recipient only. If you are not the intended recipient please notify us by >> telephone immediately on +44(0)1606 811888 or return it to us by e-mail. >> Please then delete it from your system and note that any use, >> dissemination, forwarding, printing or copying is strictly prohibited. Any >> views or opinions are solely those of the author and do not necessarily >> represent those of the company. >> >> *Encryptions and Viruses* >> Please note that this e-mail and any attachments have not been encrypted. >> They may therefore be liable to be compromised. Please also note that it is >> your responsibility to scan this e-mail and any attachments for viruses. We >> do not, to the extent permitted by law, accept any liability (whether in >> contract, negligence or otherwise) for any virus infection and/or external >> compromise of security and/or confidentiality in relation to transmissions >> sent by e-mail. >> >> *Monitoring* >> Activity and use of the company's systems is monitored to secure its >> effective use and operation and for other lawful business purposes. >> Communications using these systems will also be monitored and may be >> recorded to secure effective use and operation and for other lawful >> business purposes. >> hgvyjuv >> >> >> >> -- >> ----------------------- >> Regards >> Pradeep Kumar >> >> *Danny Webb* >> Principal OpenStack Engineer >> Danny.Webb(a)thehutgroup.com >> [image: THG Ingenuity Logo] >> www.thg.com >> <https://www.linkedin.com/company/thg-ingenuity/?originalSubdomain=uk> >> <https://twitter.com/thgingenuity?lang=en> >> > > > -- > ----------------------- > Regards > Pradeep Kumar > -- ----------------------- Regards Pradeep Kumar

1 0

[qa] Cancelling next office hours
by Martin Kopec 15 Dec '22

15 Dec '22

Hi everyone, we're cancelling office hours for the next 3 weeks due to the holidays. The next office hour is gonna be on January 10th. Thank you. Enjoy your holidays! -- Martin Kopec Senior Software Quality Engineer Red Hat EMEA IM: kopecmartin

1 0

kolla-ansible deploy glance with file as a backend on multiple nodes
by A Monster 15 Dec '22

15 Dec '22

I have made a deployment of openstack on multiple centos stream 8 nodes, and used files as a backend for glance service, but according to the documentation, when using file as a glance backend, It can only be deployed on a single node, what can I do to deploy it on multiple servers at once. Thank you.

3 2

reconfiguring the external interface to multiple interfaces doesn't work
by pradeep 14 Dec '22

14 Dec '22

Hello All, I have deployed OpenStack Yoga using kolla ansible with a single external interface initially. Later we realized that we wanted to use multiple interfaces. I have followed the docs to modify the external interface in the globals.yml file as below. *Before:* *globals.yml* neutron_external_interface: "bond2" *After:* *globals.yml* neutron_external_interface: "bond2,enp6s0f1" neutron_bridge_name: "br-ex,br-ex2" *Command used:* kolla-ansible -i multinode reconfigure --tags "common,horizon,neutron" -vv br-ex2 doesn't show up in the neutron nodes. I also noticed that my neutron-openvswitch-agent is continuously restating in both the neutron nodes. neutron-openvswitch-agent.log says as below 2022-12-15 02:12:35.121 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-int has datapath-ID 0000be5c93241c4f 2022-12-15 02:12:36.040 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Mapping physical network physnet1 to bridge br-ex 2022-12-15 02:12:36.041 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Bridge br-ex datapath-id = 0x0000801844eaf971 2022-12-15 02:12:36.057 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-ex has datapath-ID 0000801844eaf971 2022-12-15 02:12:36.063 7 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Mapping physical network physnet2 to bridge br-ex2 2022-12-15 02:12:36.063 7 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [-] Bridge br-ex2 for physical network physnet2 does not exist. Agent terminated! Here is the config file. cat /etc/kolla/neutron-openvswitch-agent/openvswitch_agent.ini [agent] tunnel_types = vxlan l2_population = true arp_responder = true extensions = qos,sfc [securitygroup] firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver [ovs] bridge_mappings = physnet1:br-ex,physnet2:br-ex2 datapath_type = system ovsdb_connection = tcp:127.0.0.1:6640 ovsdb_timeout = 10 local_ip = <IP_Address> Also, i tried to change the bridge name from br-ex to br-ex1 but it still shows up br-ex (this can be perceived as my second issue). How can I resolve both issues? Kindly help.

1 0

stackalytics is stop update,
by 韩光宇 14 Dec '22

14 Dec '22

Hi, all I see stackalytics[1] is stop update. The page show "The data was last updated on 09 Nov 2022 12:44:22 UTC". Does anyone know why? And If you want to check the contribution statistics, is there still a way? Thank you for any help. Cheers, Han [1]: www.stackalytics.io

5 6

[manila][release] Proposing to EOL Stein
by Carlos Silva 14 Dec '22

14 Dec '22

Hello! Recently in a weekly meeting we chatted about EOLing stable/stein and all the attendants were found to be in favor of this [0]. As the procedures go [1], we need to make it formal through a post in this mailing list, and see if there are objections. In case of concerns or objections, please reach out through email or the #openstack-manila IRC channel. This will impact all branched manila repositories (manila, python-manilaclient and manila-ui). If there aren't objections or strong concerns, I will be proposing the patches to EOL stable/stein within one week. [0] https://meetings.opendev.org/meetings/manila/2022/manila.2022-11-03-15.00.l… [1] https://docs.openstack.org/project-team-guide/stable-branches.html#end-of-l… Thanks, carloss

1 1

[cinder] Festival of XS reviews?
by Rajat Dhasmana 14 Dec '22

14 Dec '22

Hello Argonauts, I've come to know informally that people might not be around during this time due to the holiday season. Based on this, I would like to ask the cinder team if we would like to have the festival of XS reviews this friday or not? Date: 16 December, 2022 Time: 1400-1600 UTC Please reply with your feedback and if we've enough people joining, we can continue with the session. Thanks Rajat Dhasmana

2 1

[cinder] Bug Report from 12-14-2022
by Sofia Enriquez 14 Dec '22

14 Dec '22

This is a bug report from 12-07-2022 to 12-14-2022. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1999252 "[RBD -> SolidFire] Cache mode was not updated after volume retype." Unassigned. Medium - https://bugs.launchpad.net/nova/+bug/1999125 "cinder retype volume raise exception with shutoff vm." Unassigned. Cheers, Sofia -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[neutron] Neutron drivers meeting Friday Dec 16
by Rodolfo Alonso Hernandez 14 Dec '22

14 Dec '22

Hello Neutrinos: Please check the Neutron drivers meeting agenda: https://wiki.openstack.org/wiki/Meetings/NeutronDrivers. So far we have two topics to discuss: * https://bugs.launchpad.net/neutron/+bug/1998609 * https://bugs.launchpad.net/neutron/+bug/1998608 See you next Friday.

1 0

[neutron] Neutron meetings cancelled last week of December
by Rodolfo Alonso Hernandez 14 Dec '22

14 Dec '22

Hello Neutrinos: The Neutron meetings (team, CI and drivers) will be cancelled the last week of December. Enjoy your holidays!

1 0

[nova][placement] Spec review day on Dec 14th
by Sylvain Bauza 14 Dec '22

14 Dec '22

Hey folks, As agreed on a previous meeting [1] we will have a second Spec review day on Wed Dec 14th. Now you're aware, prepare your specs in advance so they're reviewable and ideally please be around on that day in order to reply to any comments and eventually propose a new revision, ideally the same day. As a side note, we'll have an Implementation review day on Jan 10th which will focus on reviewing feature changes related to accepted specs and we also plan to review a few implementation patches by Dec 15th. Thanks, -Sylvain [1] https://meetings.opendev.org/meetings/nova/2022/nova.2022-11-29-16.00.log.h…

1 1

[all][tc] Technical Committee next weekly meeting on 2022 Dec 14 at 1600 UTC
by Ghanshyam Mann 14 Dec '22

14 Dec '22

Hello Everyone, The technical Committee's next weekly meeting is scheduled for 2022 Dec 14, at 1600 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Tuesday, Dec 13 at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[Keystone] member role with access rule can not list hypervisors details
by Hemant Sonawane 13 Dec '22

13 Dec '22

Hello, Does application credentials created as member role with access rule need any special permissions to list hypervisors via cli? Here is the access rule i created for my application credentials [ { "path": "/v2.1/**", "method": "GET", "service": "compute" } ] +--------------------------------+---------+--------+-------------------+ | ID | Service | Method | Path | +--------------------------------+---------+--------+-------------------+ | f2417b29f8394281aaa088d7c01059 | compute | POST | /v2.1/hypervisors | | 0cacd63becc04db086cd070db978c5 | compute | POST | /v2.1/servers | | 366abc32faf543b9ab047a83eafe83 | compute | GET | /v2.1/servers | | 4a4258b6a37c49f3816b708efa179f | compute | GET | /v2.1/** | +--------------------------------+---------+--------+-------------------+ I tried to use other access rule too. After sourcing the credentials if i would like to list any of the openstack resources it simply does not work *openstack volume list openstack The request you have made requires authentication. (HTTP 401)openstack hypervisor list The request you have made requires authentication. (HTTP 401) (Request-ID: req-c4532458-5606-4c85-8f93-20e61f57e2c9)* i think this could be a bug in keystone or it might need any special permissions with access rule. Please help will be really appreciated. -- Thanks and Regards, Hemant Sonawane

1 0

[cinder] volume_attachement entries are not getting deleted from DB
by Hemant Sonawane 13 Dec '22

13 Dec '22

Hello I am using wallaby release openstack and having issues with cinder volumes as once I try to delete, resize or unshelve the shelved vms the volume_attachement entries do not get deleted in cinder db and therefore the above mentioned operations fail every time. I have to delete these volume_attachement entries manually then it works. Is there any way to fix this issue ? nova-compute logs: cinderclient.exceptions.ClientException: Unable to update attachment.(Invalid volume: duplicate connectors detected on volume Help will be really appreciated Thanks ! -- Thanks and Regards, Hemant Sonawane

3 6

[barbican] Meeting cancelled today
by Grzegorz Grasza 13 Dec '22

13 Dec '22

Hi all, I won't be able to host the Barbican meeting today. As there are no outstanding topics to discuss, I'm cancelling it. / Greg

1 0

[all] many Python 3.11 failures in OpenStack components
by Thomas Goirand 12 Dec '22

12 Dec '22

Hi, The below bugs have been reported against the OpenStack components in Debian. I would greatly appreciate help fixing these before Bookworm is in freeze (in January). If you fix a bug, it'd be great to add a link in the Debian bug, so I can see the URL of the review (even if the patch isn't merged yet). As a reminder: to write to a Debian bug, simply write to <bugnumber>@bugs.debian.org (the Debian BTS is email driven). networking-mlnx: https://bugs.debian.org/1021924 FTBFS test failures: sqlalchemy.exc.InvalidRequestError: A transaction is already begun on this Session. networking-l2gw: https://bugs.debian.org/1023764 autopkgtest failure: module 'neutron.common.config' has no attribute 'register_common_config_options' ironic: https://bugs.debian.org/1024783 needs update for python3.11: Cannot spec a Mock object ironic-inspector: https://bugs.debian.org/1024952 needs update for python3.11: Failed to resolve the hostname (meow) for node uuid1 mistral-dashboard: https://bugs.debian.org/1024958 needs update for python3.11: module 'inspect' has no attribute 'getargspec'. Did you mean: 'getargs'? murano: https://bugs.debian.org/1025013 needs update for python3.11: 'NoneType' object has no attribute 'object_store' python-cinderclient: https://bugs.debian.org/1025028 needs update for python3.11: conflicting subparser: another-fake-action python-novaclient: https://bugs.debian.org/1025110 needs update for python3.11: conflicting subparser python-sushy: https://bugs.debian.org/1025124 needs update for python3.11: AssertionError: expected call not found python-taskflow: https://bugs.debian.org/1025126 needs update for python3.11: MismatchError Cheers, Thomas Goirand (zigo)

7 8

[neutron][devstack][qa] Dropping lib/neutron module
by Slawek Kaplonski 12 Dec '22

12 Dec '22

Hi, As You maybe know (or not as this was very long time ago) there are 2 modules to deploy Neutron in devstack: * old one called lib/neutron-legacy * new one called lib/neutron The problem is that new module lib/neutron was really never finished and used widely and still everyone is using (and should use) old one lib/neutron-legacy. We discussed that few times during PTGs and we finally decided to drop "new" module lib/neutron and have then rename old "lib/neutron-legacy" to be "lib/neutron" again. Decision was made because old module works fine and do its job, and as there is nobody who would like to finish really new module. Also having 2 modules from which "legacy" one is the only one which really works can be confusing and we want to avoid that confusion. So I proposed patches [1] and [2] to drop this unfinished module. I also proposed DNM patches for Neutron [3] and Tempest [4] to test that all jobs will work fine. But if You are maybe relaying on the Neutron modules from Devstack, please propose some test patch in Your project too and check if everything works fine for You. In patch [2] I didn't really removed "lib/neutron-legacy" yet because there are some projects which are sourcing that file in their devstack module. As [1] and [2] will be merged I will be proposing patches to change that for those projects but any help with that is welcome :) [1] https://review.opendev.org/c/openstack/devstack/+/865014 [2] https://review.opendev.org/c/openstack/devstack/+/865015 [3] https://review.opendev.org/c/openstack/neutron/+/865822 [4] https://review.opendev.org/c/openstack/tempest/+/865821 -- Slawek Kaplonski Principal Software Engineer Red Hat

2 1

[neutron][ironic] Distributed routers and SNAT
by Arnaud Morin 12 Dec '22

12 Dec '22

Hello team, When using router in DVR (+ HA), we end-up having the router on all computes where needed. So far, this is nice. We want to introduce Ironic baremetal servers, with a private network access. DVR won't apply on such baremetal servers, and we know floating IP are not going to work. Anyway, we were thinking that SNAT part would be OK. After doing few tests, we noticed that all distributed routers are answering to ARP and ICMP, thus creating duplicates in the network. $ arping -c1 192.168.43.1 ARPING 192.168.43.1 60 bytes from fa:16:3f:67:97:6a (192.168.43.1): index=0 time=634.700 usec 60 bytes from fa:16:3f:dc:67:91 (192.168.43.1): index=1 time=750.298 usec --- 192.168.43.1 statistics --- 1 packets transmitted, 2 packets received, 0% unanswered (1 extra) Is there anything possible on neutron side to prevent this? FYI, I did a comparison with routers in centralized mode (+ HA). In that situation, keepalived is putting the qr-xxx interface down in qrouter namespace. In distributed mode, keepalives is running in snat- namespace and cannot manage the router interface. Any help / tip would be appreciated. Thanks! Arnaud.

4 7

[neutron] Bug deputy report for week of December 5th
by Brian Haley 12 Dec '22

12 Dec '22

Hi, I was Neutron bug deputy last week. Below is a short summary about the reported bugs. -Brian Critical bugs ------------- * https://bugs.launchpad.net/neutron/+bug/1999249 - neutron-tempest-plugin jobs timing out on nested-virt nodes - Needs owner Medium bugs ----------- * https://bugs.launchpad.net/neutron/+bug/1998749 - [L3][DVR][vlan] east-west traffic flooding on physical bridges - Needs owner * https://bugs.launchpad.net/neutron/+bug/1998751 - [L3][DVR][vlan] HA VRRP traffic flooding on physical bridges on compute nodes - Needs owner * https://bugs.launchpad.net/neutron/+bug/1998952 - The static route removes the local route from qrouter-namespace - Adding/removing route same as attached interface subnet causes issue - Possible user error, the neutron API will let you "break" things - https://bugzilla.redhat.com/show_bug.cgi?id=1836870 has link to note - Needs discussion/owner * https://bugs.launchpad.net/neutron/+bug/1999154 - ovs/ovn source deployment broken with ovs_branch=master - https://review.opendev.org/c/openstack/neutron/+/867004 Low bugs -------- * https://bugs.launchpad.net/neutron/+bug/1998833 - [OVN] "MetadataAgentOvnSbIdl" should implement "post_connect" method - https://review.opendev.org/c/openstack/neutron/+/866601 Misc bugs --------- * https://bugs.launchpad.net/neutron/+bug/1999170 - Package names missing in neutron openvswitch setup guide - Added note that this information is in the install guide, closed * https://bugs.launchpad.net/neutron/+bug/1999209 - [ovn]dnat_adn_snat will not changed if updated fixed_ips of internal port - Asked for more information * https://bugs.launchpad.net/neutron/+bug/1999238 * https://bugs.launchpad.net/neutron/+bug/1999239 * https://bugs.launchpad.net/neutron/+bug/1999240 - Functional test failures - Related to https://review.opendev.org/c/openstack/neutron/+/867075

1 0

[openstack-ansible] Environment variable overflow on Ubuntu 20,04
by Taltavull Jean-François 12 Dec '22

12 Dec '22

Hello, The 'lxc_container_create' role appends the content of the 'global_environment_variables' OSA variable to '/etc/environment', which is used by the library 'libpam'. On Ubuntu 20.04 and 'libpam-runtime' library version 1.3.1, this works fine unless one of the environment variables content size exceeds 1024 char. In this case, the variable content is truncated. In our OpenStack deployment, this is the variable 'no_proxy' that overflows. This limit has been raised to 8192 with 'libpam-runtime' version 1.4 but there is no backport available in Ubuntu 20.04 repositories. So what would you think of the following workaround: instead of using '/etc/environment' to store the global variables, the role would create a shell script in '/etc/profile.d' thus avoiding the variable truncation issue related to the 'libpam' library ? This works on Ubuntu, Debian and CentOS hosts and containers. Cheers, JF

2 2

[Image Encryption] No meetings until next year
by Josephine Seifert 12 Dec '22

12 Dec '22

Hi, due to an injury and upcoming vacation, the next meeting for the image encrpyption team will be on Monday the 9th of Jannuary. greetings and happy holidays Luzi

2 1

Kolla-ansible [nova | Ensure RabbitMQ users exist] task error
by A Monster 12 Dec '22

12 Dec '22

I'm trying to deploy openstack xena on centos stream 8 on 2 controller nodes, 8 compute and a storage cluster, using kolla-ansible but I keep getting the following error, any workaround I could do to avoid this problem ? thank you. TASK [service-rabbitmq : nova | Ensure RabbitMQ users exist] ******************************************* skipping: [Controller-02] => (item=None) skipping: [Controller-02] FAILED - RETRYING: nova | Ensure RabbitMQ users exist (5 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (4 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (3 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (2 retries left). FAILED - RETRYING: nova | Ensure RabbitMQ users exist (1 retries left). failed: [Controller-01 -> Controller-01] (item=None) => {"attempts": 5, "censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} fatal: [Controller-01 -> {{ service_rabbitmq_delegate_host }}]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false}

2 1

auto switch of glance-api containers to other controllers
by pradeep 12 Dec '22

12 Dec '22

Hello All, I understand that glance-api container run only one controller always. I have tried to check if the glance-api container to switch over to the next available controller by means of rebooting and stopping the containers, but nothing happened. Is there a way to make sure that glance-api container switches to other controllers if controller 1 is not available. Thanks for you help Regards Pradeep

3 4

[ops] Migration from CentOS streams to Ubuntu and fast forward updates
by Massimo Sgaravatto 12 Dec '22

12 Dec '22

Dear all Dear all We are now running an OpenStack deployment: Yoga on CentOS8Stream. We are now thinking about a possible migration to Ubuntu for several reasons in particular: a- 5 years support for both the Operating System and OpenStack (considering LTS releases) b- Possibility do do a clean update between two Ubuntu LTS releases c- Easier procedure (also because of b) for fast forward updates (this is what we use to do) Considering the latter item, my understanding is that an update from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga could be done in the following way (we have two controller nodes and n compute nodes): - Update of first controller node from Ubuntu 20.04 Ussuri to Ubuntu 20.04 Victoria (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Victoria to Ubuntu 20.04 Wallaby (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Wallaby to Ubuntu 20.04 Xena (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Xena to Ubuntu 20.04 Yoga (update OpenStack packages + dbsync) - Update of first controller node from Ubuntu 20.04 Yoga to Ubuntu 22.04 Yoga (update Ubuntu packages) - Update of second controller node from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga (update OpenStack and Ubuntu packages) - Update of the compute nodes from Ubuntu 20.04 Ussuri to Ubuntu 22.04 Yoga (update OpenStack and Ubuntu packages) We would do the same when migrating from Ubuntu 22.04 Yoga to Ubuntu 24.04 and the OpenStack xyz release (where xyz is the LTS release used in Ubuntu 24.04) Is this supposed to work or am I missing something ? If we decide to migrate to Ubuntu, the first step would be the reinstallation with Ubuntu 22.04/Yoga of each node currently running CentOS8 stream/Yoga. I suppose there are no problems having in the same OpenStack installation nodes running the same Openstack version but different operating systems, or am I wrong ? Thanks, Massimo

4 8

[dev][infra][qa][tact-sig] Tox 4.0.0 breaking changes
by Jeremy Stanley 11 Dec '22

11 Dec '22

Tox 4.0.0, a major rewrite, just appeared on PyPI in the past couple of hours, so jobs are breaking right and left. We're going to try pinning to latest 3.x in zuul-jobs for the near term while we work through some of the more obvious issues with it, but until then lots of tox-based Zuul jobs are going to be broken. I'll follow up to this thread once we think things should be stabilized and we can start trying to pick through what's going to need fixing for the new tox version. -- Jeremy Stanley

4 12

openstack 504 Gateway
by f.loghmani 11 Dec '22

11 Dec '22

hello I use Kolla ansible multi-region. in one of my regions that is not keystone and horizon on it, sometimes I receive the error Gateway Timeout. how could I solve this problem? (in the same time I could open other region panels and it works correctly) thanks in advance. Gateway Timeout The gateway did not receive a timely response from the upstream server or application

1 0