October 2021 - openstack-discuss - lists.openstack.org

[nova][cyborg] No meeting today due virtual PTG
by Brin Zhang(张百林) 18 Oct '21

18 Oct '21

Hi all, As agreed Cyborg Team with today meeting [1], the meeting is *CANCELLED* as all of us will be attending the virtual PTG today. If you have any idea or feature/issue want to discuss, you can add it to etherpad [1], whether you can join or not, but you should describe its details, we can talk and give a reply. [1] https://etherpad.opendev.org/p/cyborg-yoga-ptg Thanks brinzhang

1 0

[cyborg][ptg] Yoga PTG meeting
by Brin Zhang(张百林) 18 Oct '21

18 Oct '21

Hello, As part of the Yoga PTG, the Cyborg team project will meet on Wednesday October 19 , from 6UTC-8UTC,( https://ethercalc.openstack.org/8tum5yl1bx43 report 503 now), but you can join us on #openstack-cyborg channel. We have created an Etherpad to define the agenda: https://etherpad.opendev.org/p/cyborg-yoga-ptg Feel free to add topics you would like to see discussed. Thanks Brin Zhang

1 0

[tacker] Skip weekly IRC meeting
by Yasufumi Ogawa 18 Oct '21

18 Oct '21

Hi team, Since we are going to have PTG sessions thorough this week, I'd like to skip IRC meeting on Oct 19. Thanks, Yasufumi

1 0

[xena][glance] Upgrade to Xena Shows Error
by Ammad Syed 18 Oct '21

18 Oct '21

Hi, I am trying to upgrade glance from wallaby to xena. The package upgrade goes successful. Vut When I am doing database upgrade. Its showing me below error. Can you guys please advise on it. su -s /bin/bash glance -c "glance-manage db_upgrade" 2021-10-18 12:23:59.852 20534 DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py:314 2021-10-18 12:23:59.868 20534 CRITICAL glance [-] Unhandled error: TypeError: argument of type 'NoneType' is not iterable 2021-10-18 12:23:59.868 20534 ERROR glance Traceback (most recent call last): 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/bin/glance-manage", line 10, in <module> 2021-10-18 12:23:59.868 20534 ERROR glance sys.exit(main()) 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/lib/python3/dist-packages/glance/cmd/manage.py", line 557, in main 2021-10-18 12:23:59.868 20534 ERROR glance return CONF.command.action_fn() 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/lib/python3/dist-packages/glance/cmd/manage.py", line 385, in upgrade 2021-10-18 12:23:59.868 20534 ERROR glance self.command_object.upgrade(CONF.command.version) 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/lib/python3/dist-packages/glance/cmd/manage.py", line 127, in upgrade 2021-10-18 12:23:59.868 20534 ERROR glance self._sync(version) 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/lib/python3/dist-packages/glance/cmd/manage.py", line 176, in _sync 2021-10-18 12:23:59.868 20534 ERROR glance alembic_command.upgrade(a_config, version) 2021-10-18 12:23:59.868 20534 ERROR glance File "/usr/lib/python3/dist-packages/alembic/command.py", line 277, in upgrade 2021-10-18 12:23:59.868 20534 ERROR glance if ":" in revision: 2021-10-18 12:23:59.868 20534 ERROR glance TypeError: argument of type 'NoneType' is not iterable 2021-10-18 12:23:59.868 20534 ERROR glance However the db_sync was successful and below is the DB version detail. su -s /bin/bash glance -c "glance-manage db_version" 2021-10-18 12:25:14.780 20683 DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py:314 2021-10-18 12:25:14.783 20683 INFO alembic.runtime.migration [-] Context impl MySQLImpl. 2021-10-18 12:25:14.784 20683 INFO alembic.runtime.migration [-] Will assume non-transactional DDL. wallaby_contract01 su -s /bin/bash glance -c "glance-manage db_sync" 2021-10-18 12:25:25.773 20712 DEBUG oslo_db.sqlalchemy.engines [-] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python3/dist-packages/oslo_db/sqlalchemy/engines.py:314 2021-10-18 12:25:25.776 20712 INFO alembic.runtime.migration [-] Context impl MySQLImpl. 2021-10-18 12:25:25.776 20712 INFO alembic.runtime.migration [-] Will assume non-transactional DDL. Database is up to date. No migrations needed. -- Regards, Syed Ammad Ali

2 1

[neutron] Bug Deputy Report October 11 - 18
by Lucas Alvares Gomes 18 Oct '21

18 Oct '21

High: * https://bugs.launchpad.net/neutron/+bug/1946588 - "[OVN]Metadata get warn logs after boot instance server about "MetadataServiceReadyWaitTimeoutException"" - Assigned to: hailun huang * https://bugs.launchpad.net/neutron/+bug/1946748 - " [stable/stein] neutron-tempest-plugin jobs fail with "AttributeError: module 'tempest.common.utils' has no attribute 'is_network_feature_enabled'"" - Assigned to: Bernard Cafarelli Medium: * https://bugs.launchpad.net/neutron/+bug/1946589 - "[OVN] localport might not be updated when create multiple subnets for its network" - Unassigned * https://bugs.launchpad.net/neutron/+bug/1946666 - "[ovn] neutron_ovn_db_sync_util crashes (ACL already exists)" - Assigned to: Daniel Speichert * https://bugs.launchpad.net/neutron/+bug/1946713 - "[ovn]Network's availability_zones is empty" - Assigned to: hailun huang * https://bugs.launchpad.net/neutron/+bug/1947334 - "[OVN] Migration to OVN does not create the OVN QoS DB registers" - Assigned to: Rodolfo Alonso * https://bugs.launchpad.net/neutron/+bug/1947366 - "[OVN] Migration to OVN removes "connectivity" parameter from VIF details" - Assigned to: Rodolfo Alonso * https://bugs.launchpad.net/neutron/+bug/1947378 - " [OVN] VIF details "connectivity" parameter is not correctly populated" - Assigned to: Rodolfo Alonso Needs further triage: * https://bugs.launchpad.net/neutron/+bug/1946624 - "OVSDB Error: Transaction causes multiple rows in "Port_Group" table to have identical values" - Marked as Incomplete * https://bugs.launchpad.net/neutron/+bug/1946764 - "[OVN]Any dhcp options which are string type should be escape" * https://bugs.launchpad.net/neutron/+bug/1946781 - " Appropriate way to allocate /64 ipv6 per instance" Cheers, Lucas

1 0

[glance] PTL on vacation - weekly meetings update
by Abhishek Kekane 18 Oct '21

18 Oct '21

Hi All, I'm starting my vacation from 25th October and will be back on November 15th. Please direct any issues to the rest of the core team. Also there will be no weekly meeting on 28th October and tentative cancellation of 4th November and 11th November unless there is something in the agenda by Tuesday 2nd November and 9th November EOB [1]. [1] https://etherpad.opendev.org/p/glance-team-meeting-agenda Thank you, Abhishek Kekane

1 0

Openstack magnum
by Vikarna Tathe 18 Oct '21

18 Oct '21

Hello All, I am trying to create a kubernetes cluster using magnum. Image: fedora-coreos. The stack gets stucked in CREATE_IN_PROGRESS. See the output below. openstack coe cluster list +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ | uuid | name | keypair | node_count | master_count | status | health_status | +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ | 9381cd12-af60-40b1-b1c4-d3dcc93e926e | k8s-cluster-01 | ctrlkey | 2 | 1 | CREATE_IN_PROGRESS | None | +--------------------------------------+----------------+---------+------------+--------------+--------------------+---------------+ openstack stack resource show k8s-cluster-01-2nyejxo3hyvb kube_masters +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attributes | {'refs_map': None, 'removed_rsrc_list': [], 'attributes': None, 'refs': None} | | creation_time | 2021-10-18T06:44:02Z | | description | | | links | [{'href': ' http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', 'rel': 'self'}, {'href': ' http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', 'rel': 'stack'}, {'href': ' http://10.14.20.159:8004/v1/f453bf8a2afa4fbba30030738f4dd216/stacks/k8s-clu…', 'rel': 'nested'}] | | logical_resource_id | kube_masters | | physical_resource_id | 3da2083f-0b2c-4b9d-8df5-8468e0de3028 | | required_by | ['kube_cluster_deploy', 'etcd_address_lb_switch', 'api_address_lb_switch', 'kube_cluster_config'] | | resource_name | kube_masters | | resource_status | CREATE_IN_PROGRESS | | resource_status_reason | state changed | | resource_type | OS::Heat::ResourceGroup | | updated_time | 2021-10-18T06:44:02Z | +------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Vikarna

2 1

[Openstack-victoria] [LIBVIRT] Live migration doesn't work, error in libvirt
by Swogat Pradhan 17 Oct '21

17 Oct '21

Hi, I am using openstack victoria and i am facing an issue when using the live-migration feature. After choosing the live-migration of an instance from Compute1 to compute2 i am getting an error in nova-compute.log (of compute 1) stating: *ERROR nova.virt.libvirt.driver [-] [instance: 59c95d46-2cbc-4787-89f7-8b36b826ffad] Live Migration failure: operation failed: Failed to connect to remote libvirt URI qemu+tcp://compute2/system: unable to connect to server at 'compute2:16509': Connection refused: libvirt.libvirtError: operation failed: Failed to connect to remote libvirt URI qemu+tcp://compute2/system: unable to connect to server at 'compute2:16509': Connection refused* Which states that the libvirtd.tcp socket is not running (libvirtd should run on 16509 port inorder for live-migration to succeed). Journalctl -xe output *Oct 18 06:32:52 compute2 systemd[1]: libvirtd-tcp.socket: Socket service libvirtd.service already active, refusing.Oct 18 06:32:52 compute2 systemd[1]: Failed to listen on Libvirt non-TLS IP socket.* I am trying to solve the above issue by adding --listen in the libvirtd_opts parameter in the service file and also in the /etc/default/libvirtd file. But after doing that the libvirtd service doesn't start. Can someone suggest a way forward for this? Thank you With regards, Swogat Pradhan

1 1

Re: The best linux distribution on which to deploy Openstack
by Radosław Piliszek 17 Oct '21

17 Oct '21

On Sun, 17 Oct 2021 at 10:03, A Monster <amonster369(a)gmail.com> wrote: > > What about Centos 7 , what are the openstack releases that it supports ? You have said that already. Train is the latest release on CentOS 7. -yoctozepto > On Sun, 17 Oct 2021 at 08:19, Radosław Piliszek <radoslaw.piliszek(a)gmail.com> wrote: >> >> On Sat, 16 Oct 2021 at 22:46, A Monster <amonster369(a)gmail.com> wrote: >> > >> > As a centos 7 user I have many experience using this distribution however centos 7 doesn't support the newest openstack releases ( after train ) and centos 8 will soon lose the support from Redhat since it's EOL is scheduled for 31/12/2021 and the centos stream distributions are upstreams for RHEL therefor is most likely unstable. >> > >> > So which distribution should I use ? >> >> Use the one that you are most familiar/comfortable with and that is >> supported by OpenStack deployment projects. >> >> For example, with Kolla Ansible, at the moment, you can choose from >> CentOS Stream 8, Debian Bullseye and Ubuntu 20.04 (sorted >> alphabetically; all have equal support). >> Soon, it will support Rocky Linux 8 as well (and then newer releases >> as they start coming). >> >> Kolla Ansible docs for Xena: https://docs.openstack.org/kolla-ansible/xena/ >> >> -yoctozepto

1 0

The best linux distribution on which to deploy Openstack
by A Monster 17 Oct '21

17 Oct '21

As a centos 7 user I have many experience using this distribution however centos 7 doesn't support the newest openstack releases ( after train ) and centos 8 will soon lose the support from Redhat since it's EOL is scheduled for 31/12/2021 and the centos stream distributions are upstreams for RHEL therefor is most likely unstable. So which distribution should I use ?

4 3

[dev][cinder] snapshot revert to any point
by Seena Fallah 16 Oct '21

16 Oct '21

Hi, There is a lack of feature to revert to any snapshot point in supported drivers like RBD. I've made a change to support this feature. Can someone please review them? https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/812032 https://review.opendev.org/c/openstack/cinder/+/806807 Thanks

1 0

How to use hosts with no storage disks
by A Monster 15 Oct '21

15 Oct '21

In Openstack, is it possible to create compute nodes with no hard drives and use PXE in order to boot the host's system and therefore launch instances with no local drive which is needed to boot the VM's image. If not, what's the minimum storage needed to be given to hosts in order to get a fully functional system.

6 8

[release] Release countdown for week R-24, Oct 11-15
by Előd Illés 15 Oct '21

15 Oct '21

Welcome back to the release countdown emails! These will be sent at major points in the Yogadevelopment cycle, which should conclude with a final release on March 30, 2022. Development Focus ----------------- At this stage in the release cycle, focus should be on planning the Yogadevelopment cycle, assessing Yogacommunity goals and approving Yogaspecs. General Information ------------------- Yoga is a 25 weeks long development cycle.In case you haven't seen it yet, please take a look over the schedule for this release: https://releases.openstack.org/ <https://releases.openstack.org/yoga/schedule.html>yoga <https://releases.openstack.org/yoga/schedule.html>/schedule.html <https://releases.openstack.org/yoga/schedule.html> By default, the team PTL is responsible for handling the release cycle and approving release requests. This task can (and probably should) be delegated to release liaisons. Now is a good time to review release liaison information for your team and make sure it is up to date: https://opendev.org/openstack/releases/src/branch/master/data/release_liais… <https://opendev.org/openstack/releases/src/branch/master/data/release_liais…> By default, all your team deliverables from the Yogarelease are continued in Yogawith a similar release model. Upcoming Deadlines & Dates -------------------------- Yoga PTG: October 18-22 Yoga-1 milestone:November 18, 2021 Előd Illés irc: elodilles

1 0

[ironic] Yoga PTG schedule
by Iury Gregory 15 Oct '21

15 Oct '21

Hello Ironicers! In our etherpad [1] we have 18 topics for this PTG and we have a total of 11 slots. This is the proposed schedule (we will discuss in our upstream meeting on Monday). *Monday (18 Oct) - Room Juno 15:00 - 17:00 UTC* * Support OpenBMC * Persistent memory Support * Redfish Host Connection Interface * Boot from Volume + UEFI *Tuesday (19 Oct) - Room Juno 14:00 - 17:00 UTC* * Posting to placement ourselves * The rise of compossible hardware, again * Self-configuring Ironic Service * Is there any way we can drive a co-operative use mode of ironic amongst some of the users? *Wednesday (18 Oct) - Room Juno 14:00 - 16:00 UTC* * Prioritize 3rd party CI in a box * Secure RBAC items in Yoga * Bulk operations *Thursday (18 Oct) - Room Kilo 14:00 - 16:00 UTC* * having to go look at logs is an antipattern * pxe-grub * Remove instance (non-BFV, non-ramdisk) networking booting * Direct SDN Integrations *Friday (22 Oct) - Room Kilo 14:00 - 16:00 UTC* * Eliminate manual commands * Certificate Management * Stopping use of wiki.openstack.org In case we don't have enough time we can book more slots if the community is ok and the slots are available. We will also have a section in the etherpad for last-minute topics =) [1] https://etherpad.opendev.org/p/ironic-yoga-ptg -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 1

[nova][dev] Reattaching mediated devices to instance coming back from suspended state
by Gustavo Faganello Santos 15 Oct '21

15 Oct '21

Hello, everyone! I'm working on a solution for Nova to reattach previously used mediated devices (vGPU instances, in my case) to VMs coming back from suspension, which seems to have been left on hold in the past [1] because of an old libvirt limitation, and I'm having a bit of a hard time doing so, since I'm not too familiar with the repo. I have tried creating a function that does the opposite of the mdev detach function, but the get_all_devices method seems to return an empty list when looking for mdevs at the moment of resuming the VM. Looking at the instance's XML file, I noticed that the mdev property remains while the VM is suspended, but it disappears AFTER the whole resume function is executed. I'm failing to understand why the mdev list returns empty, even though the mdev property exists in the instance's XML, and also why the mdev is removed from the XML after the resume function is executed. With that in mind, does anyone know if there's been any attempt to solve this issue since it was left on hold? If not, is there anything I should know while I attempt to do so? Thanks in advance. Gustavo [1] https://opendev.org/openstack/nova/src/branch/master/nova/virt/libvirt/driv…

2 2

OpenStack Xena for Ubuntu 21.10 and Ubuntu 20.04 LTS
by Corey Bryant 15 Oct '21

15 Oct '21

The Ubuntu OpenStack team at Canonical is pleased to announce the general availability of OpenStack Xena on Ubuntu 21.10 (Impish Indri) and Ubuntu 20.04 LTS (Focal Fossa) via the Ubuntu Cloud Archive. Details of the Xena release can be found at: https://www.openstack.org/software/xena To get access to the Ubuntu Xena packages: == Ubuntu 21.10 == OpenStack Xena is available by default for installation on Ubuntu 21.10. == Ubuntu 20.04 LTS == The Ubuntu Cloud Archive for OpenStack Xena can be enabled on Ubuntu 20.04 by running the following command: sudo add-apt-repository cloud-archive:xena The Ubuntu Cloud Archive for Xena includes updates for: aodh, barbican, ceilometer, ceph (16.2.6), cinder, designate, designate-dashboard, dpdk (20.11.3), glance, gnocchi, heat, heat-dashboard, horizon, ironic, ironic-ui, keystone, magnum, magnum-ui, manila, manila-ui, masakari, mistral, murano, murano-dashboard, networking-arista, networking-bagpipe, networking-baremetal, networking-bgpvpn, networking-hyperv, networking-l2gw, networking-mlnx, networking-odl, networking-sfc, neutron, neutron-dynamic-routing, neutron-vpnaas, nova, octavia, octavia-dashboard, openstack-trove, openvswitch (2.16.0), ovn (21.09.0), ovn-octavia-provider, placement, sahara, sahara-dashboard, senlin, swift, trove-dashboard, vmware-nsx, vitrage, watcher, watcher-dashboard, zaqar, and zaqar-ui. For a full list of packages and versions, please refer to: https://openstack-ci-reports.ubuntu.com/reports/cloud-archive/xena_versions… == Known issues == OVN 21.09.0 coming soon: https://bugs.launchpad.net/ubuntu/+source/ovn/+bug/1947003 == Reporting bugs == If you have any issues please report bugs using the ‘ubuntu-bug’ tool to ensure that bugs get logged in the right place in Launchpad: sudo ubuntu-bug nova-conductor Thank you to everyone who contributed to OpenStack Xena! Corey (on behalf of the Ubuntu OpenStack Engineering team)

1 0

[TripleO] Hackfest at the PTG on Thursday
by James Slagle 15 Oct '21

15 Oct '21

Hi TripleO, As previously mentioned, we're going to have a hackfest on Thursday next week during the PTG from 1300-1700UTC. The topic will be directord+task-core -- the proposed new task execution engine for TripleO. I've prepared an etherpad for the hackfest ahead of time: https://etherpad.opendev.org/p/tripleo-directord-hackfest There are details in the etherpad about how to set up 2 nodes for the hackfest. Virtual machines would work great for this, on any platform. I've run through it on a private OpenStack cloud, and also just using local libvirt. It would be good to get those setup ahead of the hackfest if you have time between now and Thursday. I'm looking forward to Thursday and some informal hacking! -- -- James Slagle --

1 0

[Diversity] [PTG] Diversity nd Inclusion Session at PTG
by Amy Marrich 15 Oct '21

15 Oct '21

Hi Everyone, The Diversity and Inclusion WG will be meeting during the PTG on Monday October 18th at 14:00 UTC in the Diablo room. We welcome all Open Infrastructure project to attend and we would be happy to assist you with any questions you might have in regards to the Inclusive Naming initiative that OIF began last year, We plan to do a review of the current CoC or going through the current code base for the projects to find instances where changes are needed to provide both examples and patches where appropriate to assist in these endeavours. The activities we focus on will be determined by attendance during the session, Thanks, Amy (spotz) on behalf of the Diversity and Inclusion WG

1 0

[cinder][PTG] yoga PTG schedule
by Brian Rosmaita 15 Oct '21

15 Oct '21

The Cinder project team will be meeting on Tuesday 19 October through Friday 22 October from 1300-1700 UTC. For the most part, the scheduling is flexible, and we'll discuss topics roughly in the order given on the etherpad: https://etherpad.opendev.org/p/yoga-ptg-cinder We'll try to keep the "Currently at the PTG" page updated, but you know how that goes. https://ptg.opendev.org/ptg.html For your scheduling convenience, here's an outline of the schedule in a spreadsheet: https://ethercalc.openstack.org/wfno9g46fa7p - topics in red: cross-project, so the times for those are accurate - topics in blue: participatory activities for the team - topics in green: happy hour We'll be meeting in BlueJeans (except for the happy hour, which will be in meetpad). The sessions (except for happy hour) will be recorded. Connection info is on the etherpad: https://etherpad.opendev.org/p/yoga-ptg-cinder All OpenStack community members are welcome (especially for the happy hour). Looking forward to seeing everyone next week! brian

1 0

[telemetry]Yoga vPTG
by Matthias Runge 15 Oct '21

15 Oct '21

Hello there, Next week, we’ll have PTG. There will be a telemetry session on Tuesday from 4pm to 5pm UTC. The planning ether pad used is at https://etherpad.opendev.org/p/telemetry-yoga-ptg Matthias

1 0

[wallaby][neutron][ovn] SSL connection to OVN-NB/SB OVSDB
by Faisal Sheikh 15 Oct '21

15 Oct '21

Hi, I am using Openstack Wallaby release with OVN on Ubuntu 20.04. My environment consists of 2 compute nodes and 1 controller node. ovs-vswitchd (Open vSwitch) 2.15.0 Ubuntu Kernel Version: 5.4.0-88-generic compute node1 172.16.30.1 compute node2 172.16.30.3 controller/Network node IP 172.16.30.46 I want to configure the ovn southbound and northbound database to listen on SSL connection. Set a certificate, private key, and CA certificate on both compute nodes and controller nodes in /etc/neutron/plugins/ml2/ml2_conf.ini and using string ssl:IP:Port to connect the southbound/northbound database but I am unable to establish connection on SSL. It's not connecting to ovsdb-server on 6641/6642. Error in the neutron logs is like below: 2021-10-12 17:15:27.728 50561 WARNING neutron.quota.resource_registry [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] security_group_rule is already registered 2021-10-12 17:15:27.754 50561 WARNING keystonemiddleware.auth_token [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2021-10-12 17:15:27.761 50561 INFO oslo_service.service [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] Starting 1 workers 2021-10-12 17:15:27.768 50561 INFO neutron.service [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] Neutron service started, listening on 0.0.0.0:9696 2021-10-12 17:15:27.776 50561 ERROR ovsdbapp.backend.ovs_idl.idlutils [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] Unable to open stream to ssl:172.16.30.46:6641 to retrieve schema: Unknown error -1 2021-10-12 17:15:27.779 50561 CRITICAL neutron [req-94a8acc2-e0ca-4431-8bed-f26864e7740a - - - - -] Unhandled error: neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-373793 failed with "Could not retrieve schema from ssl:172.16.30.46:6641" 2021-10-12 17:15:27.779 50561 ERROR neutron Traceback (most recent call last): 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/bin/neutron-server", line 10, in <module> 2021-10-12 17:15:27.779 50561 ERROR neutron sys.exit(main()) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/server/__init__.py", line 19, in main 2021-10-12 17:15:27.779 50561 ERROR neutron server.boot_server(wsgi_eventlet.eventlet_wsgi_server) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/__init__.py", line 68, in boot_server 2021-10-12 17:15:27.779 50561 ERROR neutron server_func() 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/wsgi_eventlet.py", line 24, in eventlet_wsgi_server 2021-10-12 17:15:27.779 50561 ERROR neutron neutron_api = service.serve_wsgi(service.NeutronApiService) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 94, in serve_wsgi 2021-10-12 17:15:27.779 50561 ERROR neutron registry.publish(resources.PROCESS, events.BEFORE_SPAWN, service) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/registry.py", line 60, in publish 2021-10-12 17:15:27.779 50561 ERROR neutron _get_callback_manager().publish(resource, event, trigger, payload=payload) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 149, in publish 2021-10-12 17:15:27.779 50561 ERROR neutron return self.notify(resource, event, trigger, payload=payload) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 110, in _wrapped 2021-10-12 17:15:27.779 50561 ERROR neutron raise db_exc.RetryRequest(e) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-10-12 17:15:27.779 50561 ERROR neutron self.force_reraise() 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-10-12 17:15:27.779 50561 ERROR neutron raise self.value 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 105, in _wrapped 2021-10-12 17:15:27.779 50561 ERROR neutron return function(*args, **kwargs) 2021-10-12 17:15:27.779 50561 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 174, in notify 2021-10-12 17:15:27.779 50561 ERROR neutron raise exceptions.CallbackFailure(errors=errors) 2021-10-12 17:15:27.779 50561 ERROR neutron neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-373793 failed with "Could not retrieve schema from ssl:172.16.30.46:6641" 2021-10-12 17:15:27.779 50561 ERROR neutron 2021-10-12 17:15:27.783 50572 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to ssl:172.16.30.46:6641 to retrieve schema: Unknown error -1 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager [-] Error during notification for neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-373774 process, after_init: Exception: Could not retrieve schema from ssl:172.16.30.46:6641 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs) 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 294, in post_fork_initialize 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager self._wait_for_pg_drop_event() 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 357, in _wait_for_pg_drop_event 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager ovn_conf.get_ovn_nb_connection(), self.nb_schema_helper, self, 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 136, in nb_schema_helper 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager return impl_idl_ovn.OvsdbNbOvnIdl.schema_helper 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 721, in __get__ 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager return self.func(owner) 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 102, in schema_helper 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager cls._schema_helper = idlutils.get_schema_helper(cls.connection_string, 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 215, in get_schema_helper 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager return create_schema_helper(fetch_schema_json(connection, schema_name)) 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 204, in fetch_schema_json 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager raise Exception("Could not retrieve schema from %s" % connection) 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager Exception: Could not retrieve schema from ssl:172.16.30.46:6641 2021-10-12 17:15:27.784 50572 ERROR neutron_lib.callbacks.manager 2021-10-12 17:15:27.787 50572 INFO neutron.wsgi [-] (50572) wsgi starting up on http://0.0.0.0:9696 2021-10-12 17:15:27.924 50572 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting 2021-10-12 17:15:27.925 50572 INFO neutron.wsgi [-] (50572) wsgi exited, is_accepting=True 2021-10-12 17:15:29.709 50573 INFO neutron.common.config [-] Logging enabled! 2021-10-12 17:15:29.710 50573 INFO neutron.common.config [-] /usr/bin/neutron-server version 18.0.0 2021-10-12 17:15:29.712 50573 INFO neutron.common.config [-] Logging enabled! 2021-10-12 17:15:29.713 50573 INFO neutron.common.config [-] /usr/bin/neutron-server version 18.0.0 2021-10-12 17:15:29.899 50573 INFO keyring.backend [-] Loading KWallet 2021-10-12 17:15:29.904 50573 INFO keyring.backend [-] Loading SecretService 2021-10-12 17:15:29.907 50573 INFO keyring.backend [-] Loading Windows 2021-10-12 17:15:29.907 50573 INFO keyring.backend [-] Loading chainer 2021-10-12 17:15:29.908 50573 INFO keyring.backend [-] Loading macOS 2021-10-12 17:15:29.927 50573 INFO neutron.manager [-] Loading core plugin: ml2 2021-10-12 17:15:30.355 50573 INFO neutron.plugins.ml2.managers [-] Configured type driver names: ['flat', 'geneve'] 2021-10-12 17:15:30.357 50573 INFO neutron.plugins.ml2.drivers.type_flat [-] Arbitrary flat physical_network names allowed 2021-10-12 17:15:30.358 50573 INFO neutron.plugins.ml2.managers [-] Loaded type driver names: ['flat', 'geneve'] 2021-10-12 17:15:30.358 50573 INFO neutron.plugins.ml2.managers [-] Registered types: dict_keys(['flat', 'geneve']) 2021-10-12 17:15:30.359 50573 INFO neutron.plugins.ml2.managers [-] Tenant network_types: ['geneve'] 2021-10-12 17:15:30.359 50573 INFO neutron.plugins.ml2.managers [-] Configured extension driver names: ['port_security', 'qos'] 2021-10-12 17:15:30.360 50573 INFO neutron.plugins.ml2.managers [-] Loaded extension driver names: ['port_security', 'qos'] 2021-10-12 17:15:30.360 50573 INFO neutron.plugins.ml2.managers [-] Registered extension drivers: ['port_security', 'qos'] 2021-10-12 17:15:30.360 50573 INFO neutron.plugins.ml2.managers [-] Configured mechanism driver names: ['ovn'] 2021-10-12 17:15:30.415 50573 INFO neutron.plugins.ml2.managers [-] Loaded mechanism driver names: ['ovn'] 2021-10-12 17:15:30.415 50573 INFO neutron.plugins.ml2.managers [-] Registered mechanism drivers: ['ovn'] 2021-10-12 17:15:30.415 50573 INFO neutron.plugins.ml2.managers [-] No mechanism drivers provide segment reachability information for agent scheduling. 2021-10-12 17:15:30.456 50573 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'flat' 2021-10-12 17:15:30.456 50573 INFO neutron.plugins.ml2.drivers.type_flat [-] ML2 FlatTypeDriver initialization complete 2021-10-12 17:15:30.456 50573 INFO neutron.plugins.ml2.managers [-] Initializing driver for type 'geneve' 2021-10-12 17:15:30.456 50573 INFO neutron.plugins.ml2.drivers.type_tunnel [-] geneve ID ranges: [(1, 65536)] 2021-10-12 17:15:32.555 50573 INFO neutron.plugins.ml2.managers [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Initializing extension driver 'port_security' 2021-10-12 17:15:32.555 50573 INFO neutron.plugins.ml2.extensions.port_security [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] PortSecurityExtensionDriver initialization complete 2021-10-12 17:15:32.556 50573 INFO neutron.plugins.ml2.managers [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Initializing extension driver 'qos' 2021-10-12 17:15:32.556 50573 INFO neutron.plugins.ml2.managers [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Initializing mechanism driver 'ovn' 2021-10-12 17:15:32.556 50573 INFO neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Starting OVNMechanismDriver 2021-10-12 17:15:32.562 50573 WARNING neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Firewall driver configuration is ignored 2021-10-12 17:15:32.586 50573 INFO neutron.services.logapi.drivers.ovn.driver [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] OVN logging driver registered 2021-10-12 17:15:32.588 50573 INFO neutron.plugins.ml2.plugin [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Modular L2 Plugin initialization complete 2021-10-12 17:15:32.589 50573 INFO neutron.plugins.ml2.managers [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Got port-security extension from driver 'port_security' 2021-10-12 17:15:32.589 50573 INFO neutron.extensions.vlantransparent [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Disabled vlantransparent extension. 2021-10-12 17:15:32.589 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: ovn-router 2021-10-12 17:15:32.597 50573 INFO neutron.services.ovn_l3.plugin [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Starting OVNL3RouterPlugin 2021-10-12 17:15:32.597 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: qos 2021-10-12 17:15:32.600 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: metering 2021-10-12 17:15:32.603 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: port_forwarding 2021-10-12 17:15:32.605 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading service plugin ovn-router, it is required by port_forwarding 2021-10-12 17:15:32.606 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: segments 2021-10-12 17:15:32.684 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: auto_allocate 2021-10-12 17:15:32.685 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: tag 2021-10-12 17:15:32.687 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: timestamp 2021-10-12 17:15:32.689 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: network_ip_availability 2021-10-12 17:15:32.691 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: flavors 2021-10-12 17:15:32.693 50573 INFO neutron.manager [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loading Plugin: revisions 2021-10-12 17:15:32.695 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Initializing extension manager. 2021-10-12 17:15:32.696 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension address-group not supported by any of loaded plugins 2021-10-12 17:15:32.697 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: address-scope 2021-10-12 17:15:32.697 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension router-admin-state-down-before-update not supported by any of loaded plugins 2021-10-12 17:15:32.698 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: agent 2021-10-12 17:15:32.699 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension agent-resources-synced not supported by any of loaded plugins 2021-10-12 17:15:32.700 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: allowed-address-pairs 2021-10-12 17:15:32.701 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: auto-allocated-topology 2021-10-12 17:15:32.701 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: availability_zone 2021-10-12 17:15:32.702 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension availability_zone_filter not supported by any of loaded plugins 2021-10-12 17:15:32.703 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension data-plane-status not supported by any of loaded plugins 2021-10-12 17:15:32.703 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: default-subnetpools 2021-10-12 17:15:32.704 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension dhcp_agent_scheduler not supported by any of loaded plugins 2021-10-12 17:15:32.705 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension dns-integration not supported by any of loaded plugins 2021-10-12 17:15:32.706 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension dns-domain-ports not supported by any of loaded plugins 2021-10-12 17:15:32.706 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension dvr not supported by any of loaded plugins 2021-10-12 17:15:32.707 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension empty-string-filtering not supported by any of loaded plugins 2021-10-12 17:15:32.708 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension expose-l3-conntrack-helper not supported by any of loaded plugins 2021-10-12 17:15:32.708 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: expose-port-forwarding-in-fip 2021-10-12 17:15:32.709 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: external-net 2021-10-12 17:15:32.710 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: extra_dhcp_opt 2021-10-12 17:15:32.710 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: extraroute 2021-10-12 17:15:32.711 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension extraroute-atomic not supported by any of loaded plugins 2021-10-12 17:15:32.712 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension filter-validation not supported by any of loaded plugins 2021-10-12 17:15:32.712 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: floating-ip-port-forwarding-description 2021-10-12 17:15:32.713 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: fip-port-details 2021-10-12 17:15:32.714 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: flavors 2021-10-12 17:15:32.715 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: floating-ip-port-forwarding 2021-10-12 17:15:32.715 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension floatingip-pools not supported by any of loaded plugins 2021-10-12 17:15:32.716 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: ip_allocation 2021-10-12 17:15:32.717 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension ip-substring-filtering not supported by any of loaded plugins 2021-10-12 17:15:32.717 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: l2_adjacency 2021-10-12 17:15:32.718 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: router 2021-10-12 17:15:32.719 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension l3-conntrack-helper not supported by any of loaded plugins 2021-10-12 17:15:32.720 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: ext-gw-mode 2021-10-12 17:15:32.721 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension l3-ha not supported by any of loaded plugins 2021-10-12 17:15:32.721 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension l3-flavors not supported by any of loaded plugins 2021-10-12 17:15:32.722 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension l3-port-ip-change-not-allowed not supported by any of loaded plugins 2021-10-12 17:15:32.723 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension l3_agent_scheduler not supported by any of loaded plugins 2021-10-12 17:15:32.724 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension logging not supported by any of loaded plugins 2021-10-12 17:15:32.725 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: metering 2021-10-12 17:15:32.725 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: metering_source_and_destination_fields 2021-10-12 17:15:32.726 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: multi-provider 2021-10-12 17:15:32.727 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: net-mtu 2021-10-12 17:15:32.727 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: net-mtu-writable 2021-10-12 17:15:32.728 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: network_availability_zone 2021-10-12 17:15:32.729 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: network-ip-availability 2021-10-12 17:15:32.729 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension network-segment-range not supported by any of loaded plugins 2021-10-12 17:15:32.730 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: pagination 2021-10-12 17:15:32.731 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: port-device-profile 2021-10-12 17:15:32.731 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension port-mac-address-regenerate not supported by any of loaded plugins 2021-10-12 17:15:32.732 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: port-numa-affinity-policy 2021-10-12 17:15:32.733 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: port-resource-request 2021-10-12 17:15:32.733 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: binding 2021-10-12 17:15:32.734 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension binding-extended not supported by any of loaded plugins 2021-10-12 17:15:32.735 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: port-security 2021-10-12 17:15:32.735 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: project-id 2021-10-12 17:15:32.736 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: provider 2021-10-12 17:15:32.736 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos 2021-10-12 17:15:32.737 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-bw-limit-direction 2021-10-12 17:15:32.738 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-bw-minimum-ingress 2021-10-12 17:15:32.738 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-default 2021-10-12 17:15:32.739 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-fip 2021-10-12 17:15:32.740 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension qos-gateway-ip not supported by any of loaded plugins 2021-10-12 17:15:32.740 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-port-network-policy 2021-10-12 17:15:32.741 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-rule-type-details 2021-10-12 17:15:32.741 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: qos-rules-alias 2021-10-12 17:15:32.742 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: quotas 2021-10-12 17:15:32.743 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: quota_details 2021-10-12 17:15:32.744 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: rbac-policies 2021-10-12 17:15:32.744 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension rbac-address-group not supported by any of loaded plugins 2021-10-12 17:15:32.745 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: rbac-address-scope 2021-10-12 17:15:32.746 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension rbac-security-groups not supported by any of loaded plugins 2021-10-12 17:15:32.746 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension rbac-subnetpool not supported by any of loaded plugins 2021-10-12 17:15:32.747 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: revision-if-match 2021-10-12 17:15:32.748 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: standard-attr-revisions 2021-10-12 17:15:32.748 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: router_availability_zone 2021-10-12 17:15:32.749 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension router-service-type not supported by any of loaded plugins 2021-10-12 17:15:32.749 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: security-groups-normalized-cidr 2021-10-12 17:15:32.750 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension port-security-groups-filtering not supported by any of loaded plugins 2021-10-12 17:15:32.751 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: security-groups-remote-address-group 2021-10-12 17:15:32.756 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: security-group 2021-10-12 17:15:32.757 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: segment 2021-10-12 17:15:32.758 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: segments-peer-subnet-host-routes 2021-10-12 17:15:32.758 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: service-type 2021-10-12 17:15:32.759 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: sorting 2021-10-12 17:15:32.759 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: standard-attr-segment 2021-10-12 17:15:32.760 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: standard-attr-description 2021-10-12 17:15:32.760 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension stateful-security-group not supported by any of loaded plugins 2021-10-12 17:15:32.761 50573 WARNING neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Did not find expected name "Stdattrs_common" in /usr/lib/python3/dist-packages/neutron/extensions/stdattrs_common.py 2021-10-12 17:15:32.762 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension subnet-dns-publish-fixed-ip not supported by any of loaded plugins 2021-10-12 17:15:32.762 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension subnet_onboard not supported by any of loaded plugins 2021-10-12 17:15:32.763 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: subnet-segmentid-writable 2021-10-12 17:15:32.763 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension subnet-service-types not supported by any of loaded plugins 2021-10-12 17:15:32.764 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: subnet_allocation 2021-10-12 17:15:32.765 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension subnetpool-prefix-ops not supported by any of loaded plugins 2021-10-12 17:15:32.765 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension tag-ports-during-bulk-creation not supported by any of loaded plugins 2021-10-12 17:15:32.766 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: standard-attr-tag 2021-10-12 17:15:32.767 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Loaded extension: standard-attr-timestamp 2021-10-12 17:15:32.767 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension trunk not supported by any of loaded plugins 2021-10-12 17:15:32.768 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension trunk-details not supported by any of loaded plugins 2021-10-12 17:15:32.769 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension uplink-status-propagation not supported by any of loaded plugins 2021-10-12 17:15:32.769 50573 INFO neutron.api.extensions [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Extension vlan-transparent not supported by any of loaded plugins 2021-10-12 17:15:32.771 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:network 2021-10-12 17:15:32.771 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:subnet 2021-10-12 17:15:32.772 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:subnetpool 2021-10-12 17:15:32.772 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:port 2021-10-12 17:15:32.774 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:router 2021-10-12 17:15:32.774 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:floatingip 2021-10-12 17:15:32.778 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of CountableResource for resource:rbac_policy 2021-10-12 17:15:32.778 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:security_group 2021-10-12 17:15:32.779 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:security_group_rule 2021-10-12 17:15:32.781 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:router 2021-10-12 17:15:32.781 50573 WARNING neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] router is already registered 2021-10-12 17:15:32.781 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:floatingip 2021-10-12 17:15:32.782 50573 WARNING neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] floatingip is already registered 2021-10-12 17:15:32.783 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of CountableResource for resource:rbac_policy 2021-10-12 17:15:32.783 50573 WARNING neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] rbac_policy is already registered 2021-10-12 17:15:32.783 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:security_group 2021-10-12 17:15:32.783 50573 WARNING neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] security_group is already registered 2021-10-12 17:15:32.784 50573 INFO neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Creating instance of TrackedResource for resource:security_group_rule 2021-10-12 17:15:32.784 50573 WARNING neutron.quota.resource_registry [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] security_group_rule is already registered 2021-10-12 17:15:32.810 50573 WARNING keystonemiddleware.auth_token [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. 2021-10-12 17:15:32.816 50573 INFO oslo_service.service [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Starting 1 workers 2021-10-12 17:15:32.824 50573 INFO neutron.service [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Neutron service started, listening on 0.0.0.0:9696 2021-10-12 17:15:32.831 50573 ERROR ovsdbapp.backend.ovs_idl.idlutils [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Unable to open stream to ssl:172.16.30.46:6641 to retrieve schema: Unknown error -1 2021-10-12 17:15:32.834 50573 CRITICAL neutron [req-06f63d07-b8d8-4c20-aa87-bdf06a3b17f5 - - - - -] Unhandled error: neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-904549 failed with "Could not retrieve schema from ssl:172.16.30.46:6641" 2021-10-12 17:15:32.834 50573 ERROR neutron Traceback (most recent call last): 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/bin/neutron-server", line 10, in <module> 2021-10-12 17:15:32.834 50573 ERROR neutron sys.exit(main()) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/cmd/eventlet/server/__init__.py", line 19, in main 2021-10-12 17:15:32.834 50573 ERROR neutron server.boot_server(wsgi_eventlet.eventlet_wsgi_server) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/__init__.py", line 68, in boot_server 2021-10-12 17:15:32.834 50573 ERROR neutron server_func() 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/server/wsgi_eventlet.py", line 24, in eventlet_wsgi_server 2021-10-12 17:15:32.834 50573 ERROR neutron neutron_api = service.serve_wsgi(service.NeutronApiService) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron/service.py", line 94, in serve_wsgi 2021-10-12 17:15:32.834 50573 ERROR neutron registry.publish(resources.PROCESS, events.BEFORE_SPAWN, service) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/registry.py", line 60, in publish 2021-10-12 17:15:32.834 50573 ERROR neutron _get_callback_manager().publish(resource, event, trigger, payload=payload) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 149, in publish 2021-10-12 17:15:32.834 50573 ERROR neutron return self.notify(resource, event, trigger, payload=payload) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 110, in _wrapped 2021-10-12 17:15:32.834 50573 ERROR neutron raise db_exc.RetryRequest(e) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ 2021-10-12 17:15:32.834 50573 ERROR neutron self.force_reraise() 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise 2021-10-12 17:15:32.834 50573 ERROR neutron raise self.value 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/db/utils.py", line 105, in _wrapped 2021-10-12 17:15:32.834 50573 ERROR neutron return function(*args, **kwargs) 2021-10-12 17:15:32.834 50573 ERROR neutron File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 174, in notify 2021-10-12 17:15:32.834 50573 ERROR neutron raise exceptions.CallbackFailure(errors=errors) 2021-10-12 17:15:32.834 50573 ERROR neutron neutron_lib.callbacks.exceptions.CallbackFailure: Callback neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.pre_fork_initialize-904549 failed with "Could not retrieve schema from ssl:172.16.30.46:6641" 2021-10-12 17:15:32.834 50573 ERROR neutron 2021-10-12 17:15:32.838 50582 ERROR ovsdbapp.backend.ovs_idl.idlutils [-] Unable to open stream to ssl:172.16.30.46:6641 to retrieve schema: Unknown error -1 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager [-] Error during notification for neutron.plugins.ml2.drivers.ovn.mech_driver.mech_driver.OVNMechanismDriver.post_fork_initialize-904522 process, after_init: Exception: Could not retrieve schema from ssl:172.16.30.46:6641 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager Traceback (most recent call last): 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron_lib/callbacks/manager.py", line 197, in _notify_loop 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager callback(resource, event, trigger, **kwargs) 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 294, in post_fork_initialize 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager self._wait_for_pg_drop_event() 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 357, in _wait_for_pg_drop_event 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager ovn_conf.get_ovn_nb_connection(), self.nb_schema_helper, self, 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/mech_driver.py", line 136, in nb_schema_helper 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager return impl_idl_ovn.OvsdbNbOvnIdl.schema_helper 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/common/utils.py", line 721, in __get__ 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager return self.func(owner) 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/impl_idl_ovn.py", line 102, in schema_helper 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager cls._schema_helper = idlutils.get_schema_helper(cls.connection_string, 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 215, in get_schema_helper 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager return create_schema_helper(fetch_schema_json(connection, schema_name)) 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager File "/usr/lib/python3/dist-packages/ovsdbapp/backend/ovs_idl/idlutils.py", line 204, in fetch_schema_json 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager raise Exception("Could not retrieve schema from %s" % connection) 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager Exception: Could not retrieve schema from ssl:172.16.30.46:6641 2021-10-12 17:15:32.840 50582 ERROR neutron_lib.callbacks.manager 2021-10-12 17:15:32.842 50582 INFO neutron.wsgi [-] (50582) wsgi starting up on http://0.0.0.0:9696 2021-10-12 17:15:32.961 50582 INFO oslo_service.service [-] Parent process has died unexpectedly, exiting 2021-10-12 17:15:32.963 50582 INFO neutron.wsgi [-] (50582) wsgi exited, is_accepting=True 2021-10-12 17:15:34.722 50583 INFO neutron.common.config [-] Logging enabled! I would really appreciate any input in this regard. Best regards, Faisal Sheikh

2 2

[kolla] Cancelling next weeks meeting (20 Oct 2021)
by Michał Nasiadka 15 Oct '21

15 Oct '21

Hello koalas, Since next week is PTG - I’m cancelling the meeting on 20th Oct 2021. Best regards, Michal

1 0

[tacker] Yoga vPTG planning
by yasufum 15 Oct '21

15 Oct '21

Hi everyone, The next vPTG will be held on 18-22 October as I shared at the previous IRC meeting [1]. Registration has already opened [2]. We've decided to have the next vPTG session on the same timeslots, 6-8 UTC, as previous for most of us join from India and APAC regions. I've prepared etherpad for the next vPTG [3]. If you have any suggestion, please add your topic on it. [1] http://lists.openstack.org/pipermail/openstack-discuss/2021-July/023540.html [2] https://openinfra-ptg.eventbrite.com/ [3] https://etherpad.opendev.org/p/tacker-yoga-ptg Thanks, Yasufumi

2 1

[Heat][PTG] PTG plan
by Rico Lin 15 Oct '21

15 Oct '21

Dear all Apologize for the late notice Next week Heat team will have PTG <https://ptg.opendev.org/ptg.html> schedule at: Monday 14 -15 UTC Feel free to join and suggest topic to https://etherpad.opendev.org/p/oct2021-ptg-heat *Rico Lin* OIF Individual Board of directors, OpenStack TC, Multi-arch SIG chair, Heat PTL, Senior Software Engineer@EasyStack

1 0

[Multi-arch SIG][PTG] PTG plan
by Rico Lin 15 Oct '21

15 Oct '21

Dear all Next week, Multi-arch SIG will have PTG <https://ptg.opendev.org/ptg.html> at: 10/19 Tuesday 07 - 08 UTC time 10/19 Tuesday 14 -15 UTC time SIG have been low activity for months and we need more volunteers to join. Please sign up for PTG if you're interested. Also feel free to suggest topics PTG etherpad: https://etherpad.opendev.org/p/oct2021-ptg-multi-arch *Rico Lin* OIF Individual Board of directors, OpenStack TC, Multi-arch SIG chair, Heat PTL, Senior Software Engineer@EasyStack

1 0

[neutron] Drivers meeting agenda - 15.10.2021
by Lajos Katona 15 Oct '21

15 Oct '21

Hi Neutron Drivers! As we have no quorum last week to decide on https://bugs.launchpad.net/neutron/+bug/1946251 tomorrow we can check it again and vote. The logs of the meeting from last week: https://meetings.opendev.org/meetings/neutron_drivers/2021/neutron_drivers.… (Sorry it is 4 days long as I missed to end the meeting.....) As we have PTG next week, let's cancel drivers meeting for that Friday, and I will be on PTO the week after (29. October) so I can't chair that one. See you online tomorrow. Lajos Katona (lajoskatona)

2 1

Problème with image from snapshot
by Franck VEDEL 14 Oct '21

14 Oct '21

Hello and first sorry for my english… thanks google. Something is wrong with what I want to do: I use Wallaby, it works very well (apart from VpnaaS, I wasted too much time this summer to make it work, without success, and the bug does not seem to be fixed). Here is what I want to do and which does not work as I want: - With an admin account, I launch a Win10 instance from the image I created. The instance is working but it takes about 10 minutes to get Win10 up and running. I wanted to take a snapshot of this instance and then create a new image from this snapshot. And that users use this new image. I create the snapshot, I place the "--public" parameter on the new image. I'm trying to create a new instance from this snapshot with the admin account: it works. I create a new user, who has his project, and sees all the images. I try to create an instance with this new image and I get the message: Block Device Mapping is Invalid: failed to get snapshot f12c04f2-51e7-4817-ab9b-eda63c5b9aff. (HTTP 400) (Request-ID: req-c26dab86-c25f-409a-8390-8aa0ea8fe1cb) Is it a legal problem? Is it possible to do as I do? otherwise how should we do it? Thanks if you have ideas for helping me Franck VEDEL

3 8

[tc] No TC weekly meeting next week due to meeting in PTG
by Ghanshyam Mann 14 Oct '21

14 Oct '21

Hello Everyone, As we will be meeting in PTG next week, we are cancelling the TC's next week (21st Oct) IRC meeting. -gmann

1 0

[dev][cinder] Consultation about new cinder-backup features
by Daniel de Oliveira Pereira 14 Oct '21

14 Oct '21

Hello everyone, We have prototyped some new features on Cinder for our clients, and we think that they are nice features and good candidates to be part of upstream Cinder, so we would like to get feedback from OpenStack community about these features and if you would be willing to accept them in upstream OpenStack. Our team implemented the following features for cinder-backup service: 1. A multi-backend backup driver, that allow OpenStack users to choose, via API/CLI/Horizon, which backup driver (Ceph or NFS, in our prototype) will be used during a backup operation to create a new volume backup. 2. An improved NFS backup driver, that allow OpenStack users to back up their volumes to private NFS servers, providing the NFS hostpath at runtime via API/CLI/Horizon, while creating the volume backup. Considering that cinder was configured to use the multi-backend backup driver, this is how it works: During a volume backup operation, the user provides a "location" parameter to indicate which backend will be used, and the backup hostpath, if applicable (for NFS driver), to create the volume backup. For instance: - Creating a backup using Ceph backend: $ openstack volume backup create --name <backup_name> --location ceph <volume_id> - Creating a backup using the improved NFS backend: $ openstack volume backup create --name <backup_name> --location nfs://my.nfs.server:/backups <volume_id> If the user chooses Ceph backend, the Ceph driver will be used to create the backup. If the user chooses the NFS backend, the improved NFS driver, previously mentioned, will be used to create the backup. The backup location, if provided, is stored on Cinder database, and can be seen fetching the backup details: $ openstack volume backup show <backup_name> Briefly, this is how the features were implemented: - Cinder API was updated to add an optional location parameter to "create backup" method. Horizon, and OpenStack and Cinder CLIs were updated accordingly, to handle the new parameter. - Cinder backup controller was updated to handle the backup location parameter, and a validator for the parameter was implemented using the oslo config library. - Cinder backup object model was updated to add a nullable location property, so that the backup location could be stored on cinder database. - a new backup driver base class, that extends BackupDriver and accepts a backup context object, was implemented to handle the backup configuration provided at runtime by the user. This new backup base class requires that the concrete drivers implement a method to validate the backup context (similar to BackupDriver.check_for_setup_error) - the 2 new backup drivers, previously mentioned, were implemented using these new backup base class. - in BackupManager class, the "service" attribute, that on upstream OpenStack holds the backup driver class name, was re-implemented as a factory function that accepts a backup context object and return an instance of a backup driver, according to the backup driver configured on cinder.conf file and the backup context provided at runtime by the user. - All the backup operations continue working as usual. Could you please let us know your thoughts about these features and if you would be open to adding them to upstream Cinder? If yes, we would be willing to submit the specs and work on the upstream implementation, if they are approved. Regards, Daniel Pereira

5 6

[manila][ptg] No IRC meetings on 21st and 28th Oct 2021
by Goutham Pacha Ravi 14 Oct '21

14 Oct '21

Hi Zorillas, Since we'll be at the PTG [1], we'll skip the IRC meeting [2] on the 21st of Oct; and since a number of us may be taking some time off the week after, we'll skip that weekly occurrence (28th Oct) as well. Please feel free to grab attention to any issue via this mailing list, or hop over to #openstack-manila on OFTC. Our next weekly IRC meeting will be on 4th Nov 2021. Thanks, and see you at the PTG! Goutham [1] https://etherpad.opendev.org/p/yoga-ptg-manila-planning [2] https://wiki.openstack.org/wiki/Manila/Meetings

1 0

[neutron][PTG] Schedule for yoga PTG
by Lajos Katona 14 Oct '21

14 Oct '21

Hi, I made a first schedule for the PTG next week, please check it: https://etherpad.opendev.org/p/neutron-yoga-ptg If you have anything to change or add please raise your voice :-) There's still some parts which can change, as we will have multiple cross-project sessions. See you next week. Lajos Katona (lajoskatona)

1 0

[neutron] Team meeting
by Lajos Katona 14 Oct '21

14 Oct '21

Hi Neutrinos, As we have PTG next week, let's cancel the team meeting. Cheers Lajos Katona (lajoskatona)

1 0

[neutron] CI meeting Tuesday 19.10
by Slawek Kaplonski 14 Oct '21

14 Oct '21

Hi, As we have PTG next week, let's cancel CI meeting. See You all at the PTG sessions and on the CI meeting on Tuesday 26th of October. -- Slawek Kaplonski Principal Software Engineer Red Hat

1 0

[cinder][backup] backup big volumes leads to oom kill of cinder-backup
by Stefan Hoffmann 14 Oct '21

14 Oct '21

Hi cinder team, we have the issue, that doing backups of big volumes (5TB) fails and cinder-backup get oom killed. Looks like cinder-backup is allocating memory but didn't release it correctly. Badly we are still using cinder queens. Is this a known issue and fixed in newer releases or should I create a bug report? We found a similar bug [1] with backup restore, that got fixed. I guess something like this is also needed for backup create. Thanks for you help Stefan [1] https://bugs.launchpad.net/cinder/+bug/1865011 -- Stefan Hoffmann DevOps-Engineer Cloud&Heat Technologies GmbH Königsbrücker Straße 96 (Halle 15) | 01099 Dresden +49 351 479 367 36 stefan.hoffmann(a)cloudandheat.com | www.cloudandheat.com Die grüne Cloud für KI und ML. Think Green: Mach Deine Anwendung grüner. https://thinkgreen.cloudandheat.com/ Commercial Register: District Court Dresden Register Number: HRB 30549 VAT ID No.: DE281093504 Managing Director: Nicolas Röhrs Authorized signatory: Dr. Marius Feldmann Authorized signatory: Kristina Rübenkamp

1 0

Networks in openstack
by Midhunlal Nb 14 Oct '21

14 Oct '21

Hi Team, I have some doubt in different network types Vlan,Vxlan and flat networks. How these networks helps in openstack.What is the use of each network? Could you Please provide me a detailed answer or suggest me any document regarding this networks. Show quoted text

3 3

[kayobe][kolla][ironic] kayobe overcloud provision fails because ironic compute hosts use their inspection DHCP pool IPs
by Manuel Holtgrewe 14 Oct '21

14 Oct '21

Dear list, I am experimenting with kayobe to deploy a test installation of OpenStack wallaby. You can find my configuration here: https://github.com/openstack/kayobe-config/compare/stable/wallaby...holtgre… I am following the kayobe documentation and have successfully setup a controller and a seed node. I am at the point where I have the nodes configured and they show up in bifrost baremetal node list. I can control them via IPMI/iDRAC/RedFish and boot them into the IPA image and the nodes can be inspected and actually go into the "manageable" status. kayobe is capable of using the inspection results and assigning the root device, so far, so good. I don't know whether my network configuration is good. I want to pin the IPs of stack-1 to stack-4 and the names resolve the correct IP addresses throughout my network. Below are some more details. In summary, I have trouble because `kayobe overcloud provision` makes my 4 overcloud bare metal host boot into IPA with DHCP enabled and they get the same IPs assigned that were given to them earlier in inspection. This means that the overcloud provision command cannot SSH into the nodes because it knows them by the wrong IPs. I must be really missing something here. What is it? Below are more details. Here is what kayobe pulled from the bifrost inspection (I believe). # cat etc/kayobe/inventory/overcloud [controllers] stack-1 ipmi_address=172.16.66.41 bmc_type=idrac stack-2 ipmi_address=172.16.66.42 bmc_type=idrac stack-3 ipmi_address=172.16.66.43 bmc_type=idrac stack-4 ipmi_address=172.16.66.44 bmc_type=idrac The IPs are also fixed here # etc/kayobe/network-allocation.yml compute_net_ips: stack-1: 172.16.32.11 stack-2: 172.16.32.12 stack-3: 172.16.32.13 stack-4: 172.16.32.14 stack-seed: 172.16.32.6 However, I thought I had to provide allocation ranges for DHCP for getting introspection to work. Thus, I have the following # etc/kayobe/networks.yml compute_net_cidr: 172.16.32.0/19 compute_net_gateway: 172.16.32.1 compute_net_vip_address: 172.16.32.2 compute_net_allocation_pool_start: 172.16.32.101 compute_net_allocation_pool_end: 172.16.32.200 compute_net_inspection_allocation_pool_start: 172.16.32.201 compute_net_inspection_allocation_pool_end: 172.16.32.250 This leads to the following dnsmasq leases in the bifrost host. # cat /var/lib/dnsmasq/dnsmasq.leases 1634187260 REDACTED 172.16.32.215 * REDACTED 1634187271 REDACTED 172.16.32.243 * REDACTED 1634187257 REDACTED 172.16.32.207 * REDACTED 1634187258 REDACTED 172.16.32.218 * REDACTED What am I missing? Best wishes, Manuel

2 1

[tripleo] Unable to deploy Overcloud Nodes
by Anirudh Gupta 14 Oct '21

14 Oct '21

Hi Team, As per the link below, While executing the command to deploy the overcloud nodes https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… I am executing the command - openstack overcloud deploy --templates On running this command, I am getting the following error *ERROR: (pymysql.err.OperationalError) (1045, "Access denied for user 'heat'@'10.255.255.4' (using password: YES)")(Background on this error at: http://sqlalche.me/e/e3q8 <http://sqlalche.me/e/e3q8>)* 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud [-] Exception occured while running the command: subprocess.CalledProcessError: Command '['sudo', 'podman', 'run', '--rm', '--user', 'heat', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat.conf:/etc/heat/heat.conf:z', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher:/home/stack/overcloud-deploy/overcloud/heat-launcher:z', 'localhost/tripleo/openstack-heat-api:ephemeral', 'heat-manage', 'db_sync']' returned non-zero exit status 1. 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud Traceback (most recent call last): 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/command.py", line 34, in run 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud super(Command, self).run(parsed_args) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/osc_lib/command/command.py", line 39, in run 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return super(Command, self).run(parsed_args) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/cliff/command.py", line 185, in run 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud return_code = self.take_action(parsed_args) or 0 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 1277, in take_action 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud self.setup_ephemeral_heat(parsed_args) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/v1/overcloud_deploy.py", line 767, in setup_ephemeral_heat 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud utils.launch_heat(self.heat_launcher, restore_db=restore_db) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/utils.py", line 2706, in launch_heat 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud launcher.heat_db_sync(restore_db) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib/python3.6/site-packages/tripleoclient/heat_launcher.py", line 530, in heat_db_sync 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud subprocess.check_call(cmd) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud raise CalledProcessError(retcode, cmd) 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud subprocess.CalledProcessError: Command '['sudo', 'podman', 'run', '--rm', '--user', 'heat', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat.conf:/etc/heat/heat.conf:z', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher:/home/stack/overcloud-deploy/overcloud/heat-launcher:z', 'localhost/tripleo/openstack-heat-api:ephemeral', 'heat-manage', 'db_sync']' returned non-zero exit status 1. 2021-10-13 05:46:28.390 183680 ERROR tripleoclient.v1.overcloud_deploy.DeployOvercloud 2021-10-13 05:46:28.400 183680 ERROR openstack [-] Command '['sudo', 'podman', 'run', '--rm', '--user', 'heat', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat.conf:/etc/heat/heat.conf:z', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher:/home/stack/overcloud-deploy/overcloud/heat-launcher:z', 'localhost/tripleo/openstack-heat-api:ephemeral', 'heat-manage', 'db_sync']' returned non-zero exit status 1.: subprocess.CalledProcessError: Command '['sudo', 'podman', 'run', '--rm', '--user', 'heat', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher/heat.conf:/etc/heat/heat.conf:z', '--volume', '/home/stack/overcloud-deploy/overcloud/heat-launcher:/home/stack/overcloud-deploy/overcloud/heat-launcher:z', 'localhost/tripleo/openstack-heat-api:ephemeral', 'heat-manage', 'db_sync']' returned non-zero exit status 1. 2021-10-13 05:46:28.401 183680 INFO osc_lib.shell [-] END return value: 1 Can someone please help in resolving this issue. Are there any parameters, templates that need to be passed in order to make it work. Regards Anirudh Gupta

2 1

[tc] [ptl] 2021 User Survey Project Specific Feedback Responses
by helena＠openstack.org 13 Oct '21

13 Oct '21

4 3

[all][tc] Technical Committee next weekly meeting on Oct 14th at 1500 UTC
by Ghanshyam Mann 13 Oct '21

13 Oct '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Oct 14th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Oct 13th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[Designate] After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail
by Braden, Albert 13 Oct '21

13 Oct '21

After enabling redis, and allowing TCP 6379 and 26379, I see it in /etc/designate/designate.conf in the designate_producer container: backend_url = redis://admin:<key>@10.221.176.48:26379?sentinel=kolla&sentinel_fallback=10.221.176.173:26379&sentinel_fallback=10.221.177.38:26379&db=0&socket_timeout=60&retry_on_timeout=yes And I can get to ports 6379 and 26379 with nc: (designate-producer)[root@dva3-ctrl3 /]# nc 10.221.176.173 26379 / -ERR unknown command `/`, with args beginning with: But I still see the DB error when TF rebuilds a VM: 2021-10-13 15:35:23.941 26 ERROR oslo_messaging.notify.dispatcher designate.exceptions.DuplicateRecord: Duplicate Record What am I missing? -----Original Message----- From: Michael Johnson <johnsomor(a)gmail.com> Sent: Tuesday, October 12, 2021 11:33 AM To: Braden, Albert <abraden(a)verisign.com> Cc: openstack-discuss(a)lists.openstack.org Subject: [EXTERNAL] Re: Re: Re: After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail Caution: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I don't have a good answer for you on that as it pre-dates my history with Designate a bit. I suspect it has to do with the removal of the pool-manager and the restructuring of the controller code. Maybe someone else on the discuss list has more insight. Michael On Tue, Oct 12, 2021 at 5:47 AM Braden, Albert <abraden(a)verisign.com> wrote: > > Thank you Michael, this is very helpful. Do you have any insight into why we don't experience this in Queens clusters? We aren't running a lock manager there either, and I haven't been able to duplicate the problem there. > > -----Original Message----- > From: Michael Johnson <johnsomor(a)gmail.com> > Sent: Monday, October 11, 2021 4:24 PM > To: Braden, Albert <abraden(a)verisign.com> > Cc: openstack-discuss(a)lists.openstack.org > Subject: [EXTERNAL] Re: Re: After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail > > Caution: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. > > You will need one of the Tooz supported distributed lock managers: > Consul, Memcacded, Redis, or zookeeper. > > Michael > > On Mon, Oct 11, 2021 at 11:57 AM Braden, Albert <abraden(a)verisign.com> wrote: > > > > After investigating further, I realized that we're not running redis, and I think that means that redis_connection_string doesn't get set. Does this mean that we must run redis, or is there a workaround? > > > > -----Original Message----- > > From: Braden, Albert > > Sent: Monday, October 11, 2021 2:48 PM > > To: 'johnsomor(a)gmail.com' <johnsomor(a)gmail.com> > > Cc: 'openstack-discuss(a)lists.openstack.org' <openstack-discuss(a)lists.openstack.org> > > Subject: RE: [EXTERNAL] Re: After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail > > > > I think so. I see this: > > > > ansible/roles/designate/templates/designate.conf.j2:backend_url = {{ redis_connection_string }} > > > > ansible/group_vars/all.yml:redis_connection_string: "redis://{% for host in groups['redis'] %}{% if host == groups['redis'][0] %}admin:{{ redis_master_password }}@{{ 'api' | kolla_address(host) | put_address_in_context('url') }}:{{ redis_sentinel_port }}?sentinel=kolla{% else %}&sentinel_fallback={{ 'api' | kolla_address(host) | put_address_in_context('url') }}:{{ redis_sentinel_port }}{% endif %}{% endfor %}&db=0&socket_timeout=60&retry_on_timeout=yes" > > > > Did anything with the distributed lock manager between Queens and Train? > > > > -----Original Message----- > > From: Michael Johnson <johnsomor(a)gmail.com> > > Sent: Monday, October 11, 2021 1:15 PM > > To: Braden, Albert <abraden(a)verisign.com> > > Cc: openstack-discuss(a)lists.openstack.org > > Subject: [EXTERNAL] Re: After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail > > > > Caution: This email originated from outside the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. > > > > Hi Albert, > > > > Have you configured your distributed lock manager for Designate? > > > > [coordination] > > backend_url = <DLM URL> > > > > Michael > > > > On Fri, Oct 8, 2021 at 7:38 PM Braden, Albert <abraden(a)verisign.com> wrote: > > > > > > Hello everyone. It’s great to be back working on OpenStack again. I’m at Verisign now. I can hardly describe how happy I am to have an employer that does not attach nonsense to the bottom of my emails! > > > > > > > > > > > > We are rebuilding our clusters from Queens to Train. On the new Train clusters, customers are complaining that deleting a VM and then immediately creating a new one with the same name (via Terraform for example) intermittently results in a missing DNS record. We can duplicate the issue by building a VM with terraform, tainting it, and applying. > > > > > > > > > > > > Before applying the change, we see the DNS record in the recordset: > > > > > > > > > > > > $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra > > > > > > | f9aa73c1-84ba-4854-be71-cbb616de672c | 8d1c84082a044a53abe0d519ed9e8c60 | openstack-terra-test-host.dev-ostck.dva3.vrsn.com. | A | 10.220.4.89 | ACTIVE | NONE | > > > > > > $ > > > > > > > > > > > > and we can pull it from the DNS server on the controllers: > > > > > > > > > > > > $ for i in {1..3}; do dig @dva3-ctrl${i}.cloud.vrsn.com -t axfr dva3.vrsn.com. |grep openstack-terra; done > > > > > > openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 > > > > > > openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 > > > > > > openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 > > > > > > > > > > > > After applying the change, we don’t see it: > > > > > > > > > > > > $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra > > > > > > | f9aa73c1-84ba-4854-be71-cbb616de672c | 8d1c84082a044a53abe0d519ed9e8c60 | openstack-terra-test-host.dev-ostck.dva3.vrsn.com. | A | 10.220.4.89 | ACTIVE | NONE | > > > > > > $ > > > > > > $ for i in {1..3}; do dig @dva3-ctrl${i}.cloud.vrsn.com -t axfr dva3.vrsn.com. |grep openstack-terra; done > > > > > > $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra > > > > > > $ > > > > > > > > > > > > We see this in the logs: > > > > > > > > > > > > 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'c70e693b4c47402db088c43a5a177134-openstack-terra-test-host.de...' for key 'unique_recordset'") > > > > > > 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher [SQL: INSERT INTO recordsets (id, version, created_at, zone_shard, tenant_id, zone_id, name, type, ttl, reverse_name) VALUES (%(id)s, %(version)s, %(created_at)s, %(zone_shard)s, %(tenant_id)s, %(zone_id)s, %(name)s, %(type)s, %(ttl)s, %(reverse_name)s)] > > > > > > 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher [parameters: {'id': 'dbbb904c347241a791aa01ca33a87b23', 'version': 1, 'created_at': datetime.datetime(2021, 10, 9, 1, 53, 44, 182652), 'zone_shard': 3184, 'tenant_id': '8d1c84082a044a53abe0d519ed9e8c60', 'zone_id': 'c70e693b4c47402db088c43a5a177134', 'name': 'openstack-terra-test-host.dev-ostck.dva3.vrsn.com.', 'type': 'A', 'ttl': None, 'reverse_name': '.moc.nsrv.3avd.kctso-ved.tsoh-tset-arret-kcatsnepo'}] > > > > > > > > > > > > It appears that Designate is trying to create the new record before the deletion of the old one finishes. > > > > > > > > > > > > Is anyone else seeing this on Train? The same set of actions doesn’t cause this error in Queens. Do we need to change something in our Designate config, to make it wait until the old records are finished deleting before attempting to create the new ones?

1 0

Open stack Ansible 23.1.0.dev35 VM launching errors
by Midhunlal Nb 13 Oct '21

13 Oct '21

Hi team, -->Successfully I installed Openstack ansible 23.1.0.dev35. --->I logged in to horizon and created a new network and launched a vm but I am getting an error. Error: Failed to perform requested operation on instance "hope", the instance has an error status: Please try again later [Error: Build of instance b6f42229-06d6-4365-8a20-07df869f1610 aborted: Failed to allocate the network(s), not rescheduling.]. -->Then I checked log | fault | {'code': 500, 'created': '2021-10-08T12:26:44Z', 'message': 'Build of instance b6f42229-06d6-4365-8a20-07df869f1610 aborted: Failed to allocate the network(s), not rescheduling.', 'details': 'Traceback (most recent call last):\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7235, in _create_guest_with_network\n post_xml_callback=post_xml_callback)\n File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__\n next(self.gen)\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/compute/manager.py", line 479, in wait_for_instance_event\n actual_event = event.wait()\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/eventlet/event.py", line 125, in wait\n result = hub.switch()\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 313, in switch\n return self.greenlet.switch()\neventlet.timeout.Timeout: 300 seconds\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/compute/manager.py", line 2397, in _build_and_run_instance\n accel_info=accel_info)\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 4200, in spawn\n cleanup_instance_disks=created_disks)\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/virt/libvirt/driver.py", line 7258, in _create_guest_with_network\n raise exception.VirtualInterfaceCreateException()\nnova.exception.VirtualInterfaceCreateException: Virtual Interface creation failed\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/compute/manager.py", line 2219, in _do_build_and_run_instance\n filter_properties, request_spec, accel_uuids)\n File "/openstack/venvs/nova-23.1.0.dev42/lib/python3.6/site-packages/nova/compute/manager.py", line 2458, in _build_and_run_instance\n reason=msg)\nnova.exception.BuildAbortException: Build of instance b6f42229-06d6-4365-8a20-07df869f1610 aborted: Failed to allocate the network(s), not rescheduling.\n'} | Please help me with this error. Thanks & Regards Midhunlal N B

3 9

Learn how users are using OpenStack at OpenInfra Live Keynotes
by Kendall Nelson 13 Oct '21

13 Oct '21

Hello Everyone, You might have heard that the OpenInfra Foundation is hosting its largest free virtual event of this year—OpenInfra Live: Keynote OpenInfra Live: Keynotes: https://openinfra.dev/live/keynotes Date and time: November 17-18 (1500-1700 UTC on each day) Register for free: https://www.eventbrite.com/e/openinfra-live-keynotes-tickets-169507530587 This two day special episode is your best opportunity to meet the newest players to the OpenInfra space and hear about how open source projects, such as OpenStack and Kubernetes, are supporting OpenInfra use cases like hybrid cloud. You will also have the chance to deep dive into the OpenStack user survey results since the launch of the OpenStack User Survey in 2013. Here is a preview of the OpenStack user survey results findings: - Over 300 OpenStack deployments were logged this year, including a significant number of new clouds—in the last 18 months, over 100 new OpenStack clouds have been built, growing the total number of cores under OpenStack management to more than 25,000,000. - Hybrid cloud scenarios continue to be popular, but over half of User Survey respondents indicated that the majority of their cloud infrastructure runs on OpenStack. Upgrades continue to be a challenge that the upstream community tackles with each additional release, but the User Survey shows the majority of organizations are running within the last seven releases. The full report of the OpenStack user survey will be distributed during the OpenInfra Live: Keynotes, so make sure you are registered for the event [2]. Can’t make it to the event? Register anyway, and we will email you a link to the recording after the event! At OpenInfra Live Keynotes, you will also have the opportunity to - interact with leaders of open source projects like OpenStack and Kubernetes to hear how the projects are supporting OpenInfra use cases like hybrid cloud - gain insight into public cloud economics and the role open source technologies play - celebrate as we announce this year’s Superuser Awards winner. This will be the one time everyone will be coming together this year. Come interact with the global OpenInfra community—Live! -Kendall Nelson (diablo_rojo) [1]: https://openinfra.dev/live/keynotes [2]: https://www.eventbrite.com/e/openinfra-live-keynotes-tickets-169507530587

1 0

[cinder] festival of XS reviews 15 October 2021
by Brian Rosmaita 13 Oct '21

13 Oct '21

Hello Cinder community members, This is a reminder that the most recent edition of the Cinder Festival of XS Reviews will be held at the end of this week on Friday 15 October. who: Everyone! what: The Cinder Festival of XS Reviews when: Friday 15 October 2021 from 1400-1600 UTC where: https://meetpad.opendev.org/cinder-festival-of-reviews This recurring meeting can be placed on your calendar by using this handy ICS file: http://eavesdrop.openstack.org/calendars/cinder-festival-of-reviews.ics See you there! brian

1 0

[nova][placement] No meeting next week
by Sylvain Bauza 13 Oct '21

13 Oct '21

As agreed during yesterday's meeting [1], Tuesday Oct 19th's meeting is *CANCELLED* as all of us will be attending the virtual PTG. I'm more than happy tho to see all of you next week thru video ! I'll add the PTG connection details in https://etherpad.opendev.org/p/nova-yoga-ptg -Sylvain [1] https://meetings.opendev.org/meetings/nova/2021/nova.2021-10-12-16.00.log.h…

1 0

[tripleo] Unable to execute pre-introspection and pre-deployment command
by Anirudh Gupta 13 Oct '21

13 Oct '21

3 6

[docs] has_project_guide key
by Peter Matulis 13 Oct '21

13 Oct '21

Does the has_project_guide key in the www/project-data/<RELEASE>.yaml file have any meaning? One of my projects has been dragging this key along from release to release and I do not see it documented [1]. I want to avoid unintended results if that key is removed. Thanks. [1]: https://docs.openstack.org/doc-contrib-guide/doc-tools/template-generator.h… Peter

1 0

[cinder] Bug deputy report for week of 10-13-2021
by Sofia Enriquez 13 Oct '21

13 Oct '21

This is a bug report from 10-06-2021-15-09 to 10-13-2021. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- Medium - https://bugs.launchpad.net/cinder/+bug/1946736 'Powerflex driver: update supported storage versions'. - https://bugs.launchpad.net/cinder/+bug/1946350 '[gate-failure] nova-live-migration evacuation failure due to slow lvchange -a command in c-vol during volume attachment update'. - https://bugs.launchpad.net/cinder/+bug/1946340 '[gate-failure] Unable to stack on fedora34 due to cinder pulling in oslo.vmware and suds-jurko that uses use_2to3 that is invalid with setuptools >=58.0.0'. - https://bugs.launchpad.net/cinder/+bug/1946263 'NetApp ONTAP Failing migrating volume from/to FlexGroup pool'. Low - https://bugs.launchpad.net/cinder/+bug/1946618 'Add same volume to the group-update does not show proper error'. Wishlist - https://bugs.launchpad.net/cinder/+bug/1946645 '[doc] Install and configure a storage node in cinder'. Cheers -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[KEYSTONE][POLICIES] - Overrides that don't work?
by Gaël THEROND 13 Oct '21

13 Oct '21

Hi team, I'm having a weird behavior with my Openstack platform that makes me think I may have misunderstood some mechanisms on the way policies are working and especially the overriding. So, long story short, I've few services that get custom policies such as glance that behave as expected, Keystone's one aren't. All in all, here is what I'm understanding of the mechanism: This is the keystone policy that I'm looking to override: https://paste.openstack.org/show/bwuF6jFISscRllWdUURL/ This policy default can be found in here: https://opendev.org/openstack/keystone/src/branch/master/keystone/common/po… Here is the policy that I'm testing: https://paste.openstack.org/show/bHQ0PXvOro4lXNTlxlie/ I know, this policy isn't taking care of the admin role but it's not the point. >From my understanding, any user with the project-manager role should be able to add any available user on any available group as long as the project-manager domain is the same as the target. However, when I'm doing that, keystone complains that I'm not authorized to do so because the user token scope is 'PROJECT' where it should be 'SYSTEM' or 'DOMAIN'. Now, I wouldn't be surprised of that message being thrown out with the default policy as it's stated on the code with the following: https://opendev.org/openstack/keystone/src/branch/stable/ussuri/keystone/co… So the question is, if the custom policy doesn't override the default scope_types how am I supposed to make it work? I hope it was clear enough, but if not, feel free to ask me for more information. PS: I've tried to assign this role with a domain scope to my user and I've still the same issue. Thanks a lot everyone!

2 4

[kolla-ansible][neutron] configuration question
by Ignazio Cassano 13 Oct '21

13 Oct '21

Hello Everyone, I need to know if it possible configure kolla neutron ovs with more than one bridge mappings, for example: bridge_mappings = physnet1:br-ex,physnet2:br-ex2 I figure out that in standard configuration ansible playbook create br-ex and add the interface with variable "neutron_external_interface" under br-ex. What can I do if I need to do if I wand more than one bridge ? How kolla ansible playbook can help in this case ? I could use multiple bridges in /etc/kolla/config neutron configuration files, but I do not know how ansible playbook can do the job. because I do not see any variable can help me in /etc/kolla/globals.yml Thanks Ignazio

2 3

Re: [neutron] OVN and dynamic routing
by Francois 13 Oct '21

13 Oct '21

...forgot to add the mailing list in the reply On Wed, 13 Oct 2021 at 10:01, Francois <rigault.francois(a)gmail.com> wrote: > > On Tue, 12 Oct 2021 at 21:46, Dan Sneddon <dsneddon(a)redhat.com> wrote: > > > > On 10/12/21 07:03, Francois wrote: > > > Hello Neutron! > > > I am looking into running stacks with OVN on a leaf-spine network, and > > > have some floating IPs routed between racks. > > > > > > Basically each rack is assigned its own set of subnets. > > > Some VLANs are stretched across all racks: the provisioning VLAN used > > > by tripleo to deploy the stack, and the VLANs for the controllers API > > > IPs. However, each tenant subnet is local to a rack: for example each > > > OVN chassis has a ovn-encap-ip with an IP of the tenant subnet of its > > > own rack. Traffic between 2 racks is sent to a spine, and leaves and > > > spines run some eVPN-like thing: each pair of ToR is a vtep, traffic > > > is encapsulated as VXLAN, and routes between vteps are exchanged with > > > BGP. > > > > > > > There has been a lot of work put into TripleO to allow you to provision > > hosts across L3 boundaries using DHCP relay. You can create a routed > > provisioning network using "helper-address" or vendor-specific commands > > on your top-of-rack switches, and a different subnet and DHCP address > > pool per rack. > Yes I saw that in the doc. I was not planning on using this for reasons I mentioned in another reply (this provisioning network is ""useless most of the time"" since there is almost no provisioning happening :D ) If any, I would love to work on Ironic DHCP-less deployments which was almost working last time I tried and I saw Ironic team contributing fixes since then. > > > > > > I am looking into supporting floating IPs in there: I expect floating > > > IPs to be able to move between racks, as such I am looking into > > > publishing the route for a FIP towards an hypervisor, through BGP. > > > Each fip is a /32 subnet with an hypervisor tenant's IP as next hop. > > > > This is becoming a very common architecture, and that is why there are > > several projects working to achieve the same goal with slightly > > different implementations. > > > > > > > > It seems there are several ideas to achieve this (it was discussed > > > [before][1] in ovs conference) > > > - using [neutron-dynamic-routing][2] - that seems to have some gaps > > > for OVN. It uses os-ken to talk to switches and exchange routes > > > - using [OVN BGP agent][3] that uses FRR, it seems there is a related > > > [RFE][4] for integration in tripleo > > > > > > There is btw also a [BGPVPN][5] project (it does not match my usecase > > > as far as I tried to understand it) that also has some code that talks > > > BGP to switches, already integrated in tripleo. > > > > > > For my tests, I was able to use the neutron-dynamic-routing project > > > (almost) as documented, with a few changes: > > > - for traffic going from VMs to outside the stack, the hypervisor was > > > trying to resolve the "gateway of fIPs" with ARP request which does > > > not make any sense. I created a dummy port with the mac address of the > > > virtual router of the switches: > > > ``` > > > $ openstack port list --mac-address 00:1c:73:00:00:11 -f yaml > > > - Fixed IP Addresses: > > > - ip_address: 10.64.254.1 > > > subnet_id: 8f37 > > > ID: 4028 > > > MAC Address: 00:1c:73:00:00:11 > > > Name: lagw > > > Status: DOWN > > > ``` > > > this prevent the hypervisor to send ARP requests to a non existent gateway > > > - for traffic coming back, we start the neutron-bgp-dragent agent on > > > the controllers. We create the right bgp speaker, peers, etc. > > > - neutron-bgp-dragent seems to work primarily with ovs ml2 plugin, it > > > selects fips and join with ports owned by a "floatingip_agent_gateway" > > > which does not exist on OVN. We can define ourselves some ports so > > > that the dragent is able to find the tenant IP of a host: > > > ``` > > > openstack port create --network provider --device-owner > > > network:floatingip_agent_gateway --host cpu35d.cloud --fixed-ip > > > ip-address=10.64.245.102 ag2 > > > ``` > > > - when creating a floating IP and assigning a port to it, Neutron > > > reads changes from OVN SB and fills the binding information into the > > > port: > > > ``` > > > $ openstack port show -c binding_host_id `openstack floating ip show > > > 10.64.254.177 -f value -c port_id` > > > +-----------------+----------------------------------------+ > > > | Field | Value | > > > +-----------------+----------------------------------------+ > > > | binding_host_id | cpu35d.cloud | > > > +-----------------+----------------------------------------+ > > > ``` > > > this allows the dragent to publish the route for the fip > > > ``` > > > $ openstack bgp speaker list advertised routes bgpspeaker > > > +------------------+---------------+ > > > | Destination | Nexthop | > > > +------------------+---------------+ > > > | 10.64.254.177/32 | 10.64.245.102 | > > > +------------------+---------------+ > > > ``` > > > - traffic reaches the hypervisor but (for reason I don't understand) I > > > had to add a rule > > > ``` > > > $ ip rule > > > 0: from all lookup local > > > 32765: from all iif vlan1234 lookup ovn > > > 32766: from all lookup main > > > 32767: from all lookup default > > > $ ip route show table ovn > > > 10.64.254.177 dev vlan1234 scope link > > > ``` > > > so that the traffic coming for the fip is not immediately discarded by > > > the hypervisor (it's not an ideal solution but it is a workaround that > > > makes my one fIP work!) > > > > > > So all in all it seems it would be possible to use the > > > neutron-dynamic-routing agent, with some minor modifications (eg: to > > > also publish the fip of the OVN L3 gateway router). > > > > > > I am wondering whether I have overlooked anything, and if such kind of > > > deployment (OVN + neutron dynamic routing or similar) is already in > > > use somewhere. Does it make sense to have a RFE for better integration > > > between OVN and neutron-dynamic-routing? > > > > I have been helping to contribute to integrating FRR with OVN in order > > to advertise FIPs and provider network IPs into BGP. The OVN BGP Agent > > is very new, and I'm pretty sure that nobody is using it in production > > yet. However the initial implementation is fairly simple and hopefully > > it will mature quickly. > > > > As you discovered, the solution that uses neutron-bgp-dragent and os-ken > > is not compatible with OVN > Pretty much the contrary, it basically worked. There are a few differences but the gap seems very tiny (unless I overlooked something and I'm fundamentally wrong) I don't understand why a new project would be needed to make it work for OVN. > > >, that is why ovs-bgp-agent is being > > developed. You should be able to try the ovs-bgp-agent with FRR and > > properly configured routing switches, it functions for the basic use case. > > > > The OVN BGP Agent will ensure that FIP and provider network IPs are > > present in the kernel as a /32 or /128 host route, which is then > > advertised into the BGP fabric using the FRR BGP daemon. If the default > > route is received from BGP it will be installed into the kernel by the > > FRR zebra daemon which syncs kernel routes with the FRR BGP routing > > table. The OVN BGP agent installs flows for the Neutron network gateways > > that hand off traffic to the kernel for routing. Since the kernel > > routing table is used, the agent isn't compatible with DPDK fast > > datapath yet. > > > > We don't have good documentation for the OVN BGP integration yet. I've > > only recently been able to make it my primary priority, and some of the > > other engineers which have done the initial proof of concept are moving > > on to other projects. There will be some discussions at the upcoming > > OpenStack PTG about this work, but I am hopeful that the missing pieces > > for your use case will come about in the Yoga cycle. > I did not try to run the OVN BGP agent but I saw your blog posts and I think it's enough to get started with. I still don't get why an extra OVN BGP agent would be needed. One thing I was wondering from the blog posts (and your reply here) is whether every single compute would need connectivity to the physical switches to publish the routes - as the dragent runs on the controller node you only need to configure connectivity between the controllers and the physical switches while in the FRR case you need to open much more. > Would the developments in the Yoga cycle be focused on the OVN BGP agent only, and so there is no interest in improving the neutron-dynamic-routing project ? > Thanks for your insightful comments :) > > > > > > > > > Thanks > > > Francois

1 0

[TripleO] Timeout while introspecting Overcloud Node
by Anirudh Gupta 12 Oct '21

12 Oct '21

Hi Team, We were trying to provision Overcloud Nodes using the Tripleo wallaby release. For this, on Undercloud machine (Centos 8.4), we downloaded the ironic-python and overcloud images from the following link: https://images.rdoproject.org/centos8/wallaby/rdo_trunk/current-tripleo/ After untarring, we executed the command *openstack overcloud image upload* This command setted the images at path /var/lib/ironic/images folder successfully. Then we uploaded our instackenv.json file and executed the command *openstack overcloud node introspect --all-manageable* On the overcloud node, we are getting the Timeout error while getting the agent.kernel and agent.ramdisk image. *http://10.0.1.10/8088/agent.kernel......Connection <http://10.0.1.10/8088/agent.kernel......Connection> timed out (http://ipxe.org/4c0a6092 <http://ipxe.org/4c0a6092>)* *http://10.0.1.10/8088/agent.kernel......Connection <http://10.0.1.10/8088/agent.kernel......Connection> timed out (http://ipxe.org/4c0a6092 <http://ipxe.org/4c0a6092>)* However, from another test machine, when I tried *wget http://10.0.1.10/8088/agent.kernel <http://10.0.1.10/8088/agent.kernel>* - It successfully worked Screenshot is attached for the reference Can someone please help in resolving this issue. Regards Anirudh Gupta

1 1

[TripleO] Issue in running Pre-Introspection
by Anirudh Gupta 12 Oct '21

12 Oct '21

Hi Team, I tried installing Undercloud using the below link: https://docs.openstack.org/project-deploy-guide/tripleo-docs/latest/deploym… I am getting the following error: (undercloud) [stack@undercloud ~]$ openstack tripleo validator run --group pre-introspection Selected log directory '/home/stack/validations' does not exist. Attempting to create it. +--------------------------------------+---------------------------------+--------+------------+-----------------+-------------------+-------------+ | UUID | Validations | Status | Host_Group | Status_by_Host | Unreachable_Hosts | Duration | +--------------------------------------+---------------------------------+--------+------------+-----------------+-------------------+-------------+ | 7029c1f6-5ab4-465d-82d7-3f29058012ce | check-cpu | PASSED | localhost | localhost | | 0:00:02.531 | | db059017-30f1-4b97-925e-3f55b586d492 | check-disk-space | PASSED | localhost | localhost | | 0:00:04.432 | | e23dd9a1-90d3-4797-ae0a-b43e55ab6179 | check-ram | PASSED | localhost | localhost | | 0:00:01.324 | | 598ca02d-258a-44ad-b78d-3877321cdfe6 | check-selinux-mode | PASSED | localhost | localhost | | 0:00:01.591 | | c4435b4c-b432-4a1e-8a99-00638034a884 | *check-network-gateway | FAILED* | undercloud | *No host matched* | | | | cb1eed23-ef2f-4acd-a43a-86fb09bf0372 | *undercloud-disk-space | FAILED* | undercloud | *No host matched* | | | | abde5329-9289-4b24-bf16-c4d82b03e67a | *undercloud-neutron-sanity-check | FAILED* | undercloud | *No host matched* | | | | d0e5fdca-ece6-4a37-b759-ed1fac31a10f | *ctlplane-ip-range | FAILED* | undercloud | No host matched | | | | 91511807-225c-4852-bb52-6d0003c51d49 | *dhcp-introspection | FAILED* | undercloud | No host matched | | | | e96f7704-d2fb-465d-972b-47e2f057449c |* undercloud-tokenflush | FAILED *| undercloud | No host matched | | | As per the validation link, https://docs.openstack.org/tripleo-validations/wallaby/validations-pre-intr… check-network-gateway If gateway in undercloud.conf is different from local_ip, verify that the gateway exists and is reachable Observation - In my case IP specified in local_ip and gateway, both are pingable, but still this error is being observed ctlplane-ip-range¶ <https://docs.openstack.org/tripleo-validations/wallaby/validations-pre-intr…> Check the number of IP addresses available for the overcloud nodes. Verify that the number of IP addresses defined in dhcp_start and dhcp_end fields in undercloud.conf is not too low. - ctlplane_iprange_min_size: 20 Observation - In my case I have defined more than 20 IPs Similarly for disk related issue, I have dedicated 100 GB space in /var and / Filesystem Size Used Avail Use% Mounted on devtmpfs 12G 0 12G 0% /dev tmpfs 12G 84K 12G 1% /dev/shm tmpfs 12G 8.7M 12G 1% /run tmpfs 12G 0 12G 0% /sys/fs/cgroup /dev/mapper/cl-root 100G 2.5G 98G 3% / /dev/mapper/cl-home 47G 365M 47G 1% /home /dev/mapper/cl-var 103G 1.1G 102G 2% /var /dev/vda1 947M 200M 747M 22% /boot tmpfs 2.4G 0 2.4G 0% /run/user/0 tmpfs 2.4G 0 2.4G 0% /run/user/1000 Despite setting al the parameters, still I am not able to pass pre-introspection checks. *"NO Host Matched" *is found in the table. Regards Anirudh Gupta

2 3

[neutron] OVN and dynamic routing
by Francois 12 Oct '21

12 Oct '21

Hello Neutron! I am looking into running stacks with OVN on a leaf-spine network, and have some floating IPs routed between racks. Basically each rack is assigned its own set of subnets. Some VLANs are stretched across all racks: the provisioning VLAN used by tripleo to deploy the stack, and the VLANs for the controllers API IPs. However, each tenant subnet is local to a rack: for example each OVN chassis has a ovn-encap-ip with an IP of the tenant subnet of its own rack. Traffic between 2 racks is sent to a spine, and leaves and spines run some eVPN-like thing: each pair of ToR is a vtep, traffic is encapsulated as VXLAN, and routes between vteps are exchanged with BGP. I am looking into supporting floating IPs in there: I expect floating IPs to be able to move between racks, as such I am looking into publishing the route for a FIP towards an hypervisor, through BGP. Each fip is a /32 subnet with an hypervisor tenant's IP as next hop. It seems there are several ideas to achieve this (it was discussed [before][1] in ovs conference) - using [neutron-dynamic-routing][2] - that seems to have some gaps for OVN. It uses os-ken to talk to switches and exchange routes - using [OVN BGP agent][3] that uses FRR, it seems there is a related [RFE][4] for integration in tripleo There is btw also a [BGPVPN][5] project (it does not match my usecase as far as I tried to understand it) that also has some code that talks BGP to switches, already integrated in tripleo. For my tests, I was able to use the neutron-dynamic-routing project (almost) as documented, with a few changes: - for traffic going from VMs to outside the stack, the hypervisor was trying to resolve the "gateway of fIPs" with ARP request which does not make any sense. I created a dummy port with the mac address of the virtual router of the switches: ``` $ openstack port list --mac-address 00:1c:73:00:00:11 -f yaml - Fixed IP Addresses: - ip_address: 10.64.254.1 subnet_id: 8f37 ID: 4028 MAC Address: 00:1c:73:00:00:11 Name: lagw Status: DOWN ``` this prevent the hypervisor to send ARP requests to a non existent gateway - for traffic coming back, we start the neutron-bgp-dragent agent on the controllers. We create the right bgp speaker, peers, etc. - neutron-bgp-dragent seems to work primarily with ovs ml2 plugin, it selects fips and join with ports owned by a "floatingip_agent_gateway" which does not exist on OVN. We can define ourselves some ports so that the dragent is able to find the tenant IP of a host: ``` openstack port create --network provider --device-owner network:floatingip_agent_gateway --host cpu35d.cloud --fixed-ip ip-address=10.64.245.102 ag2 ``` - when creating a floating IP and assigning a port to it, Neutron reads changes from OVN SB and fills the binding information into the port: ``` $ openstack port show -c binding_host_id `openstack floating ip show 10.64.254.177 -f value -c port_id` +-----------------+----------------------------------------+ | Field | Value | +-----------------+----------------------------------------+ | binding_host_id | cpu35d.cloud | +-----------------+----------------------------------------+ ``` this allows the dragent to publish the route for the fip ``` $ openstack bgp speaker list advertised routes bgpspeaker +------------------+---------------+ | Destination | Nexthop | +------------------+---------------+ | 10.64.254.177/32 | 10.64.245.102 | +------------------+---------------+ ``` - traffic reaches the hypervisor but (for reason I don't understand) I had to add a rule ``` $ ip rule 0: from all lookup local 32765: from all iif vlan1234 lookup ovn 32766: from all lookup main 32767: from all lookup default $ ip route show table ovn 10.64.254.177 dev vlan1234 scope link ``` so that the traffic coming for the fip is not immediately discarded by the hypervisor (it's not an ideal solution but it is a workaround that makes my one fIP work!) So all in all it seems it would be possible to use the neutron-dynamic-routing agent, with some minor modifications (eg: to also publish the fip of the OVN L3 gateway router). I am wondering whether I have overlooked anything, and if such kind of deployment (OVN + neutron dynamic routing or similar) is already in use somewhere. Does it make sense to have a RFE for better integration between OVN and neutron-dynamic-routing? Thanks Francois [1]: https://www.openvswitch.org/support/ovscon2020/slides/OVS-CONF-2020-OVN-WIT… [2]: https://docs.openstack.org/neutron/latest/admin/config-bgp-dynamic-routing.… [3]: https://ltomasbo.wordpress.com/2021/02/04/openstack-networking-with-bgp/ [4]: https://specs.openstack.org/openstack/tripleo-specs/specs/wallaby/triplo-bg… [5]: https://github.com/openstack/tripleo-heat-templates/blob/master/environment…

4 5

After rebuilding Queens clusters on Train, race condition causes Designate record creation to fail
by Braden, Albert 12 Oct '21

12 Oct '21

Hello everyone. It's great to be back working on OpenStack again. I'm at Verisign now. I can hardly describe how happy I am to have an employer that does not attach nonsense to the bottom of my emails! We are rebuilding our clusters from Queens to Train. On the new Train clusters, customers are complaining that deleting a VM and then immediately creating a new one with the same name (via Terraform for example) intermittently results in a missing DNS record. We can duplicate the issue by building a VM with terraform, tainting it, and applying. Before applying the change, we see the DNS record in the recordset: $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra | f9aa73c1-84ba-4854-be71-cbb616de672c | 8d1c84082a044a53abe0d519ed9e8c60 | openstack-terra-test-host.dev-ostck.dva3.vrsn.com. | A | 10.220.4.89 | ACTIVE | NONE | $ and we can pull it from the DNS server on the controllers: $ for i in {1..3}; do dig @dva3-ctrl${i}.cloud.vrsn.com -t axfr dva3.vrsn.com. |grep openstack-terra; done openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 openstack-terra-test-host.dev-ostck.dva3.vrsn.com. 1 IN A 10.220.4.89 After applying the change, we don't see it: $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra | f9aa73c1-84ba-4854-be71-cbb616de672c | 8d1c84082a044a53abe0d519ed9e8c60 | openstack-terra-test-host.dev-ostck.dva3.vrsn.com. | A | 10.220.4.89 | ACTIVE | NONE | $ $ for i in {1..3}; do dig @dva3-ctrl${i}.cloud.vrsn.com -t axfr dva3.vrsn.com. |grep openstack-terra; done $ openstack recordset list dva3.vrsn.com. --all |grep openstack-terra $ We see this in the logs: 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'c70e693b4c47402db088c43a5a177134-openstack-terra-test-host.de...' for key 'unique_recordset'") 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher [SQL: INSERT INTO recordsets (id, version, created_at, zone_shard, tenant_id, zone_id, name, type, ttl, reverse_name) VALUES (%(id)s, %(version)s, %(created_at)s, %(zone_shard)s, %(tenant_id)s, %(zone_id)s, %(name)s, %(type)s, %(ttl)s, %(reverse_name)s)] 2021-10-09 01:53:44.307 27 ERROR oslo_messaging.notify.dispatcher [parameters: {'id': 'dbbb904c347241a791aa01ca33a87b23', 'version': 1, 'created_at': datetime.datetime(2021, 10, 9, 1, 53, 44, 182652), 'zone_shard': 3184, 'tenant_id': '8d1c84082a044a53abe0d519ed9e8c60', 'zone_id': 'c70e693b4c47402db088c43a5a177134', 'name': 'openstack-terra-test-host.dev-ostck.dva3.vrsn.com.', 'type': 'A', 'ttl': None, 'reverse_name': '.moc.nsrv.3avd.kctso-ved.tsoh-tset-arret-kcatsnepo'}] It appears that Designate is trying to create the new record before the deletion of the old one finishes. Is anyone else seeing this on Train? The same set of actions doesn't cause this error in Queens. Do we need to change something in our Designate config, to make it wait until the old records are finished deleting before attempting to create the new ones?

2 6

[Octavia] Can not create LB on SRIOV network
by Zhang, Jing C. (Nokia - CA/Ottawa) 12 Oct '21

12 Oct '21

I can not create Octavia LB on SRIOV network in Train. I went to Octavia story board, did a search but was unable to figure out (the story for SRIOV...). I left a comment under this story, I re-post my questions there, hoping someone knows the answer. Thank you so much Jing https://storyboard.openstack.org/#!/story/2006886 Add VM SRIOV Interface Config Guide (Openstack) Hi, In Openstack train release, creating Octavia LB on SRIOV network fails. I come here to search if there is already a plan to add this support, and see this story. This story gives the impression that the capability is already supported, it is a matter of adding user guide. So, my question is, in which Openstack release, creating LB on SRIOV network is supported? Thank you

3 7

[Xena] It works!
by tjoen 12 Oct '21

12 Oct '21

Just testing every release since Train on an LFS system with Python-3.9 cryptography-35.0.0 is necessary Thank you all

3 5

[ironic] No weekly meeting on Oct18
by Iury Gregory 12 Oct '21

12 Oct '21

Hello ironicers! Just a reminder that on Oct 18, we won't have our weekly meeting because we have a session in the PTG. -- *Att[]'sIury Gregory Melo Ferreira * *MSc in Computer Science at UFCG* *Part of the ironic-core and puppet-manager-core team in OpenStack* *Software Engineer at Red Hat Czech* *Social*: https://www.linkedin.com/in/iurygregory *E-mail: iurygregory(a)gmail.com <iurygregory(a)gmail.com>*

1 0

[nova & libvirt] about attach disk on aarch64
by Jaze Lee 12 Oct '21

12 Oct '21

Hello, We run stein openstack in our environment. And already set libvirt value in nova.conf hw_machine_type=aarch64=virt num_pcie_ports = 15 We test and find sometimes disks can not be attached correctly. For example, Built vm with 6 disks, only three disk be there. The others will be inactive in virsh. No obvious error can be found in nova-compute, libvirt, vm os logs. libvirt:5.0.0 qemu:2.12.0-44 openstack-nova: 19.3.2 librbd1:ibrbd1-14.2.16-1.el7.aarch64 Any Suggestions? Thanks a lot

2 1

[wallaby][neutron] Distributed floating IP
by Ammad Syed 12 Oct '21

12 Oct '21

Hi, I am using openstack wallaby. I have seen an issue not sure if its a bug or configuration related issue. I am using ml2/ovn backend with distributed floating IP enabled. I have made my compute node 1 as a gateway chassis where the routers are scheduled. I have then created an instance and NATed a public IP. The instance deployed on compute 2. When I see the IP address via curl ipinfo.io it shows the floating IP that I have NATed. Then I migrated the instance to compute node 1. I had many ping drops for a couple of seconds then its back to normal. I have then seen the IP address via curl ipinfo.io. It showed me the SNAT IP address of router. Then I migrated the instance back to compute node 2, I had ping drops for 20 seconds and then the instance came back. I have seen the IP via curl, it showed the floating IP that I have nated with instance. Is it the expected behavior ? Ammad

2 2

[docs] Keystone docs missing for Xena
by Ben Nemec 12 Oct '21

12 Oct '21

Hey, I was just looking for the Keystone docs and discovered that they are not listed on https://docs.openstack.org/xena/projects.html. If I s/wallaby/xena/ on the wallaby version then it resolves, so it looks like the docs are published they just aren't included in the index for some reason. -Ben

2 1

[kuryr] Virtual PTG October 2021
by Maysa De Macedo Souza 12 Oct '21

12 Oct '21

Hello, I booked the following slots for Kuryr during the Yoga PTG: Oct 19 7-8 UTC and Oct 22 13-14 UTC. If you have any topic ideas you would like to discuss, please include them in the etherpad[1], also it would be interesting to include your name there if you plan to attend any Kuryr session. See you on the next PTG. Cheers, Maysa. [1] https://etherpad.opendev.org/p/kuryr-yoga-ptg

1 2

Restarting Openstack Victoria using kolla-ansible
by Karera Tony 12 Oct '21

12 Oct '21

Hello Team, I am trying to deploy openstack Victoria ..... Below I install kolla-ansible on the deployment server , I first clone the * git clone --branch stable/victoria https://opendev.org/openstack/kolla-ansible* but when I run the deployment without uncommenting openstack_release .. By default it deploys wallaby And when I uncomment and type victoria ...Some of the containers keep restarting esp Horizon Any idea on how to resolve this ? Even the kolla content that I use for deployment, I get it from the kolla-ansible directory that I cloned Regards Tony Karera

2 2

Edge sessions at the upcoming PTG
by Ildiko Vancsa 11 Oct '21

11 Oct '21

Hi, I’m reaching out to you to share the agenda of the OpenInfra Edge Computing Group that we put together for the upcoming PTG. I would like to invite everyone who is interested in discussing edge challenges and finding solutions! We summarized our plans for the event in a short blog post to give some context to each of the topic that we picked to discuss in details. We picked key areas like security, networking, automation and tools, containers and more: https://superuser.openstack.org/articles/the-project-teams-gathering-is-com… Our etherpad for the sessions: https://etherpad.opendev.org/p/ecg-ptg-october-2021 Please let me know if you have any questions about the agenda or topics. Thanks and Best Regards, Ildikó

1 2

[TripleO] PTG proposed schedule
by James Slagle 11 Oct '21

11 Oct '21

I have put up a tentative schedule in the etherpad for each of our proposed sessions: https://etherpad.opendev.org/p/tripleo-yoga-topics If there are any scheduling conflicts with other sessions, please let me know and we will do our best to adjust. We also have time to add a 4th session on Monday, Tuesday, Wednesday, so if you have some last minute topics, feel free to add them. Thanks, and looking forward to seeing (virtually) everyone next week! -- -- James Slagle --

1 0

OpenInfra Live - October 14, 2021 at 9am CT
by Ashlee Ferguson 11 Oct '21

11 Oct '21

Hi everyone, This week’s OpenInfra Live episode is brought to you by the OpenStack community. Networking is complex, and Neutron is one of the most difficult parts of OpenStack to scale. In this episode of the Large Scale OpenStack show, we will explore early architectural choices you can make, recommended drivers, features to avoid if your ultimate goal is to scale to a very large deployment. Join OpenStack developers and operators as they share their Neutron scaling best practices. Episode: Large Scale OpenStack: Neutron scaling best practices Date and time: October 14, 2021 at 9am CT (1400 UTC) You can watch us live on: YouTube: https://www.youtube.com/watch?v=4ZLqILbLIpQ <https://www.youtube.com/watch?v=4ZLqILbLIpQ> LinkedIn: https://www.linkedin.com/video/event/urn:li:ugcPost:6851936962715222016/ <https://www.linkedin.com/video/event/urn:li:ugcPost:6851936962715222016/> Facebook: https://www.facebook.com/104139126308032/posts/4407685335953368/ <https://www.facebook.com/104139126308032/posts/4407685335953368/> WeChat: recording will be posted on OpenStack WeChat after the live stream Speakers: Thierry Carrez (OpenInfra Foundation) David Comay (Bloomberg) Ibrahim Derraz (Exaion) Slawek Kaplonski (Red Hat) Lajos Katona (Ericsson) Mohammed Naser (VEXXHOST) Michal Nasiadka (StackHPC) Have an idea for a future episode? Share it now at ideas.openinfra.live <https://openinfrafoundation.formstack.com/forms/openinfralive>. Register now for OpenInfra Live: Keynotes, a special edition of OpenInfra Live on November 17-18th starting at 1500 UTC: https://openinfralivekeynotes.eventbrite.com/ <https://openinfralivekeynotes.eventbrite.com/> Thanks! Ashlee

1 0

[Barbican] HSM integration with FIPS Operation Enabled
by rajiv mucheli 11 Oct '21

11 Oct '21

Hi, I looked into the available documentation and article but i had no luck validating if Openstack Barbican integration with FIPS Operation mode Enabled works. Any suggestions? The below barbican backend guide shares the available plugins with HSM : https://docs.openstack.org/security-guide/secrets-management/barbican.html Does Barbican now support module generated IV ? which is required for FIPS support in Thales A790 HSM. Regards, Rajiv

1 0

[all][PTG] October 2021 - PTGbot, Etherpads, & IRC
by Ashlee Ferguson 11 Oct '21

11 Oct '21

Hello! We just wanted to take a second to point out a couple things that have changed since the last PTG as we all get ready for the next PTG. Firstly, the PTGbot is up to date and ready to go *at it's new URL[1]*-- as are the autogenerated etherpads! There you can find the schedule page, etherpads, etc. If you/your team have already created an etherpad, please feel free to use the PTGbot to override the default, auto-generated one[2]. Secondly, just a reminder that with the migration to being more inclusive of all Open Infrastructure Foundation projects we will be using the #openinfra-events IRC channel on the OFTC network! And again, if you haven't yet, please register[3]! Its free and important for getting the zoom information, etc. Thanks! Ashlee (ashferg) and Kendall (diablo_rojo) [1] PTGbot: https://ptg.opendev.org/ [2] PTGbot Etherpad Override Command: https://opendev.org/openstack/ptgbot/src/branch/master/README.rst#etherpad [3] PTG Registration: https://openinfra-ptg.eventbrite.com

1 0

[neutron] Bug deputy, report of week 2021-10-4
by Rodolfo Alonso Hernandez 11 Oct '21

11 Oct '21

Hello Neutrinos: This is the last week report: High: - https://bugs.launchpad.net/neutron/+bug/1946187: HA routers not going to be "primary" at all. Unassigned. - https://bugs.launchpad.net/neutron/+bug/1931696. ovs offload broken from neutron 16.3.0 onwards. Assigned. - https://review.opendev.org/c/openstack/neutron/+/812641 - https://bugs.launchpad.net/neutron/+bug/1946318: [ovn] Memory consumption grows over time due to MAC_Binding entries in SB database. Assigned. - https://review.opendev.org/c/openstack/neutron/+/812805 - https://bugs.launchpad.net/neutron/+bug/1946456: [OVN] Scheduling of HA Chassis Group for external port does not work when no chassis has 'enable-chassis-as-gw' option set. Unassigned. - https://bugs.launchpad.net/neutron/+bug/1946588: [OVN]Metadata get warn logs after boot instance server about "MetadataServiceReadyWaitTimeoutException". Assigned. - https://review.opendev.org/c/openstack/neutron/+/813376 Medium: - https://bugs.launchpad.net/neutron/+bug/1946186: Fullstack test neutron.tests.fullstack.test_l3_agent.TestHAL3Agent.test_router_fip_qos_after_admin_state_down_up failing intermittently. Unassigned. - https://bugs.launchpad.net/neutron/+bug/1946479: [OVN migration] qr- interfaces and trunk subports aren't cleaned after migration to ML2/OVN. Assigned. - https://review.opendev.org/c/openstack/neutron/+/813186 - https://review.opendev.org/c/openstack/neutron/+/813187 - https://bugs.launchpad.net/neutron/+bug/1946589: [OVN] localport might not be updated when create multiple subnets for its network. Unassigned. Low: - https://bugs.launchpad.net/neutron/+bug/1945954: [os-ken] Missing subclass for SUBTYPE_RIB_*_MULTICAST in mrtlib. Assigned. - https://review.opendev.org/c/openstack/os-ken/+/812293 - https://bugs.launchpad.net/neutron/+bug/1946023: [OVN] Check OVN Port_Group compatibility. Assigned. - https://review.opendev.org/c/openstack/neutron/+/812176 - https://bugs.launchpad.net/neutron/+bug/1946250: Neutron API reference should explain the intended behavior of port security extension. Unassigned. Whishlist: - https://bugs.launchpad.net/neutron/+bug/1946251: [RFE] API: allow to disable anti-spoofing but not SGs. Assigned. Duplicated: - https://bugs.launchpad.net/neutron/+bug/1945646: Nova fails to live migrate instance with upper-case port MAC Incomplete: - https://bugs.launchpad.net/neutron/+bug/1946535: Segment plugin disabled delete network will raise exception. - Maybe “segments” plugin is loaded in this deployment. - https://bugs.launchpad.net/neutron/+bug/1946624: OVSDB Error: Transaction causes multiple rows in "Port_Group" table to have identical values. - Maybe duplicated of https://bugs.launchpad.net/neutron/+bug/1938766. Regards.

1 0

[kolla] parent tags
by Sam Morrison 11 Oct '21

11 Oct '21

I’m trying to be able to build a projects container without having to rebuild the parents which have different tags. The workflow I’m trying to achieve is: Build base and openstack-base with a tag of wallaby Build a container image for barbican with a tag of the version of barbican that is returned when doing `git describe` Build a container image for nova with a tag of the version of barbican that is returned when doing `git describe` etc.etc. I don’t seem to be able to do this without having to also build a new base and openstack-base with the same tag which is slow and also means a lot of disk space. Just wondering how other people do this sort of stuff? Any ideas? Thanks, Sam

3 4

[Horizon] Yoga PTG Schedule
by vishal manchanda 10 Oct '21

10 Oct '21

Hello everyone, I have booked the below slots for Horizon Yoga PTG: Monday, October 18, 15:00 - 17:00 UTC Tuesday, October 19, 15:00 - 17:00 UTC Wednesday, October 20, 16:00 - 17:00 UTC I have also created Etherpad to collect topics for ptg discussion [1]. Feel free to add your topics. Please Let me know in case you have any topics to discuss and the above schedule doesn't work for you and We will see how to manage that. Also Please register for PTG if you haven't done it yet [2]. Thank you, Vishal Manchanda(irc: vishalmanchanda) [1] https://etherpad.opendev.org/p/horizon-yoga-ptg [2] https://www.eventbrite.com/e/project-teams-gathering-october-2021-tickets-1…

1 0

[sdk]: Remove volumes stuck in error deleting status
by dmeng 09 Oct '21

09 Oct '21

Hello there, Hope everything is going well. I would like to know if there is any method that could remove the volume stuck in the "error-deleting" status? We are using the chameleon openstack cloud but we are not the admin user there, so couldn't use the "cinder force-delete" or "cinder reset-state" command. Wondering if there is any other method we could use to remove those volumes in our own project? And also wondering what might cause this "error-deleting" problem? We use openstacksdk block storage service, "cinder.delete_volume()" method to remove volumes, and it works fine before. Thanks and have a great day! Catherine

2 1

Why is kolla-kubernetes not maintained anymore？
by fusco lu 09 Oct '21

09 Oct '21

hi,everyone Can you tell me the reason for not being maintained?

1 0

[kolla][wallaby][cinder] netapp driver failed do create volume from images
by Ignazio Cassano 09 Oct '21

09 Oct '21

Hello, I've just updated my kolla wallaby with latest images. When I create volume from image on ceph it works. When I create volume from image on nfs netapp ontap, it does not work. The following is reported in cinder-volume.log: 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 950, in _create_from_image_cache_or_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server model_update = self._create_from_image_download( 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/flows/manager/create_volume.py", line 766, in _create_from_image_download 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server volume_utils.copy_image_to_volume(self.driver, context, volume, 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server File "/var/lib/kolla/venv/lib/python3.8/site-packages/cinder/volume/volume_utils.py", line 1158, in copy_image_to_volume 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server raise exception.ImageCopyFailure(reason=ex.stderr) 2021-10-08 13:11:20.590 26 ERROR oslo_messaging.rpc.server cinder.exception.ImageCopyFailure: Failed to copy image to volume: qemu-img: /var/lib/cinder/mnt/55fcb53b884ca983ed3d6fa8aac57810/volume-68fd6214-c24a-4104-ae98-e9d0b60aa136: error while converting raw: Failed to lock byte 101 Any help please ? Ignazio

2 7

[all][tc] What's happening in Technical Committee: summary 8th Oct, 21: Reading: 5 min
by Ghanshyam Mann 08 Oct '21

08 Oct '21

Hello Everyone, Here is this week's summary of the Technical Committee activities. 1. TC Meetings: ============ * TC this week video meeting held on Oct 7th Thursday. * Most of the meeting discussions are summarized below (Completed or in-progress activities section). We forgot to record the meeting, an apology for that. I am summarizing each topic discussion, or you can also check some summary and transcript (these are autogenerated and not so perfect, though) @ - https://meetings.opendev.org/meetings/tc/2021/tc.2021-10-07-15.01.log.html - https://review.opendev.org/c/openstack/governance/+/813112/1/reference/tc-m… * We will have next week's IRC meeting on Oct 14th, Thursday 15:00 UTC, feel free the topic in agenda[1] by Oct 13th. 2. What we completed this week: ========================= * Added the cinder-netapp charm to Openstack charms[2] * Retired puppet-freezer[3] 3. Activities In progress: ================== TC Tracker for Xena cycle ------------------------------ * TC is using the etherpad[4] for Xena cycle working item. We will be checking and updating the status biweekly on the same etherpad. * Current status is: 9 completed, 3 to-be-discussed in PTG, 1 in-progress Open Reviews ----------------- * Four open reviews for ongoing activities[5]. Place to maintain the external hosted ELK, E-R, O-H services ------------------------------------------------------------------------- * We continue the discussion[6] on final checks and updates from Allison. Below is the summary: * Allison updated on the offer of the donated credits (45k/year from AWS) to run the ELK services. * Current ELK stack will be migrated on OpenSearch (an open source fork of Elasticsearch) cluster on AWS managed services. * We discussed and agreed to start/move these ELK service work under TACT SIG with the help of Daniel Pawlik, Ross Tomlinson, and Allison (along with Jeremy, Clark as backup/helping in migration). * Allison will continue work on setting up the accounts. * A huge thanks to Allison for driving and arranging the required resources. Add project health check tool ----------------------------------- * Using stats of review/code should not be the only criteria to decide the project health as it depends on many other factors, including the nature of project. * We agreed to use the generated stats only as an early warning tool and not to publish those anywhere which can be wrongly interpreted as project health or so. * We will continue discussing it in PTG for the next steps on this and what to do on TC liaison things. * Meanwhile, we are reviewing Rico proposal on collecting stats tool[7]. Stable Core team process change --------------------------------------- * Current proposal is under review[8]. Feel free to provide early feedback if you have any. Call for 'Technical Writing' SIG Chair/Maintainers ---------------------------------------------------------- * As you might have read in the email from Elod[9], Stephen, who is the current chair for this SIG is not planning to continue to chair. * This SIG has accomplished the work for what it was formed, and now most of the documentation are managed on projects side. TC agreed to move this SIG to complete state and move the repos under TC (tc members will be added to core group in those repos). * Any advisory work on documentation which is what this SIG was doing, will be handle in TC. TC tags analysis ------------------- * Operator feedback is asked on open infra newsletter too, and we will continue the discussion in PTG and will take the final decision based on feedback we receive, if any[10]. Project updates ------------------- * Retiring js-openstack-lib [11] Yoga release community-wide goal ----------------------------------------- * Please add the possible candidates in this etherpad [12]. * Current status: "Secure RBAC" is selected for Yoga cycle[13]. PTG planning ---------------- * We are collecting the PTG topics in etherpad[14], feel free to add any topic you would like to discuss. * We discussed the live stream of one of the TC PTG sessions like we did last time. Once we have more topics in etherpad, then we can select the appropriate one. Test support for TLS default: ---------------------------------- * Rico has started a separate email thread over testing with tls-proxy enabled[15], we encourage projects to participate in that testing and help to enable the tls-proxy in gate testing. 4. How to contact the TC: ==================== If you would like to discuss or give feedback to TC, you can reach out to us in multiple ways: 1. Email: you can send the email with tag [tc] on openstack-discuss ML[16]. 2. Weekly meeting: The Technical Committee conduct a weekly meeting every Thursday 15 UTC [17] 3. Office hours: The Technical Committee offers a weekly office hour every Tuesday at 0100 UTC [18] 4. Ping us using 'tc-members' nickname on #openstack-tc IRC channel. [1] https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting [2] https://review.opendev.org/c/openstack/governance/+/809011 [3] https://review.opendev.org/c/openstack/governance/+/808679 [4] https://etherpad.opendev.org/p/tc-xena-tracke [5] https://review.opendev.org/q/projects:openstack/governance+status:open [6] https://etherpad.opendev.org/p/elk-service-maintenance-plan [7] https://review.opendev.org/c/openstack/governance/+/810037 [8] https://review.opendev.org/c/openstack/governance/+/810721 [9] http://lists.openstack.org/pipermail/openstack-discuss/2021-October/025161.… [10] https://governance.openstack.org/tc/reference/tags/index.html [11] https://review.opendev.org/c/openstack/governance/+/798540 [12] https://review.opendev.org/c/openstack/governance/+/807163 [13] https://etherpad.opendev.org/p/y-series-goals [14] https://etherpad.opendev.org/p/tc-yoga-ptg [15] http://lists.openstack.org/pipermail/openstack-discuss/2021-June/023000.html [16] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss [17] http://eavesdrop.openstack.org/#Technical_Committee_Meeting [18] http://eavesdrop.openstack.org/#Technical_Committee_Office_hours -gmann

1 0

Introducing Hubtty, a Gertty fork for Github code reviews
by Martin André 08 Oct '21

08 Oct '21

Hi all, First off, apologies if this isn't the right forum, this has nothing to do with OpenStack development. I'm trying to reach out to the many Gertty users hiding here who might want a similar tool for their Github code reviews. I'm happy to announce the first release of Hubtty [1], a fork of Gertty that I adapted to the Github API and workflow. It has the same look and feel but differs in a few things I detailed in the release changelog [2]. This first version focuses on porting Gertty to Github and works reasonably well. Myself and other intrepid developers already use it every day and I personally find it very convenient for managing incoming PR reviews. In the coming versions I'd like to integrate better with the Github features and improve UX. Try it with `pip install hubtty` and let me know what you think of it. Note that Hubtty can't submit reviews to repositories for which the parent organization has enabled third-party application restrictions without explicitly allowing hubtty [3]. I'm working around the issue by using a token generated by the `gh` app. Martin [1] https://github.com/hubtty/hubtty [2] https://github.com/hubtty/hubtty/blob/v0.1/CHANGELOG.md [3] https://github.com/hubtty/hubtty/issues/20

1 0

[charms] Yoga PTG
by Alex Kavanagh 08 Oct '21

08 Oct '21

Hi all, The OpenStack charms PTG sessions are booked as: - Wednesday 20th October - 14.00 - 17.00 UTC in the Icehouse room - Thursday 21st October - 14.00 - 17.00 UTC also in the Icehouse room Please add your name and discussion topic proposals to the etherpad. [1]. The etherpad also has links to the PTG main site, schedule and Charms. Thank you in advance and see you soon! Alex (tinwood) [1] https://etherpad.opendev.org/p/charms-yoga-ptg

1 0

[heat] xena stack deployment failed
by Ammad Syed 08 Oct '21

08 Oct '21

Hi, I have upgraded my heat from wallaby to xena. When I am trying to create magnum cluster its giving below error in heat engine logs. Currently in whole stack, I have upgraded heat and magnum to latest release. Before upgrading heat from xena to wallaby, the stack deployment was successful. 2021-10-08 12:06:14.107 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] REQ: curl -g -i -X GET http://controller-khi04.rapid.pk:8774/v2.1/98687873a146418eaeeb54a01693669f -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}ef71182888b7ad3b8719ee8d48e37c6cd526e1f25f8395f733770917aead9b7b" -H "X-OpenStack-Nova-API-Version: 2.1" _http_log_request /usr/lib/python3/dist-packages/keystoneauth1/session.py:519 2021-10-08 12:06:14.121 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] RESP: [404] Connection: keep-alive Content-Length: 112 Content-Type: application/json Date: Fri, 08 Oct 2021 07:06:14 GMT X-Compute-Request-Id: req-d07e25b0-375f-48e6-ac4b-d76b41848e6a X-Openstack-Request-Id: req-d07e25b0-375f-48e6-ac4b-d76b41848e6a _http_log_response /usr/lib/python3/dist-packages/keystoneauth1/session.py:550 2021-10-08 12:06:14.121 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] RESP BODY: {"message": "The resource could not be found.<br /><br />\n\n\n", "code": "404 Not Found", "title": "Not Found"} _http_log_response /usr/lib/python3/dist-packages/keystoneauth1/session.py:582 2021-10-08 12:06:14.122 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] GET call to compute for http://controller-khi04.rapid.pk:8774/v2.1/98687873a146418eaeeb54a01693669f used request id req-d07e25b0-375f-48e6-ac4b-d76b41848e6a request /usr/lib/python3/dist-packages/keystoneauth1/session.py:954 2021-10-08 12:06:14.122 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] REQ: curl -g -i -X GET http://controller-khi04.rapid.pk:8774/v2.1/ -H "Accept: application/json" -H "User-Agent: python-novaclient" -H "X-Auth-Token: {SHA256}ef71182888b7ad3b8719ee8d48e37c6cd526e1f25f8395f733770917aead9b7b" -H "X-OpenStack-Nova-API-Version: 2.1" _http_log_request /usr/lib/python3/dist-packages/keystoneauth1/session.py:519 2021-10-08 12:06:14.128 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] RESP: [200] Connection: keep-alive Content-Length: 399 Content-Type: application/json Date: Fri, 08 Oct 2021 07:06:14 GMT Openstack-Api-Version: compute 2.1 Vary: OpenStack-API-Version, X-OpenStack-Nova-API-Version X-Compute-Request-Id: req-9b1f2443-fdee-41dd-9139-57376afd7bef X-Openstack-Nova-Api-Version: 2.1 X-Openstack-Request-Id: req-9b1f2443-fdee-41dd-9139-57376afd7bef _http_log_response /usr/lib/python3/dist-packages/keystoneauth1/session.py:550 2021-10-08 12:06:14.128 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] RESP BODY: {"version": {"id": "v2.1", "status": "CURRENT", "version": "2.88", "min_version": "2.1", "updated": "2013-07-23T11:33:21Z", "links": [{"rel": "self", "href": "http://controller-khi04.rapid.pk:8774/v2.1/"}, {"rel": "describedby", "type": "text/html", "href": "http://docs.openstack.org/"}], "media-types": [{"base": "application/json", "type": "application/vnd.openstack.compute+json;version=2.1"}]}} _http_log_response /usr/lib/python3/dist-packages/keystoneauth1/session.py:582 2021-10-08 12:06:14.128 2064 DEBUG novaclient.v2.client [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] GET call to compute for http://controller-khi04.rapid.pk:8774/v2.1/ used request id req-9b1f2443-fdee-41dd-9139-57376afd7bef request /usr/lib/python3/dist-packages/keystoneauth1/session.py:954 2021-10-08 12:06:14.129 2064 INFO heat.engine.resource [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] CREATE: ServerGroup "worker_nodes_server_group" Stack "k8s-cluster6-7cnnuz4hrfrz" [5501d6a4-59a6-4f76-b25e-ec43e0822361] 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource Traceback (most recent call last): 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource File "/usr/lib/python3/dist-packages/heat/engine/resource.py", line 916, in _action_recorder 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource yield 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource File "/usr/lib/python3/dist-packages/heat/engine/resource.py", line 1028, in _do_action 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource yield from self.action_handler_task(action, args=handler_args) 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource File "/usr/lib/python3/dist-packages/heat/engine/resource.py", line 970, in action_handler_task 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource handler_data = handler(*args) 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource File "/usr/lib/python3/dist-packages/heat/engine/resources/openstack/nova/server_group.py", line 98, in handle_create 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource server_group = client.server_groups.create(name=name, 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource File "/usr/lib/python3/dist-packages/novaclient/api_versions.py", line 393, in substitution 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource return methods[-1].func(obj, *args, **kwargs) 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource TypeError: create() got an unexpected keyword argument 'policies' 2021-10-08 12:06:14.129 2064 ERROR heat.engine.resource 2021-10-08 12:06:14.143 2064 INFO heat.engine.stack [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] Stack CREATE FAILED (k8s-cluster6-7cnnuz4hrfrz): Resource CREATE failed: TypeError: resources.worker_nodes_server_group: create() got an unexpected keyword argument 'policies' 2021-10-08 12:06:14.146 2064 DEBUG heat.engine.stack [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] Persisting stack k8s-cluster6-7cnnuz4hrfrz status CREATE FAILED _send_notification_and_add_event /usr/lib/python3/dist-packages/heat/engine/stack.py:1109 2021-10-08 12:06:15.009 2061 INFO heat.engine.stack [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] Triggering resource 2434 for update 2021-10-08 12:06:15.040 2063 DEBUG heat.engine.worker [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] [88ec8f6f-7e10-4c6a-be97-03de82938eb2] Traversal cancelled; re-trigerring. check_resource /usr/lib/python3/dist-packages/heat/engine/worker.py:193 2021-10-08 12:06:16.016 2061 INFO heat.engine.stack [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] Triggering resource 2437 for update 2021-10-08 12:06:16.048 2061 DEBUG heat.engine.worker [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] [88ec8f6f-7e10-4c6a-be97-03de82938eb2] Traversal cancelled; re-trigerring. check_resource /usr/lib/python3/dist-packages/heat/engine/worker.py:193 2021-10-08 12:06:17.026 2061 INFO heat.engine.stack [req-31d11ee8-fc47-4399-b424-97adabf31331 admin admin - - -] Triggering resource 2443 for update 2021-10-08 12:06:17.066 2062 DEBUG heat.engine.worker [req-61cb3eba-0cf0-47f7-8fdb-8e9375888dc4 - - - - -] [88ec8f6f-7e10-4c6a-be97-03de82938eb2] Traversal cancelled; re-trigerring. check_resource /usr/lib/python3/dist-packages/heat/engine/worker.py:193 - Ammad

3 2

[neutron] East-West networking issue on DVR after failed attempt at starting a new instance
by J-P Methot 07 Oct '21

07 Oct '21

Hi, We use Openstack Wallaby installed through Kolla-ansible on this setup. Here's a quick rundown of the issue we just noticed: -We try popping an instance which fails because of a storage issue. -Nova tries to create the instance on 3 different nodes before failing. -We notice that instances on these 3 nodes and only those instances cannot connect to each other anymore. -Doing Tcpdump tests, we realize that pings are received by each instance, but never replied to. -Restarting the neutron-openvswitch-agent container fixes this issue. I suspect l2population might have something to do with this. Is the ARP table rebuilt when the openvswitch-agent is restarted? -- Jean-Philippe Méthot Senior Openstack system administrator Administrateur système Openstack sénior PlanetHoster inc.

2 1

[neutron] Drivers meeting agenda - 08.10.2021
by Lajos Katona 07 Oct '21

07 Oct '21

Hi Neutrinos, The agenda for tomorrow's drivers meeting is at [1]. We have 1 RFE to discuss: * https://bugs.launchpad.net/neutron/+bug/1946251 API: allow to disable anti-spoofing but not SGs [1] https://wiki.openstack.org/wiki/Meetings/NeutronDrivers#Agenda See you at the meeting tomorrow. Lajos Katona (lajoskatona)

1 0

RDO vSocial during the PTG
by Amy Marrich 07 Oct '21

07 Oct '21

Hi Everyone, I'm pleased to announce that RDO will be sponsoring a virtual social during the PTG, Thursday at 17:00 during the break. Last PTG's Trivia Social was a great success, but to do something different this time around we will be doing a virtual Escape Room. The room is a mixture of text and images so should be bandwidth friendly and we'll use Meetpad for the breakout rooms. We'll be doing an Intermediate level room <https://trappedintheweb.com/escape-rooms/cabin-fever>and the team that finishes first will receive prizes! Because I need to purchase passes you will need to register in advance at: https://eventyay.com/e/e7299da7 There is a team signup page you'll receive after registering where you can add your name to a team of 5-8 people. While we can definitely have more people participating in the teams then passes, the intent is to allow everyone to actively participate. Thanks, Amy (spotz)

1 0

Neutron VPNaaS - how to change driver from OpenSwan to StrongSwan? [kolla-ansible][neutron]
by Adam Tomas 06 Oct '21

06 Oct '21

Hi everyone, because I still have problem with growing memory consumption when vpnaas extension is enabled (https://bugs.launchpad.net/neutron/+bug/1940071) I’m trying to test another solutions. And now - while openstack uses strongswan (and this driver is described in documentation) as vpnaas driver, kolla-ansible (v11, victoria) uses openswan… So is there any way to force kolla-ansible to use strongswan? Best regards Adam Tomaś

1 0

[all][tc] Technical Committee next weekly meeting on Oct 7th at 1500 UTC
by Ghanshyam Mann 06 Oct '21

06 Oct '21

Hello Everyone, Technical Committee's next weekly meeting is scheduled for Oct 7th at 1500 UTC. If you would like to add topics for discussion, please add them to the below wiki page by Wednesday, Oct 6th, at 2100 UTC. https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting -gmann

1 1

[Swift][Interop] PTG time
by Arkady Kanevsky 06 Oct '21

06 Oct '21

Swift team, Interop team would like time on Yoga PTG Monday or Tu between 21-22 UTC to discuss Interop guideline coverage for Swift. Thanks, Arkady -- Arkady Kanevsky, Ph.D. Phone: 972 707-6456 Corporate Phone: 919 729-5744 ext. 8176456

3 5

[install] Install on OVH dedicated servers
by P. P. 06 Oct '21

06 Oct '21

Hello all, I know that OVH uses Openstack to offer their public cloud services. I would like to know if someone was able to use their dedicated servers to build a private cloud based on Openstack. Do you think OVH dedicated server hardware + Vrack can provide sufficient requirements for a production environment? Thank you. P.

2 3

[masakari] Yoga PTG
by Radosław Piliszek 06 Oct '21

06 Oct '21

Hello all, This is a reminder that Masakari Yoga PTG is to happen Tuesday October 19, 2021 06:00 - 08:00 (UTC). Please add your name and discussion topic proposals to the etherpad. [1]. Thank you in advance and see you soon! [1] https://etherpad.opendev.org/p/masakari-yoga-ptg -yoctozepto

1 0

OpenStack Xena is officially released!
by Herve Beraud 06 Oct '21

06 Oct '21

The official OpenStack Xena release announcement has been sent out: http://lists.openstack.org/pipermail/openstack-announce/2021-October/002056… Thanks to all who were a part of the Xena development cycle! This marks the official opening of the releases repo for Yoga, and freezes are now lifted. Xena is now a fully normal stable branch, and the normal stable policy now applies. Thanks! Hervé Beraud and the Release Management team -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

3 2

API used for update Image in Glance
by Adivya Singh 06 Oct '21

06 Oct '21

Hi Team, Can you please list me , what i have to do different while updating Image in Glance using a API call, Can Some body share the Syntax for the same, Do i need to create a JSON file for the same Regards Adivya Singh

3 2

[cinder] Bug deputy report for week of 10-06-2021
by Sofia Enriquez 06 Oct '21

06 Oct '21

This is a bug report from 09-29-2021-15-09 to 10-06-2021. Agenda: https://etherpad.opendev.org/p/cinder-bug-squad-meeting ----------------------------------------------------------------------------------------- High - https://bugs.launchpad.net/cinder/+bug/1946059 "NFS: revert to snapshot not working". Assigned to Rajat Dhasmana. Medium - https://bugs.launchpad.net/cinder/+bug/1945824 "[Pure Storage] Clone CG from CG snapshot fails in PowerVC". Assigned to Simon Dodsle. - https://bugs.launchpad.net/cinder/+bug/1945571 "C-bak configure more than one worker issue". Unassigned. Low - https://bugs.launchpad.net/cinder/+bug/1946167 "ddt version incompatibility for victoria branch ". Unassigned. Incomplete - https://bugs.launchpad.net/cinder/+bug/1945500 "[stable/wallaby] filter reserved image properties". Unassigned. Cheers, -- Sofía Enriquez she/her Software Engineer Red Hat PnT <https://www.redhat.com> IRC: @enriquetaso @RedHat <https://twitter.com/redhat> Red Hat <https://www.linkedin.com/company/red-hat> Red Hat <https://www.facebook.com/RedHatInc> <https://www.redhat.com>

1 0

[monasca][Release-job-failures] Release of openstack/monasca-agent for ref refs/tags/6.0.0 failed
by Herve Beraud 06 Oct '21

06 Oct '21

Hello Monasca team, Please have a look at the job error below. Indeed your publish-monasca-agent-docker-images fail with a cryptography error [1]. Indeed cryptography has been recently upgraded from version 3.4.8 to the version 35.0.0 [2]. This could explain the reason why this job fails to build rust. We think that your used docker image needs some updating to solve this issue. For more details about the experienced issue please have a look at the jobs links below (the forwarded email). You should also note that the same problem appears with monasca-notification. Thank you for reading. [1] ``` 2021-10-06 11:22:30.035239 | ubuntu-focal | writing manifest file 'src/cryptography.egg-info/SOURCES.txt' 2021-10-06 11:22:30.035260 | ubuntu-focal | copying src/cryptography/py.typed -> build/lib.linux-x86_64-3.6/cryptography 2021-10-06 11:22:30.035282 | ubuntu-focal | creating build/lib.linux-x86_64-3.6/cryptography/hazmat/bindings/_rust 2021-10-06 11:22:30.035303 | ubuntu-focal | copying src/cryptography/hazmat/bindings/_rust/__init__.pyi -> build/lib.linux-x86_64-3.6/cryptography/hazmat/bindings/_rust 2021-10-06 11:22:30.035325 | ubuntu-focal | copying src/cryptography/hazmat/bindings/_rust/asn1.pyi -> build/lib.linux-x86_64-3.6/cryptography/hazmat/bindings/_rust 2021-10-06 11:22:30.035358 | ubuntu-focal | copying src/cryptography/hazmat/bindings/_rust/ocsp.pyi -> build/lib.linux-x86_64-3.6/cryptography/hazmat/bindings/_rust 2021-10-06 11:22:30.035381 | ubuntu-focal | copying src/cryptography/hazmat/bindings/_rust/x509.pyi -> build/lib.linux-x86_64-3.6/cryptography/hazmat/bindings/_rust 2021-10-06 11:22:30.035422 | ubuntu-focal | running build_ext 2021-10-06 11:22:30.035446 | ubuntu-focal | generating cffi module 'build/temp.linux-x86_64-3.6/_openssl.c' 2021-10-06 11:22:30.035467 | ubuntu-focal | creating build/temp.linux-x86_64-3.6 2021-10-06 11:22:30.035489 | ubuntu-focal | running build_rust 2021-10-06 11:22:30.035510 | ubuntu-focal | 2021-10-06 11:22:30.035532 | ubuntu-focal | =============================DEBUG ASSISTANCE============================= 2021-10-06 11:22:30.035553 | ubuntu-focal | If you are seeing a compilation error please try the following steps to 2021-10-06 11:22:30.035575 | ubuntu-focal | successfully install cryptography: 2021-10-06 11:22:30.035596 | ubuntu-focal | 1) Upgrade to the latest pip and try again. This will fix errors for most 2021-10-06 11:22:30.035618 | ubuntu-focal | users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip 2021-10-06 11:22:30.035639 | ubuntu-focal | 2) Read https://cryptography.io/en/latest/installation/ for specific 2021-10-06 11:22:30.035660 | ubuntu-focal | instructions for your platform. 2021-10-06 11:22:30.035682 | ubuntu-focal | 3) Check our frequently asked questions for more information: 2021-10-06 11:22:30.035703 | ubuntu-focal | https://cryptography.io/en/latest/faq/ 2021-10-06 11:22:30.035724 | ubuntu-focal | 4) Ensure you have a recent Rust toolchain installed: 2021-10-06 11:22:30.035763 | ubuntu-focal | https://cryptography.io/en/latest/installation/#rust 2021-10-06 11:22:30.035786 | ubuntu-focal | 2021-10-06 11:22:30.035807 | ubuntu-focal | Python: 3.6.8 2021-10-06 11:22:30.035828 | ubuntu-focal | platform: Linux-5.4.0-88-generic-x86_64-with 2021-10-06 11:22:30.035850 | ubuntu-focal | pip: n/a 2021-10-06 11:22:30.035871 | ubuntu-focal | setuptools: 58.2.0 2021-10-06 11:22:30.035893 | ubuntu-focal | setuptools_rust: 0.12.1 2021-10-06 11:22:30.035914 | ubuntu-focal | =============================DEBUG ASSISTANCE============================= 2021-10-06 11:22:30.035936 | ubuntu-focal | 2021-10-06 11:22:30.035959 | ubuntu-focal | error: can't find Rust compiler 2021-10-06 11:22:30.035981 | ubuntu-focal | 2021-10-06 11:22:30.036009 | ubuntu-focal | If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler. 2021-10-06 11:22:30.036040 | ubuntu-focal | 2021-10-06 11:22:30.036061 | ubuntu-focal | To update pip, run: 2021-10-06 11:22:30.036082 | ubuntu-focal | 2021-10-06 11:22:30.036103 | ubuntu-focal | pip install --upgrade pip 2021-10-06 11:22:30.036124 | ubuntu-focal | 2021-10-06 11:22:30.036145 | ubuntu-focal | and then retry package installation. 2021-10-06 11:22:30.036166 | ubuntu-focal | 2021-10-06 11:22:30.036206 | ubuntu-focal | If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain. 2021-10-06 11:22:30.036237 | ubuntu-focal | 2021-10-06 11:22:30.036259 | ubuntu-focal | This package requires Rust >=1.41.0. 2021-10-06 11:22:30.036280 | ubuntu-focal | ---------------------------------------- 2021-10-06 11:22:30.036305 | ubuntu-focal | [0m [91m ERROR: Failed building wheel for cryptography ``` [2] https://opendev.org/openstack/requirements/commit/1fa22ce584ef8a5f5ec0c0e60… ---------- Forwarded message --------- De : <zuul(a)openstack.org> Date: mer. 6 oct. 2021 à 13:37 Subject: [Release-job-failures] Release of openstack/monasca-agent for ref refs/tags/6.0.0 failed To: <release-job-failures(a)lists.openstack.org> Build failed. - openstack-upload-github-mirror https://zuul.opendev.org/t/openstack/build/1bb52ee7d5e74741b8b5f180ba48061f : SUCCESS in 1m 32s - release-openstack-python https://zuul.opendev.org/t/openstack/build/ecd6b609a8f1440f98f521d018aae2bb : SUCCESS in 5m 39s - announce-release https://zuul.opendev.org/t/openstack/build/6022c2ed55bb411498b5359ed606a3a1 : SUCCESS in 7m 43s - propose-update-constraints https://zuul.opendev.org/t/openstack/build/0f767adb04294e05ab2f54175c38c12e : SUCCESS in 7m 46s - publish-monasca-agent-docker-images https://zuul.opendev.org/t/openstack/build/09e6e50adb8649e8a61083e8fa0cc602 : POST_FAILURE in 13m 19s _______________________________________________ Release-job-failures mailing list Release-job-failures(a)lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/release-job-failures -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/ https://twitter.com/4383hberaud

2 1

[masakari] Proposal to cancel weekly meetings
by Radosław Piliszek 06 Oct '21

06 Oct '21

Dears, Due to low attendance and the current schedule being uncomfortable for me, I propose to cancel the weekly meetings and suggest we coordinate via this mailing list and do ad-hoc chats on IRC as I'm simply lurking there most of the time and answering the messages. Kind regards, -yoctozepto

1 1

[cyborg] No meeting on 06th October
by Brin Zhang(张百林) 06 Oct '21

06 Oct '21

Hi All, All Cyborg Core contributor are in holiday, so we will cancel this weekly meeting (October 6th). According to schedule will meeting directly on October 13th Thanks. brinzhang Inspur Electronic Information Industry Co.,Ltd.

1 0

[glance] Yoga PTG schedule
by Abhishek Kekane 05 Oct '21

05 Oct '21

Hello All, Greetings!!! Yoga PTG is around the corner and if you haven't already registered, please do so as soon as possible [1]. I have created an etherpad [2] and also added day wise topics along with timings we are going to discuss. Kindly let me know if you have any concerns with allotted time slots. We also have some slots open on Tuesday, Wednesday and Thursday for unplanned discussions. So please feel free to add your topics if you still haven't added yet. As a reminder, these are the time slots for our discussion. Tuesday 19 October 2021 1400 UTC to 1700 UTC Wednesday 20 October 2021 1400 UTC to 1700 UTC Thursday 21 October 2021 1400 UTC to 1700 UTC Friday 22 October 2021 1400 UTC to 1700 UTC NOTE: At the moment we don't have any sessions scheduled on Friday, if there are any last moment request(s)/topic(s) we will discuss them on Friday else we will conclude our PTG on Thursday 21st October. We will be using bluejeans for our discussion, kindly try to use it once before the actual discussion. The meeting URL is mentioned in etherpad [2] and will be the same throughout the PTG. [1] https://www.eventbrite.com/e/project-teams-gathering-october-2021-tickets-1… [2] https://etherpad.opendev.org/p/yoga-glance-ptg Thank you, Abhishek

1 0

[all][ptg] October 2021 Registration & Schedule
by Ashlee Ferguson 05 Oct '21

05 Oct '21

Hi everyone! The October 2021 Project Teams Gathering is right around the corner and the official schedule is live! You can download it here [0], or find it on the PTG website [1]. The PTGbot should be up to date by the end of the week [2] to reflect what is in the ethercalc which is now locked! The PTGbot is the during-event website to keep track of what's being discussed and any last-minute schedule changes. It is driven from the discussion in the #openinfra-events IRC channel where the PTGbot listens. Friendly reminder that the IRC network has changed from freenode to OFTC. Also, please don't forget to register[3] because that's how you'll receive event details, passwords, and other relevant information about the PTG. Please let us know if you have any questions! Thanks! Ashlee & Kendall (diablo_rojo) [0] Schedule https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c0… [1] PTG Website www.openstack.org/ptg [2] PTGbot: https://ptgbot.opendev.org/ [3] PTG Registration: https://openinfra-ptg.eventbrite.com

1 0

[Magnum] Virtual PTG October 2021
by feilong 05 Oct '21

05 Oct '21

Hello team, Our Yoga PTG will be held virtually in October. We have booked Oct 18, at 22:00-00:00 UTC and Oct 20 9:00-11:00UTC [1] for our Yoga PTG. I opened an etherpad [2] to organize the topics of the meetings. Please feel free to add your topics! Thank you. [1] https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c0… <https://ethercalc.openstack.org/8tum5yl1bx43> [2] https://etherpad.opendev.org/p/magnum-ptg-yoga <https://etherpad.opendev.org/p/cloudkitty-ptg-yoga> -- Cheers & Best regards, ------------------------------------------------------------------------------ Feilong Wang (王飞龙) (he/him) Head of Research & Development Catalyst Cloud Aotearoa's own Mob: +64 21 0832 6348 | www.catalystcloud.nz Level 6, 150 Willis Street, Wellington 6011, New Zealand CONFIDENTIALITY NOTICE: This email is intended for the named recipients only. It may contain privileged, confidential or copyright information. If you are not the named recipient, any use, reliance upon, disclosure or copying of this email or its attachments is unauthorised. If you have received this email in error, please reply via email or call +64 21 0832 6348. ------------------------------------------------------------------------------

1 1

[kolla] Neutron-metering is not creating the bandwidth metric
by Uriel Medina 05 Oct '21

05 Oct '21

Hello everyone. I'm having issues with the Neutron-Metering component inside Kolla-Ansible, and I was hoping that you guys could help me :) The problem is that Neutron Metering doesn't create/get the bandwidth metric. I create a report of this inside the Neutron Launchpad, thinking that maybe the Metering component had troubles: https://bugs.launchpad.net/neutron/+bug/1945560 With the help of Bence, we've discovered that the messages of the creation of metering labels and metering rules were OK inside RabbitMQ. After that, I deployed a new DevStack environment and, with the right configuration, the Neutron Metering is working as it should, only inside the DevStack environment. That made me think that maybe Kolla Ansible had a flag to avoid the modification of iptables and I found the flag "docker_disable_default_iptables_rules" which I set to "no", as I'm using the Wallaby version. Setting this flag didn't do the trick and I was thinking that maybe there is another flag or component of Kolla Ansible that prevents the modification of iptables, apart from "docker_disable_default_iptables_rules". Thanks in advance. Greetings!

1 0

[baremetal-sig][ironic] Tue Oct 12, 2021, 2pm & 6pm UTC: Ironic User & Operator Feedback (Session 2)
by Arne Wiebalck 05 Oct '21

05 Oct '21

Dear all, Due to popular demand and since we had to cut things short last month, the Bare Metal SIG has scheduled two meetings next week to continue the user/operator/admin feedback: - Tue Oct 12, 2021, at 2pm UTC (EMEA friendly), and - Tue Oct 12, 2021, at 6pm UTC (AMER friendly) So come along, meet other Ironicers and discuss your Ironic successes, pain points, issues, experiences and ideas with the community and in particular the upstream developers! Everyone, in particular not-yet Ironicers, are welcome to join! All details can be found on: - https://etherpad.opendev.org/p/bare-metal-sig Hope to see you there! Julia & Arne (for the Bare Metal SIG)

1 0

[tripleo][ansible] Openstack Ansible collections (modules) Yoga PTG
by Sagi Shnaidman 05 Oct '21

05 Oct '21

Hi, all Openstack Ansible collection (modules) project has its Yoga PTG session on Wed 20 Oct 13.00-14.00 UTC in Cactus room. Please add topics for discussion in the etherpad: https://etherpad.opendev.org/p/osac-yoga-ptg Thanks and see you in PTG! -- Best regards Sagi Shnaidman

1 0

[nova][placement] Asia friendly meeting slot on 7th of Oct
by Sylvain Bauza 05 Oct '21

05 Oct '21

Hi, This is a reminder that we will hold our monthly Asian-friendly Nova meeting timeslot on next Thursday 8:00 UTC [1]. Feel free to join us in the #openstack-nova IRC channel on the OFTC server [2] so we could discuss some topics like how to help synchronously or asynchronously contributors that are not in the European and American timezones. If you have problems joining us with IRC, please let me know by replying this email. Thanks, -Sylvain [1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=2021-10-07T08:00:… [2] https://docs.openstack.org/contributors/common/irc.html

1 0

[ironic][molteniron][qa] Anyone still using MoltenIron?
by Julia Kreger 05 Oct '21

05 Oct '21

Out of curiosity, is anyone still using MotltenIron? A little historical context: It was originally tooling that came out of IBM to reserve physical nodes in a CI cluster in order to perform testing. It was never intended to be released as it was tooling *purely* for CI job usage. The reason I'm asking is that the ironic team is considering going ahead and retiring the repository. Thanks! -Julia

3 3

[kolla] skipping base if already exist
by Sam Morrison 04 Oct '21

04 Oct '21

Hi, We’ve started to use kolla to build container images and trying to figure out if I’m doing it wrong it it’s just not how kolla works. What I’m trying to do it not rebuild the base and openstack-base images when we build an image for a project. Example. We build a horizon image and it builds and pushes up to our registry the following kolla/ubuntu-source-base <> kolla/ubuntu-source-openstack-base kolla/ubuntu-source-horizon Now I can rebuild this without having to again build the base images with —skip parents But now I want to build a barbican image and I can’t use skip-parents as the barbican image also requires barbican-base. Which means I need to go and rebuild the ubuntu base and Openstack base images again. Is there a way to essentially skip parents but only if they don’t exist in the registry already? Or make skip-parents only mean skip base and Openstack-base? Thanks in advance, Sam

2 2