July 2022 - openstack-discuss - lists.openstack.org

[nova] nova hypervisor oom killed some openstack guest
by hai wu 25 Jul '22

25 Jul '22

nova hypervisor sometimes oom would kill some openstack guests. Is it possible to not allow kernel to oom kill any openstack guests? ram is not oversubscribed much ..

5 11

[IRONIC] - Various questions around network features.
by Gaël THEROND 25 Jul '22

25 Jul '22

I everyone, I’m currently working back again with Ironic and it’s amazing! However, during our demo session to our users few questions arise. We’re currently deploying nodes using a private vlan that can’t be reached from outside of the Openstack network fabric (vlan 101 - 192.168.101.0/24) and everything is fine with this provisioning network as our ToR switch all know about it and other Control plan VLANs such as the internal APIs VLAN which allow the IPA Ramdisk to correctly and seamlessly be able to contact the internal IRONIC APIs. (When you declare a port as a trunk allowed all vlan on a aruba switch it seems it automatically analyse the CIDR your host try to reach from your VLAN and route everything to the corresponding VLAN that match the destination IP). So know, I still get few tiny issues: 1°/- When I spawn a nova instance on a ironic host that is set to use flat network (From horizon as a user), why does the nova wizard still ask for a neutron network if it’s not set on the provisioned host by the IPA ramdisk right after the whole disk image copy? Is that some missing development on horizon or did I missed something? 2°/- In a flat network layout deployment using direct deploy scenario for images, am I still supposed to create a ironic provisioning network in neutron? >From my understanding (and actually my tests) we don’t, as any host booting on the provisioning vlan will catch up an IP and initiate the bootp sequence as the dnsmasq is just set to do that and provide the IPA ramdisk, but it’s a bit confusing as many documentation explicitly require for this network to exist on neutron. 3°/- My whole Openstack network setup is using Openvswitch and vxlan tunnels on top of a spine/leaf architecture using aruba CX8360 switches (for both spine and leafs), am I required to use either the networking-generic-switch driver or a vendor neutron driver ? If that’s right, how will this driver be able to instruct the switch to assign the host port the correct openvswitch vlan id and register the correct vxlan to openvswitch from this port? I mean, ok neutron know the vxlan and openvswitch the tunnel vlan id/interface but what is the glue of all that? 4°/- I’ve successfully used openstack cloud oriented CentOS and debian images or snapshot of VMs to provision my hosts, this is an awesome feature, but I’m wondering if there is a way to let those host cloud-init instance to request for neutron metadata endpoint? I was a bit surprised about the ironic networking part as I was expecting the IPA ramdisk to at least be able to set the host os with the appropriate network configuration file for whole disk images that do not use encryption by injecting those information from the neutron api into the host disk while mounted (right after the image dd). All in all I really like the ironic approach of the baremetal provisioning process, and I’m pretty sure that I’m just missing a bit of understanding of the networking part but it’s really the most confusing part of it to me as I feel like if there is a missing link in between neutron and the host HW or the switches. Thanks a lot anyone that will take time to explain me this :-)

2 4

[neutron] Bug Deputy Report Jul 18 - Jul 24
by Oleg Bondarev 25 Jul '22

25 Jul '22

Hello Neutron Team, Bug report for the week of Jul 18 is below: Critical: ---------- - https://bugs.launchpad.net/neutron/+bug/1981963 - Some jobs broken post pyroute2 update to 0.7.1 - In progress - https://review.opendev.org/c/openstack/requirements/+/850295 is merged - https://bugs.launchpad.net/neutron/+bug/1982206 - stable: Neutron unit tests timeout on stable/ussuri and stable/victoria (perhaps wallaby also) - Fix released - https://review.opendev.org/c/openstack/neutron/+/850670 High: ------- - https://bugs.launchpad.net/neutron/+bug/1982367 - Race condition between DHCP and L2 agent during evacuate vms may cause VM ends up in ERROR state - In Progress - Assigned to Slawek Medium: ------------ - https://bugs.launchpad.net/neutron/+bug/1982569 - [OVN] metadata does not work when using neutron-dhcp-agent for baremetal ports - Confirmed - Unasigned - https://bugs.launchpad.net/neutron/+bug/1982110 - [taap-as-a-service] Project requires "webtest" library for testing - Fix committed - https://review.opendev.org/c/openstack/neutron-vpnaas/+/850335 RFEs: --------- - https://bugs.launchpad.net/neutron/+bug/1982287 - [rfe][ovn] Support address group for ovn driver - Approved on drivers meeting - Assigned to liushy - https://bugs.launchpad.net/neutron/+bug/1982541 - [RFE] OVN E/W routing for external (baremetal) VLAN ports - New - Assigned to Michal Nasiadka Undecided: --------------- - https://bugs.launchpad.net/neutron/+bug/1982373 - nova/neutron ignore and overwrite custom device owner fields - Opinion - most probably it's in Nova scope Thanks, Oleg

1 0

[Openstack] sdn multisite???
by Ignazio Cassano 23 Jul '22

23 Jul '22

Hello All, is there any sdn solution for openstack multi site sdn ? I read something about network to network api extension for neutron presented in an openstack summit but it seems no more adopted. Any help, please? We have 3 separated openstack installations and I wonder if there is some solution for controlling network (vxlan to vxlan connection, sec groups airchestration and so on) Openstack cascading seems only a dream. Ignazio

2 2

ovn-bgp-agent installation issue
by Satish Patel 22 Jul '22

22 Jul '22

Folks, I am trying to create lab of of ovn-bgp-agent using this blog https://ltomasbo.wordpress.com/2021/02/04/ovn-bgp-agent-testing-setup/ So far everything went well but I'm stuck at the bgp-agent installation and I encounter following error when running bgp-agent. Any suggestions? root@rack-1-host-2:/home/vagrant/bgp-agent# bgp-agent 2022-07-22 04:02:39.123 111551 INFO bgp_agent.config [-] Logging enabled! 2022-07-22 04:02:39.475 111551 CRITICAL bgp-agent [-] Unhandled error: AssertionError 2022-07-22 04:02:39.475 111551 ERROR bgp-agent Traceback (most recent call last): 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/bin/bgp-agent", line 10, in <module> 2022-07-22 04:02:39.475 111551 ERROR bgp-agent sys.exit(start()) 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/agent.py", line 76, in start 2022-07-22 04:02:39.475 111551 ERROR bgp-agent bgp_agent_launcher = service.launch(config.CONF, BGPAgent()) 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/agent.py", line 44, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent self.agent_driver = driver_api.AgentDriverBase.get_instance( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/driver_api.py", line 25, in get_instance 2022-07-22 04:02:39.475 111551 ERROR bgp-agent agent_driver = stevedore_driver.DriverManager( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/driver.py", line 54, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(DriverManager, self).__init__( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/named.py", line 78, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent extensions = self._load_plugins(invoke_on_load, 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/extension.py", line 221, in _load_plugins 2022-07-22 04:02:39.475 111551 ERROR bgp-agent ext = self._load_one_plugin(ep, 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/named.py", line 156, in _load_one_plugin 2022-07-22 04:02:39.475 111551 ERROR bgp-agent return super(NamedExtensionManager, self)._load_one_plugin( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/extension.py", line 257, in _load_one_plugin 2022-07-22 04:02:39.475 111551 ERROR bgp-agent obj = plugin(*invoke_args, **invoke_kwds) 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/ovn_bgp_driver.py", line 64, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent self._sb_idl = ovn.OvnSbIdl( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/utils/ovn.py", line 62, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(OvnSbIdl, self).__init__( 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/utils/ovn.py", line 31, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(OvnIdl, self).__init__(remote, schema) 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 283, in __init__ 2022-07-22 04:02:39.475 111551 ERROR bgp-agent schema = schema_helper.get_idl_schema() 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 2323, in get_idl_schema 2022-07-22 04:02:39.475 111551 ERROR bgp-agent self._keep_table_columns(schema, table, columns)) 2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 2330, in _keep_table_columns 2022-07-22 04:02:39.475 111551 ERROR bgp-agent assert table_name in schema.tables 2022-07-22 04:02:39.475 111551 ERROR bgp-agent AssertionError 2022-07-22 04:02:39.475 111551 ERROR bgp-agent After googling I found one more agent at https://opendev.org/x/ovn-bgp-agent and its also throwing an error. Which agent should I be using? root@rack-1-host-2:~# ovn-bgp-agent 2022-07-22 04:04:36.780 111761 INFO ovn_bgp_agent.config [-] Logging enabled! 2022-07-22 04:04:37.247 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopped 2022-07-22 04:04:37.248 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' starting 2022-07-22 04:04:37.248 111761 INFO ovn_bgp_agent.drivers.openstack.utils.frr [-] Add VRF leak for VRF ovn-bgp-vrf on router bgp 64999 2022-07-22 04:04:37.248 111761 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'privsep-helper', '--privsep_context', 'ovn_bgp_agent.privileged.vtysh_cmd', '--privsep_sock_path', '/tmp/tmp4cie9eiz/privsep.sock'] 2022-07-22 04:04:37.687 111761 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap 2022-07-22 04:04:37.598 111769 INFO oslo.privsep.daemon [-] privsep daemon starting 2022-07-22 04:04:37.613 111769 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0 2022-07-22 04:04:37.617 111769 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_NET_ADMIN|CAP_SYS_ADMIN/none 2022-07-22 04:04:37.617 111769 INFO oslo.privsep.daemon [-] privsep daemon running as pid 111769 2022-07-22 04:04:37.987 111769 ERROR ovn_bgp_agent.privileged.vtysh [-] Unable to execute vtysh with ['/usr/bin/vtysh', '--vty_socket', '/run/frr/', '-c', 'copy /tmp/tmpiz5s_wvs running-config']. Exception: Unexpected error while running command. Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config Exit code: 1 Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n' Stderr: '' Traceback (most recent call last): File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/privileged/vtysh.py", line 30, in run_vtysh_config return processutils.execute(*full_args) File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 438, in execute raise ProcessExecutionError(exit_code=_returncode, oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config Exit code: 1 Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n' Stderr: '' 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service [-] Error starting thread.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config Exit code: 1 Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n' Stderr: '' 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Traceback (most recent call last): 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_service/service.py", line 806, in run_service 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service service.start() 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/agent.py", line 50, in start 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service self.agent_driver.start() 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/ovn_bgp_driver.py", line 73, in start 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service frr.vrf_leak(constants.OVN_BGP_VRF, CONF.bgp_AS, CONF.bgp_router_id) 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/utils/frr.py", line 110, in vrf_leak 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service _run_vtysh_config_with_tempfile(vrf_config) 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/utils/frr.py", line 93, in _run_vtysh_config_with_tempfile 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service ovn_bgp_agent.privileged.vtysh.run_vtysh_config(f.name) 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 271, in _wrap 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service return self.channel.remote_call(name, args, kwargs, 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 215, in remote_call 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service raise exc_type(*result[2]) 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command. 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Exit code: 1 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n' 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Stderr: '' 2022-07-22 04:04:37.990 111761 ERROR oslo_service.service 2022-07-22 04:04:37.993 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopping 2022-07-22 04:04:37.994 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopped

2 3

Regarding Designate as a DNS in Open Stack Xena
by Adivya Singh 22 Jul '22

22 Jul '22

hi Team, I have a use case where Customer want to map the Full qualified domain name to the FLoating IP to connect to the Server. Besides creating a designation lxc Container using ansible and Create Zones, Do we need to do anything else from our side. I mean how the records are manager using designate Regards Adivya Singh

2 1

[dev][designate][dns] Adding private DNS feature
by Sergey Drozdov 22 Jul '22

22 Jul '22

Dear Sir/Madam, We are running OpenStack at scale and now have a requirement to have private DNS and were wondering if the designate team have any appetite for this? If yes, then further discussion is warranted as we would be happy to get the ball rolling on this. Best Regards, Sergey Drozdov Software Engineer The Hut Group

3 3

Mentors Needed - Grace Hopper Open Source Day + OpenStack
by Kendall Nelson 22 Jul '22

22 Jul '22

Hello Everyone! We are again signed up to participate in Open Source Day at the Grace Hopper Conference. It's a virtual event, being held on Friday, September 16, 2022, from 8am to 3pm Pacific Time. If you are interested in mentoring for this one day event, please let me know ASAP. I am supposed to give them a list of mentors by the end of this week. Day of, we will essentially get participants to setup a dev environment (gerrit, etc) and work on a bug and get it pushed. At this point I was thinking of making use of gaps in the SDK/OSC, but if your project has some low hanging fruit that you want to bring along, that works too! Looking forward to working with you!! -Kendall (diablo_rojo)

6 9

[all][TC] Bare rechecks
by Slawek Kaplonski 22 Jul '22

22 Jul '22

Hi, New stats from last 7 days about bare rechecks in each team are available in [1]: +--------------------+---------------+--------------+-------------------+ | Team | Bare rechecks | All Rechecks | Bare rechecks [%] | +--------------------+---------------+--------------+-------------------+ | skyline | 2 | 2 | 100.0 | | sahara | 2 | 2 | 100.0 | | trove | 4 | 4 | 100.0 | | tacker | 6 | 6 | 100.0 | | horizon | 25 | 25 | 100.0 | | magnum | 2 | 2 | 100.0 | | masakari | 2 | 2 | 100.0 | | Telemetry | 1 | 1 | 100.0 | | kolla | 71 | 74 | 95.95 | | OpenStack Charms | 16 | 17 | 94.12 | | requirements | 33 | 37 | 89.19 | | cinder | 137 | 163 | 84.05 | | kuryr | 5 | 6 | 83.33 | | OpenStack-Helm | 23 | 29 | 79.31 | | tripleo | 97 | 125 | 77.6 | | glance | 17 | 22 | 77.27 | | Puppet OpenStack | 23 | 30 | 76.67 | | ironic | 43 | 57 | 75.44 | | keystone | 6 | 8 | 75.0 | | octavia | 32 | 44 | 72.73 | | swift | 8 | 11 | 72.73 | | manila | 27 | 38 | 71.05 | | OpenStackSDK | 17 | 24 | 70.83 | | oslo | 12 | 17 | 70.59 | | ec2-api | 2 | 3 | 66.67 | | Quality Assurance | 13 | 20 | 65.0 | | neutron | 33 | 52 | 63.46 | | nova | 84 | 133 | 63.16 | | heat | 3 | 5 | 60.0 | | Release Management | 1 | 2 | 50.0 | | designate | 4 | 8 | 50.0 | | barbican | 2 | 12 | 16.67 | | OpenStackAnsible | 0 | 1 | 0.0 | +--------------------+---------------+--------------+-------------------+ [1] https://etherpad.opendev.org/p/recheck-weekly-summary[1] -- Slawek Kaplonski Principal Software Engineer Red Hat

3 3

[dev] directions to the right project team
by Sergey Drozdov 22 Jul '22

22 Jul '22

Too whom it may concern, I am relatively new to contributing to OpenStack and therefore am not sure who to direct this question too. I have the following issue. The firm I am currently working at is running OpenStack at sale with circa 6000 different projects. Whenever we try to access a projects tab via horizon we experience a timeout (accessing through the API works fine). To that end we were hoping to implement something akin to pagination or filtering. I was hoping to work on this and contribute the outcome to OpenStack. Is this something worthwhile and which group should I talk to; keystone, horizon or both ? I apologise in advance if I have missed something. Best Regards, Sergey

6 8