Hello Shrey:

First of all, let me say that the ML2 Linux Bridge mechanism driver is now considered as "experimental support". That means we no longer have active developers working on this driver and we always recommend using others like ML2/OVS or ML2/OVN (or ML2/SR-IOV in case you have the needed hardware).

Let me also point you to launchpad [1] that is the place to report a defect like this one. Please open a bug in this link.

In order to debug and try to reproduce this issue, can you please print the values you are passing in [2] (the name, the namespace name and kwargs)?

Thanks!

[1]https://bugs.launchpad.net/neutron/
[2]https://github.com/openstack/neutron/blob/85b82d4452ed3199c7f1f7c2455d2a75faaa2991/neutron/agent/linux/ip_lib.py#L321


On Thu, Mar 16, 2023 at 2:19 PM Shrey Chauhan <shr.chauhan@gmail.com> wrote:

Hi,

I am running an openstack xena environment

 

My neutron version

dnf list installed | grep neutron


openstack-neutron.noarch 1:19.4.0-2.el8 @ecnlocalrepo
openstack-neutron-common.noarch 1:19.4.0-2.el8 @ecnlocalrepo
openstack-neutron-linuxbridge.noarch 1:19.4.0-2.el8 @ecnlocalrepo
openstack-neutron-ml2.noarch 1:19.4.0-2.el8 @ecnlocalrepo
python3-neutron.noarch 1:19.4.0-2.el8 @ecnlocalrepo
python3-neutron-lib.noarch 2.15.2-1.el8 @ecnlocalrepo
python3-neutronclient.noarch 7.6.0-1.el8 @ecnlocalrepo


I observed when I create a vm, sometimes the vm was not getting the right dhcp ip which was getting assigned from openstack:
I looked inside the vm dhcp call was just timing out, I am still not able to figure out what is wrong with the setup.
One thing that we have noticed the Linux bridge logs and are just filled with these errors, the whole log file is just filled with these errors:
2023-03-16 07:38:29.182 118098 INFO neutron.plugins.ml2.drivers.agent._common_agent [req-3e756b88-6d40-4aac-b161-4ec8dce4d1c9 - - - - -] Port tapc0ef9dda-42 updated. Details: {'device': 'tapc0ef9dda-42', 'network_id': 'e67e45d1-cf29-416b-89ef-353db4ef3586', 'port_id': 'c0ef9dda-4278-4e97-a014-09bc125bb57d', 'mac_address': 'fa:16:3e:32:f4:94', 'admin_state_up': True, 'network_type': 'vxlan', 'segmentation_id': 1908, 'physical_network': None, 'mtu': 1450, 'fixed_ips': [{'subnet_id': '67d6e448-0976-4fc8-a54e-7df40d0a438d', 'ip_address': '169.254.195.108'}], 'device_owner': 'network:router_ha_interface', 'allowed_address_pairs': [], 'port_security_enabled': False, 'qos_policy_id': None, 'network_qos_policy_id': None, 'profile': {}, 'propagate_uplink_status': False}

2023-03-16 07:38:29.191 118098 INFO neutron.plugins.ml2.drivers.linuxbridge.agent.arp_protect [req-3e756b88-6d40-4aac-b161-4ec8dce4d1c9 - - - - -] Skipping ARP spoofing rules for port 'tapc0ef9dda-42' because it has port security disabled

2023-03-16 07:38:29.239 118393 ERROR pr2modules.netlink [-]   File "/usr/lib64/python3.6/threading.py", line 905, in _bootstrap

    self._bootstrap_inner()

  File "/usr/lib64/python3.6/threading.py", line 937, in _bootstrap_inner

    self.run()

  File "/usr/lib64/python3.6/threading.py", line 885, in run

    self._target(*self._args, **self._kwargs)

  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 69, in _worker

    work_item.run()

  File "/usr/lib64/python3.6/concurrent/futures/thread.py", line 56, in run

    result = self.fn(*self.args, **self.kwargs)

  File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 477, in _process_cmd

    ret = func(*f_args, **f_kwargs)

  File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 274, in _wrap

    return func(*args, **kwargs)

  File "/usr/lib/python3.6/site-packages/neutron/privileged/agent/linux/ip_lib.py", line 317, in create_interface

    return ip.link("add", ifname=ifname, kind=kind, **kwargs)

  File "/usr/lib/python3.6/site-packages/pr2modules/iproute/linux.py", line 1461, in link

    msg_flags=msg_flags)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 397, in nlm_request

    return tuple(self._genlm_request(*argv, **kwarg))

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 888, in nlm_request

    self.put(msg, msg_type, msg_flags, msg_seq=msg_seq)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/nlsocket.py", line 636, in put

    self.sendto_gate(msg, addr)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/rtnl/iprsocket.py", line 61, in _gate_linux

    msg.encode()

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/rtnl/ifinfmsg/__init__.py", line 511, in encode

    return super(ifinfbase, self).encode()

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1062, in encode

    offset = self.encode_nlas(offset)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1323, in encode_nlas

    nla_instance.encode()

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1062, in encode

    offset = self.encode_nlas(offset)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1323, in encode_nlas

    nla_instance.encode()

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1062, in encode

    offset = self.encode_nlas(offset)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1323, in encode_nlas

    nla_instance.encode()

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1059, in encode

    offset, diff = self.ft_encode(offset)

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1493, in ft_encode

    log.error(''.join(traceback.format_stack()))

 

2023-03-16 07:38:29.239 118393 ERROR pr2modules.netlink [-] Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/pr2modules/netlink/__init__.py", line 1491, in ft_encode

    struct.pack_into(efmt, self.data, offset, value)

struct.error: required argument is not an integer

 

2023-03-16 07:38:29.239 118393 ERROR pr2modules.netlink [-] error pack: B b'inherit' <class 'bytes'>

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent [req-3e756b88-6d40-4aac-b161-4ec8dce4d1c9 - - - - -] Error in agent loop. Devices info: {'current': {'tapc0ef9dda-42'}, 'timestamps': {'tapc0ef9dda-42': 27}, 'added': {'tapc0ef9dda-42'}, 'removed': set(), 'updated': set()}: struct.error: required argument is not an integer

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent Traceback (most recent call last):

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 465, in daemon_loop

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     sync = self.process_network_devices(device_info)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     result = f(*args, **kwargs)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 214, in process_network_devices

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     resync_a = self.treat_devices_added_updated(devices_added_updated)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/osprofiler/profiler.py", line 160, in wrapper

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     result = f(*args, **kwargs)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 231, in treat_devices_added_updated

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     self._process_device_if_exists(device_details)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/agent/_common_agent.py", line 258, in _process_device_if_exists

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     device, device_details['device_owner'])

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 585, in plug_interface

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     network_segment.mtu)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 520, in add_tap_interface

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return False

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 227, in __exit__

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.force_reraise()

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/oslo_utils/excutils.py", line 200, in force_reraise

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise self.value

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 512, in add_tap_interface

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     tap_device_name, device_owner, mtu)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 545, in _add_tap_interface

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     mtu):

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 484, in ensure_physical_in_bridge

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     return self.ensure_vxlan_bridge(network_id, segmentation_id, mtu)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 259, in ensure_vxlan_bridge

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     interface = self.ensure_vxlan(segmentation_id, mtu)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/plugins/ml2/drivers/linuxbridge/agent/linuxbridge_neutron_agent.py", line 356, in ensure_vxlan

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     self.local_int, **args)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/neutron/agent/linux/ip_lib.py", line 322, in add_vxlan

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     privileged.create_interface(name, self.namespace, "vxlan", **kwargs)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/oslo_privsep/priv_context.py", line 272, in _wrap

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     r_call_timeout)

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent   File "/usr/lib/python3.6/site-packages/oslo_privsep/daemon.py", line 216, in remote_call

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent     raise exc_type(*result[2])

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent struct.error: required argument is not an integer

2023-03-16 07:38:29.242 118098 ERROR neutron.plugins.ml2.drivers.agent._common_agent

2023-03-16 07:38:30.849 118098 INFO neutron.plugins.ml2.drivers.agent._common_agent [req-3e756b88-6d40-4aac-b161-4ec8dce4d1c9 - - - - -] Linux bridge agent Agent out of sync with plugin!

2023-03-16 07:38:30.850 118098 INFO neutron.agent.securitygroups_rpc [req-3e756b88-6d40-4aac-b161-4ec8dce4d1c9 - - - - -] Preparing filters for devices {'tapc0ef9dda-42'}



I have been struggling with these in our environment, any suggestion what could be the reason behind this?
What is wrong in my setup here?
Thanks in advance for any help

 

Sent from Mail for Windows