Open Stack

Folks,

I am getting some strange errors on my kolla based OVN deployment. I have
only 5 nodes so it's not a large deployment. Are there any ovn related
tuning options which I missed ?

I have the following timers configured at present.

ovn-openflow-probe-interval="60"

ovn-remote-probe-interval="60000"


root at ctrl1:~# tail -f /var/log/kolla/openvswitch/ov*.log
==> /var/log/kolla/openvswitch/ovn-controller.log <==
2023-06-09T06:45:47.615Z|00091|lflow_cache|INFO|Detected cache inactivity
(last active 30001 ms ago): trimming cache
2023-06-09T07:11:12.908Z|00092|lflow_cache|INFO|Detected cache inactivity
(last active 30002 ms ago): trimming cache
2023-06-09T07:12:21.225Z|00093|lflow_cache|INFO|Detected cache inactivity
(last active 30001 ms ago): trimming cache
2023-06-10T19:13:10.382Z|00094|lflow_cache|INFO|Detected cache inactivity
(last active 30001 ms ago): trimming cache
2023-06-10T19:17:10.734Z|00095|lflow_cache|INFO|Detected cache inactivity
(last active 30002 ms ago): trimming cache
2023-06-10T19:18:33.270Z|00096|lflow_cache|INFO|Detected cache inactivity
(last active 30004 ms ago): trimming cache
2023-06-10T19:25:23.987Z|00097|lflow_cache|INFO|Detected cache inactivity
(last active 30002 ms ago): trimming cache
2023-06-10T19:32:03.981Z|00098|lflow_cache|INFO|Detected cache inactivity
(last active 30003 ms ago): trimming cache
2023-06-10T19:36:59.153Z|00099|lflow_cache|INFO|Detected cache inactivity
(last active 30006 ms ago): trimming cache
2023-06-10T20:18:34.798Z|00100|lflow_cache|INFO|Detected cache inactivity
(last active 30002 ms ago): trimming cache

==> /var/log/kolla/openvswitch/ovn-nb-db.log <==
2023-06-10T20:52:34.461Z|00518|reconnect|WARN|tcp:192.168.1.11:56798:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.463Z|00519|reconnect|WARN|tcp:192.168.1.13:57048:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.464Z|00520|reconnect|WARN|tcp:192.168.1.11:56792:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.465Z|00521|reconnect|WARN|tcp:192.168.1.13:57016:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.466Z|00522|reconnect|WARN|tcp:192.168.1.11:56746:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.466Z|00523|reconnect|WARN|tcp:192.168.1.11:56742:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.468Z|00524|reconnect|WARN|tcp:192.168.1.11:56786:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.472Z|00525|reconnect|WARN|tcp:192.168.1.11:56784:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.474Z|00526|reconnect|WARN|tcp:192.168.1.13:57044:
connection dropped (Connection reset by peer)
2023-06-10T20:52:34.484Z|00527|reconnect|WARN|tcp:192.168.1.11:56760:
connection dropped (Connection reset by peer)

==> /var/log/kolla/openvswitch/ovn-northd.log <==
2023-06-10T20:52:34.377Z|00695|reconnect|INFO|tcp:192.168.1.12:6641:
connected
2023-06-10T20:52:34.379Z|00696|ovsdb_cs|INFO|tcp:192.168.1.12:6641:
clustered database server is not cluster leader; trying another server
2023-06-10T20:52:34.379Z|00697|ovsdb_cs|INFO|tcp:192.168.1.12:6641:
clustered database server is not cluster leader; trying another server
2023-06-10T20:52:34.379Z|00698|reconnect|INFO|tcp:192.168.1.12:6641:
connection attempt timed out
2023-06-10T20:52:34.380Z|00699|reconnect|INFO|tcp:192.168.1.11:6641:
connecting...
2023-06-10T20:52:34.380Z|00700|reconnect|INFO|tcp:192.168.1.11:6641:
connected
2023-06-10T20:52:34.408Z|00701|ovsdb_cs|INFO|tcp:192.168.1.11:6641:
clustered database server is not cluster leader; trying another server
2023-06-10T20:52:34.408Z|00702|reconnect|INFO|tcp:192.168.1.11:6641:
connection attempt timed out
2023-06-10T20:52:35.409Z|00703|reconnect|INFO|tcp:192.168.1.13:6641:
connecting...
2023-06-10T20:52:35.409Z|00704|reconnect|INFO|tcp:192.168.1.13:6641:
connected

==> /var/log/kolla/openvswitch/ovn-sb-db.log <==
2023-06-10T20:51:27.588Z|00496|raft|INFO|Transferring leadership to write a
snapshot.
2023-06-10T20:51:27.597Z|00497|raft|INFO|rejected append_reply (not leader)
2023-06-10T20:51:27.597Z|00498|raft|INFO|rejected append_reply (not leader)
2023-06-10T20:51:27.597Z|00499|raft|INFO|server d051 is leader for term 3574
2023-06-10T20:51:27.663Z|00500|jsonrpc|WARN|tcp:192.168.1.13:45144: receive
error: Connection reset by peer
2023-06-10T20:51:27.664Z|00501|reconnect|WARN|tcp:192.168.1.13:45144:
connection dropped (Connection reset by peer)
2023-06-10T20:51:27.665Z|00502|jsonrpc|WARN|tcp:192.168.1.12:47858: receive
error: Connection reset by peer
2023-06-10T20:51:27.665Z|00503|reconnect|WARN|tcp:192.168.1.12:47858:
connection dropped (Connection reset by peer)
2023-06-10T20:51:27.667Z|00504|jsonrpc|WARN|tcp:192.168.1.11:41500: receive
error: Connection reset by peer
2023-06-10T20:51:27.667Z|00505|reconnect|WARN|tcp:192.168.1.11:41500:
connection dropped (Connection reset by peer)

==> /var/log/kolla/openvswitch/ovsdb-server.log <==
2023-05-29T02:05:02.891Z|00027|reconnect|WARN|unix#67466: connection
dropped (Connection reset by peer)
2023-05-31T16:20:33.494Z|00028|reconnect|ERR|tcp:127.0.0.1:59928: no
response to inactivity probe after 5 seconds, disconnecting
2023-06-01T20:43:23.516Z|00001|vlog|INFO|opened log file
/var/log/kolla/openvswitch/ovsdb-server.log
2023-06-01T20:43:23.520Z|00002|ovsdb_server|INFO|ovsdb-server (Open
vSwitch) 2.17.3
2023-06-01T20:43:33.522Z|00003|memory|INFO|7216 kB peak resident set size
after 10.0 seconds
2023-06-01T20:43:33.522Z|00004|memory|INFO|atoms:826 cells:770 monitors:5
sessions:3
2023-06-03T07:44:05.774Z|00005|reconnect|ERR|tcp:127.0.0.1:40098: no
response to inactivity probe after 5 seconds, disconnecting
2023-06-03T09:41:49.039Z|00006|reconnect|ERR|tcp:127.0.0.1:60042: no
response to inactivity probe after 5 seconds, disconnecting
2023-06-11T02:05:08.802Z|00007|jsonrpc|WARN|unix#26478: receive error:
Connection reset by peer
2023-06-11T02:05:08.803Z|00008|reconnect|WARN|unix#26478: connection
dropped (Connection reset by peer)

==> /var/log/kolla/openvswitch/ovs-vswitchd.log <==
2023-06-10T19:16:50.733Z|00138|connmgr|INFO|br-int<->unix#3: 4 flow_mods in
the 2 s starting 10 s ago (4 adds)
2023-06-10T19:18:13.267Z|00139|connmgr|INFO|br-int<->unix#3: 14 flow_mods
10 s ago (14 adds)
2023-06-10T19:19:13.267Z|00140|connmgr|INFO|br-int<->unix#3: 4 flow_mods 39
s ago (4 adds)
2023-06-10T19:22:23.652Z|00141|connmgr|INFO|br-int<->unix#3: 2 flow_mods 10
s ago (2 adds)
2023-06-10T19:24:11.917Z|00142|connmgr|INFO|br-int<->unix#3: 16 flow_mods
10 s ago (16 deletes)
2023-06-10T19:25:11.917Z|00143|connmgr|INFO|br-int<->unix#3: 42 flow_mods
in the 26 s starting 44 s ago (28 adds, 14 deletes)
2023-06-10T19:31:41.912Z|00144|connmgr|INFO|br-int<->unix#3: 113 flow_mods
in the 2 s starting 10 s ago (6 adds, 107 deletes)
2023-06-10T19:36:29.369Z|00145|connmgr|INFO|br-int<->unix#3: 109 flow_mods
in the 9 s starting 10 s ago (103 adds, 6 deletes)
2023-06-10T19:39:49.885Z|00146|connmgr|INFO|br-int<->unix#3: 2 flow_mods 10
s ago (2 adds)
2023-06-10T20:18:12.593Z|00147|connmgr|INFO|br-int<->unix#3: 111 flow_mods
in the 2 s starting 10 s ago (6 adds, 105 deletes)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20230610/9e74b61f/attachment.htm>