[neutron][eventlet] RPC handler thread model incompatible with eventlet
Hello Neutrinos: I think you are aware about the effort to move all CI jobs to use WSGI. Other OpenStack projects have already migrated to this implementation to the detriment of the eventlet server. That will help us in the evenlet deprecation task. However some modules are still using eventlet, thus even when using WSGI in the Neutron API, it is still needed to monkey patch the system libraries. The most recent case is LP#2075147 [1]. To that purpose [2] was proposed. Again, we had another adverse effect of [2] in the RPC event handler, that is used to inform the agents, via RPC, about resource events (creation, update, deletion); this is why only the ML2/OVS jobs were affected in the CI, as opposed to ML2/OVN that is not using RPC. With [2] in place, the RPC handler thread model was not working correctly (check LP#2077790 [3]). The thread in charge of sending the RPC messages got stuck during the DB retrieval process or the RPC cast itself. That was leading to a delay in the event processing in the agents. In the CI, the most affected tests were those related to the SG rule creation/deletion. The OVS agent didn't receive on time the SG rule creation/deletion and didn't create the needed OF rules in the local OVS. In order to fix this issue, [4] has been proposed. This patch proposes to handle the RPC messages in the same thread of the API call. Because the RPC call is a ``cast`` call (no reply is expected), that won't block the API call and will just increase the time a bit. Once eventlet is deprecated and removed, it would be possible to go back to the previous model using a thread (preemptive in this case) to make the RPC calls. Regards. [1]https://bugs.launchpad.net/neutron/+bug/2075147 [2]https://review.opendev.org/c/openstack/neutron/+/925376 [3]https://bugs.launchpad.net/neutron/+bug/2077790 [4]https://review.opendev.org/c/openstack/neutron/+/926922
Related to these patches is also [1]. "neutron-tempest-plugin-sfc", that uses ML2/OVS, was also affected by this RPC cast delay. [1]https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/927109
participants (1)
-
Rodolfo Alonso Hernandez