[neutron][requirements] Lock failures within Neutron with Eventlet 0.37.0
Hey Neutrinos, Mid-September we released Eventlet 0.37.0 [1]. Then Jens diligently proposed a patch [2] to bump our global requirements to this newest version of Eventlet. Unfortunately this requirement upgrade is stalled by the Neutron cross job [3]. There are plenty of tests failing with lock errors in threading context (`RuntimeError: cannot notify on un-acquired lock`, please see the logs [3]). A new patch [4] changed the way we manage RLocks and threading at runtime. This patch Upgrade RLocks as the last thing we do. Green existing locks after patching modules, since patching modules might involve imports that create new locks. Neutron tests might potentially need some adjustment to properly work with this new Eventlet version. The support of Python 3.13 is coming and it includes this behavioural change, so if we want the Python 3.13 support of Eventlet in OpenStack, we need first to adjust our OpenStack deliverables to prepare the ground. Can you please check this issue with your Neutron expert eyes. Feel free to also take a look at the changelog of the version 0.37.0 of Eventlet [5], but I think the other patches are irrelevant to this context and can be safely ignored as they are unrelated to locks or threading. To finish Masakari and Mistral also failed but the errors seem a bit different than the Neutron ones. In all the cases Masakri and Mistral cross jobs are non-voting jobs. Thanks for reading. [1] https://github.com/eventlet/eventlet/issues/979 [2] https://review.opendev.org/c/openstack/requirements/+/933257 [3] https://zuul.opendev.org/t/openstack/build/e399c76765f34bfda62e2125b80caf84/... [4] https://github.com/eventlet/eventlet/commit/06ec82896ebb9a26edaf6e1ad4d63393... [5] https://github.com/eventlet/eventlet/issues/979 -- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/
Let me create a LP bug for Neutron [1] and investigate it ASAP. [1]https://bugs.launchpad.net/neutron/+bug/2088453 On Mon, Nov 18, 2024 at 11:31 AM Herve Beraud <hberaud@redhat.com> wrote:
Hey Neutrinos,
Mid-September we released Eventlet 0.37.0 [1]. Then Jens diligently proposed a patch [2] to bump our global requirements to this newest version of Eventlet. Unfortunately this requirement upgrade is stalled by the Neutron cross job [3]. There are plenty of tests failing with lock errors in threading context (`RuntimeError: cannot notify on un-acquired lock`, please see the logs [3]).
A new patch [4] changed the way we manage RLocks and threading at runtime. This patch Upgrade RLocks as the last thing we do. Green existing locks after patching modules, since patching modules might involve imports that create new locks. Neutron tests might potentially need some adjustment to properly work with this new Eventlet version. The support of Python 3.13 is coming and it includes this behavioural change, so if we want the Python 3.13 support of Eventlet in OpenStack, we need first to adjust our OpenStack deliverables to prepare the ground.
Can you please check this issue with your Neutron expert eyes.
Feel free to also take a look at the changelog of the version 0.37.0 of Eventlet [5], but I think the other patches are irrelevant to this context and can be safely ignored as they are unrelated to locks or threading.
To finish Masakari and Mistral also failed but the errors seem a bit different than the Neutron ones. In all the cases Masakri and Mistral cross jobs are non-voting jobs.
Thanks for reading.
[1] https://github.com/eventlet/eventlet/issues/979 [2] https://review.opendev.org/c/openstack/requirements/+/933257 [3] https://zuul.opendev.org/t/openstack/build/e399c76765f34bfda62e2125b80caf84/... [4] https://github.com/eventlet/eventlet/commit/06ec82896ebb9a26edaf6e1ad4d63393... [5] https://github.com/eventlet/eventlet/issues/979
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/
Thank you Rodolfo. Le lun. 18 nov. 2024 à 13:14, Rodolfo Alonso Hernandez <ralonsoh@redhat.com> a écrit :
Let me create a LP bug for Neutron [1] and investigate it ASAP.
[1]https://bugs.launchpad.net/neutron/+bug/2088453
On Mon, Nov 18, 2024 at 11:31 AM Herve Beraud <hberaud@redhat.com> wrote:
Hey Neutrinos,
Mid-September we released Eventlet 0.37.0 [1]. Then Jens diligently proposed a patch [2] to bump our global requirements to this newest version of Eventlet. Unfortunately this requirement upgrade is stalled by the Neutron cross job [3]. There are plenty of tests failing with lock errors in threading context (`RuntimeError: cannot notify on un-acquired lock`, please see the logs [3]).
A new patch [4] changed the way we manage RLocks and threading at runtime. This patch Upgrade RLocks as the last thing we do. Green existing locks after patching modules, since patching modules might involve imports that create new locks. Neutron tests might potentially need some adjustment to properly work with this new Eventlet version. The support of Python 3.13 is coming and it includes this behavioural change, so if we want the Python 3.13 support of Eventlet in OpenStack, we need first to adjust our OpenStack deliverables to prepare the ground.
Can you please check this issue with your Neutron expert eyes.
Feel free to also take a look at the changelog of the version 0.37.0 of Eventlet [5], but I think the other patches are irrelevant to this context and can be safely ignored as they are unrelated to locks or threading.
To finish Masakari and Mistral also failed but the errors seem a bit different than the Neutron ones. In all the cases Masakri and Mistral cross jobs are non-voting jobs.
Thanks for reading.
[1] https://github.com/eventlet/eventlet/issues/979 [2] https://review.opendev.org/c/openstack/requirements/+/933257 [3] https://zuul.opendev.org/t/openstack/build/e399c76765f34bfda62e2125b80caf84/... [4] https://github.com/eventlet/eventlet/commit/06ec82896ebb9a26edaf6e1ad4d63393... [5] https://github.com/eventlet/eventlet/issues/979
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/
participants (2)
-
Herve Beraud
-
Rodolfo Alonso Hernandez