I looked into the failure in heat, and it seems the issue is narrowed down to the "RuntimeError: do not call blocking functions from the mainloop" exception raised from eventlet. However I'm running out of ideas of any wrong usage causing it. Quick search explains this may be mostly caused by incomplete monkey_patch-ing but we do it at the first step of each command. I see sys is imported early in the console script generated by setuptools but idk if that can be a cause and also is fix-able. One suspicious thing is that we use threadgroup to run stack update stacks but the implementations are all imported from oslo.services which depend on eventlet (and not any other threading/concurrency mechanism). So I'm wondering if I anyone has better idea about the current issue. My rough view is that the issue is caused by not heat itself but something underlying but I can be wrong as I'm still struggling to understand the full details inside eventlet after some years. Thank you, Takashi On 11/23/24 11:34 AM, Ghanshyam Mann wrote:
---- On Mon, 18 Nov 2024 21:17:35 -0800 Ghanshyam Mann wrote ---
---- On Sat, 16 Nov 2024 21:08:52 -0800 Ghanshyam Mann wrote ---
---- On Mon, 21 Oct 2024 07:03:11 -0700 Ghanshyam Mann wrote ---
Hello Everyone,
As per the 2025.2 cycle testing runtime, we need to run upstream CI/CD jobs on Ubuntu Noble (24.04) [1]. For better tracking and speeding up this work, it is the community-wide goal for this cycle[2]. . . .
Current Status: ============
* Projects Green on Noble: 19 (mentioned in etherpad). * Projects Failing on Noble: 20
The below projects are failing, and testing change can be found in etherpad:
* Adjutant (Operations Processes automation) * Cloudkitty (Rating service) * Freezer (Backup, Restore, and Disaster Recovery service) * Heat (Orchestration service) * Horizon (Dashboard) * Magnum (Container Infrastructure Management service) * Masakari (Instances High Availability service) * Monasca (Monitoring) * Neutron (Networking service) * Octavia (Load-balancer service) * Oslo (Common libraries) * Requirements (Common dependency management) * Skyline (Dashboard) * Swift (Object Storage service) * Tacker (NFV Orchestration service) * Telemetry (Telemetry service) * Venus (Log management service) * Vitrage (RCA (Root Cause Analysis) service) * Zaqar (Message service) * Zun (Containers service)
Link: https://etherpad.opendev.org/p/migrate-to-noble
IMP: Deadline: Nov 29th 2024 (R18) to migrate the devstack/tempest/tox-based jobs to Noble and after that project gate who have not fixed the failure will be broken.
-gmann
-gmann
[1] https://etherpad.opendev.org/p/migrate-to-noble
-gmann
[1] https://governance.openstack.org/tc/reference/runtimes/2025.1.html [2] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubuntu... [3] https://review.opendev.org/c/openstack/tempest/+/932156
-gmann