Search results for "#eventlet-removal" - openstack-discuss

Re: [oslo] Specs proposal to use cotyledon and futurist.

by smooney＠redhat.com

On Mon, 2024-06-24 at 12:00 +0200, Daniel Bengtsson wrote: > Hi there, > > Further to my previous mail, I have decided to create a specs > proposal[1] within oslo. The idea is to use cotyledon and futurist in > oslo.service and not to deprecate the project. im in favour of that approch and left some suggestions regarding the mechanics of how we could do that in the review. basically make cotyledon optional in this cycle make it the default in 2025.1 and deprecate the eventlet backend remove the eventlet support in 2025.2 or 2026.1 based on the progress of the eventlet removal work in the consuming projects. oslo messaging has a configurable executor https://github.com/openstack/oslo.messaging/blob/0d3338fd1d0094150491f81955… which the calling appllication initalise https://github.com/openstack/nova/blob/master/nova/rpc.py#L220-L226 we could either take the same approch with oslo.service or add a config option in oslo.service to choose the backbend. that would allow applications to move individual executable as that executable supprot the new backend. i.e. nova-scheduler could move to the new backbend before nova-compute. if there is a config option or environmental vairable either in the lib or applications that would allow use to experiment with the new backend in the ci on a per job basis too but thats just a side effect of keeping both implementations in oslo.service for a few release. > Please don't hesitate to > send me feedback. > > [1] https://review.opendev.org/c/openstack/oslo-specs/+/922597 >

1 year, 1 month

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-11)

by Goutham Pacha Ravi

Hello Stackers, We're eleven weeks away from the date of the coordinated release of OpenStack 2025.2 ("Flamingo") [1]. The feature freeze date for this release cycle is 2025-08-28. OpenStack project teams are busy working on and polishing features to meet this deadline. In the past week, we marked the shade and os-client-config packages as deprecated [2]. The functionality provided by these two packages has long been available through openstacksdk. We don't plan to create any new releases for these packages unless critical bugs are discovered. Besides that, there have been no significant governance updates; however, several proposals are under community review. As part of the OpenInfra Foundation's transition under the Linux Foundation, your individual membership must be renewed. The OIF staff have made this a super-easy one-click process [3]. Please complete this by 2025-07-19 to contest or vote in the next OIF Board elections. OpenStack's own PTL and TC elections will occur between 23:45 UTC on 2025-08-27 and 23:45 UTC on 2025-09-17 [4]. To contest or vote in these elections, you must renew your Foundation membership by 00:00 UTC on 2025-08-20. Don't wait—this process only takes a couple of minutes! === Weekly Meeting === The last weekly IRC meeting of the OpenStack TC took place on 2025-07-08 [5]. We noted that consensus is emerging on the updated Eventlet removal timelines [6]. We followed up on the removal of all references to the Contributor License Agreement from governance docs and OpenStack manuals. There may still be some references lurking in various project documentation websites. We request contributors to file bugs or submit simple documentation updates to switch these references to the Developer Certificate of Origin [7]. OpenStack CI jobs saw some multinode hiccups over the week due to bugs in zuul-launcher. These have been resolved by the OpenDev Infra team, and CentOS/Rocky 10 along with Debian Trixie node images are expected to be served by zuul-launcher soon. The proposal to retire Monasca [8] was discussed at length. While the project has been inactive for several cycles, the TC agreed to give maintainers a chance to state their intentions before moving ahead with retirement. We also agreed that we need better visibility into inactive projects for users and operators. We'll be tweaking the appropriate badges that show up on the repositories and adding notices on project documentation websites. Lastly, we were updated that work is underway to fully remove CLA references from Gerrit. The next weekly meeting of the OpenStack TC is on 2025-07-15 at 17:00 UTC. This meeting will be hosted over IRC in OFTC's #openstack-tc channel. Please find the meeting details, including the agenda, on the meeting's wiki page [9]. I hope you'll be able to join us. === Governance Proposals === ==== Merged ==== - Deprecate shade, os-client-config | https://review.opendev.org/c/openstack/governance/+/953549 ==== Open for review ==== - Remove Monasca from active project | https://review.opendev.org/c/openstack/governance/+/953671 - Make Eventlet removal deadlines more acceptable for operators | https://review.opendev.org/c/openstack/governance/+/952903 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 - os-test-images never releases | https://review.opendev.org/c/openstack/governance/+/954249 === Upcoming Events === - 2025-07-19: OpenInfra Days, Indonesia: https://2025.openinfra.id/ - 2025-07-26: OpenInfra Days, Vietnam: https://www.vietopeninfra.org/void2025 - 2025-08-06: Nominations open for OpenStack elections: https://governance.openstack.org/election/ === How to Contact the TC === You can reach the TC in several ways: - Email: send an email with the tag [tc] on this mailing list. - Ping us using the tc-members keyword on the #openstack-tc IRC channel on OFTC. - Join us at our weekly meeting: The Technical Committee meets every week on Tuesdays at 17:00 UTC [9]. Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] Deprecation of shade and os-client-config: https://review.opendev.org/c/openstack/governance/+/953549 [3] Renew OIF membership: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… [4] OpenStack Election Schedule: https://summit2025.openinfra.org/a/schedule [5] TC Meeting IRC Log 2025-07-08: https://meetings.opendev.org/meetings/tc/2025/tc.2025-07-08-17.00.html [6] Eventlet removal timeline proposal: https://review.opendev.org/c/openstack/governance/+/952903 [7] DCO instructions: https://docs.openstack.org/contributors/common/dco.html [8] Retiring monasca: https://review.opendev.org/c/openstack/governance/+/953671 [9] TC Meeting Agenda, 2025-07-15: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

4 weeks, 2 days

Re: [devstack][nova][cinder][ironic][glance][swift][neutron][all] Deprecating/removing non-uWSGI deployment mechanisms?

by Tobias Urdin

Hello, This is an interesting topic and IMO has been in a weird state for years and years and I can only deduct that this has been because of, yet again, the broad spectrum of ways that OpenStack can be deployed. Every guide, every deployment tool, every project has their own ways; supported way, and recommend way, of deploying and multiple projects has either bugs with one but no the other or the other way around making deployment even harder. Specially on this topic though, this drops testing of one of those ways, is the plan/should we interpret this as a guideline on what developers need to test their features? Should we take this as advice on how to deploy the services and start dropping any usage of mod_wsgi in deployment tools? If no, should we expect mod_wsgi to break when eventlet is phased out because it will not be tested? I think this topic really need to be a specification (and perhaps a community goal) around what deliverables should support. If we want to take measures, perhaps it's time to consolidate and minimize the headache for deployers and just drop everything except for one way, the right way (that we decide and then stick with). I don't want this to sound like throwing shade at this effort, the opposite I want it to move along, but in a clear way, and just perhaps we should just drop everything and use oslo.service and go back to how it was before mod_wsgi was recommended now that we're getting rid of eventlet and just deploy more services instead of using something like mod_wsgi to handle N copies of services. /Tobias On Wed, Oct 16, 2024 at 11:12:03AM UTC, Stephen Finucane wrote: > o/ > > tl;dr: I'd like to remove the ability to deploy services under mod_wsgi from > DevStack. I'd also like to remove the ability to deploy services under eventlet, > where this feature is not currently being tested, and deprecate it where it is. > > I took a foray through DevStack last week and realised that, depending on the > service in question, there are currently 3 possible ways an API can be deployed > by DevStack: > > * Eventlet server > * Apache + uWSGI > * Apache + mod_wsgi > > In addition, there are a number of overlapping config options that could be set > to modify which of these was in use: > > * ENABLE_HTTPD_MOD_WSGI_SERVICES > * WSGI_MODE > * Service-specific options like NOVA_USE_MOD_WSGI, CINDER_USE_MOD_WSGI etc. > > Potential confusion aside, there are two issues I see with this. Firstly, and > most obviously, we are working on eventlet removal, which will necessarily > involve eventlet server removal [1]. In addition, there is a goal [2][3] to > remove use of '.wsgi' scripts in favour of Python module paths. Quite a few > services have migrated to this already, and those that haven't are going to need > to do so by time pip 25.0 comes out (since that removes the deprecated legacy > editable install flow [4] meaning these files will not be generated by pbr and > DevStack deployments will start failing...you have been warned). However, > mod_wsgi need .wsgi scripts. I didn't realise we cared about mod_wsgi in > DevStack still before last week, so to complete the wsgi script -> module > migration for services that can be deployed with mod_wsgi (e.g. Keystone), we're > going to have to either (a) package a .wsgi script in DevStack or (b) remove the > ability to deploy with mod_wsgi. > > My immediate thought on this was to simply remove all non-uWSGI deployment paths > immediately and deprecate the config options for removal to force DevStack > plugins to adapt. However, some services still want to test eventlet server for > their API while they stabilise support for standard WSGI-based deployments > (neutron), while others explicitly rely on evenlet to generate a single > standalone binary (ironic, glance, ...). That means we can't just go remove > everything now and must instead take a phased approach. I would therefore like > to propose we do the following: > > * Remove the ability to deploy services under mod_wsgi from DevStack and > deprecate/remove the WSGI_MODE flag. To be clear, other deployment tools can > continue to use mod_wsgi but DevStack won't. > * Entirely remove the ability to deploy under eventlet from services that are > not currently testing this. This includes Nova and Cinder. > * Invert the default for services that are testing eventlet, so that they test > with uWSGI by default now and jobs will need to opt-in to eventlet rather > than opt-in to uWSGI. I think this just affects Neutron. > * Optionally, and likely only one Neutron is no longer testing eventlet (next > cycle?), deprecate ENABLE_HTTPD_MOD_WSGI_SERVICES and {SERVICE}_USE_MOD_WSGI > config options entirely and replace it with {SERVICE}_STANDALONE options like > Glance has. This will affect Ironic and maybe more. > > I have proposed a series that will do much of this. The WIP patches indicate the > things that I am less clear about, such as Swift's readiness for deploying under > uWSGI. I would be grateful if folks from the individual service teams could take > a look and let me know what they think: > > https://review.opendev.org/c/openstack/devstack/+/932193/1 > > Cheers, > > Stephen > > [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… > [2] https://review.opendev.org/c/openstack/governance/+/902807 > [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… > [4] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/ >

9 months, 4 weeks

Re: [eventlet-removal]When to drop eventlet support

by Arnaud Morin

Hey team, Just my two cents in this topic. As an operator, I am eager to get rid of eventlet everywhere. The current situation is painful, very hard to debug on daily basis. There is no day without hearing from the team: oh, you know, maybe it's an eventlet bug :) Cheers, Arnaud On 16.06.25 - 11:11, Balazs Gibizer wrote: > On Sat, Jun 14, 2025 at 1:24 AM <thomas(a)goirand.fr> wrote: > > > > > > On Jun 13, 2025 20:52, Jay Faulkner <jay(a)gr-oss.io> wrote: > > > > > > I'm confused a bit -- the implementation details of our threading modules are not a public API that we owe deprecation periods for. Why are we treating it as such? > > > > > > -JayF > > > > Right. Plus I don't get why operators get to choose what class of bugs they may experience, and how they will know beter than contributors. > > The new concurrency model in nova (native threading) needs different > performance tuning than the previous (eventlet). The cost of having > 1000 eventlets is negligible but having 1000 threads to replace that > will blow up the memory usage of the service. Operators expressed that > having such tuning effort happening during upgrade without a temporary > way back to the old model is scary. And honestly I agree. > > Similarly we expect nasty bugs in the new model as it is a significant > architectural change. So having no way to go back to a known good > state temporarily while the bug is fixed or worked around is scarry. > > Third, if we want to keep green CI while we are transforming nova > services to the new model without keeping a big feature branch then > we need to be able to land code that passes CI while things are half > transformed. The only way we can do that is if we support both > concurrency modes in parallel for a while. > > Cheers, > gibi > > > > > Cheers, > > > > Thomas Goirand (zigo) > > >

1 month, 3 weeks

Re: [eventlet-removal]When to drop eventlet support

by Sean Mooney

On 16/06/2025 13:09, Thomas Goirand wrote: > On 6/14/25 05:07, Dmitriy Rabotyagov wrote: >> >> >> On Sat, 14 Jun 2025, 01:24 , <thomas(a)goirand.fr >> <mailto:thomas@goirand.fr>> wrote: >> >> >> >> Right. Plus I don't get why operators get to choose what class of >> bugs they may experience, and how they will know beter than >> contributors. >> Cheers, >> >> >> Well, at least some class of bugs might be already known and >> operators might be willing to accept them, while the new ones could >> be not known by project maintainers and take quite some time to >> realize and patch. >> >> So being able to rollback to old behavior might save the day. > > This is FUD. IMO, either upstream OpenStack provides a working setup, > either it's not. sayint its FUD is not helpful. we got a driect ask form operator and soem core to not do a hard switch over. and while i wanted to only support one model for each binary at a time we were sepcificly ask to make it configurable. > In the later case, your only available action is help fixing bugs. It > is not up to the operators to double-guess what may or may not work. correct we are not planning to document how to change mode we were planning to only use this configuration in ci and operator would be told for a given release deploy this way. this is an internal impelation detail however we are not prepared to deprecate usign eventlet until we are convicned that we can run properly without it. > For beginners, this would be a horrible nightmare if default options > simply wouldn't work. We *must* ship OpenStack working by default. no one is suggesting we do otherwise. > > Cheers, > > Thomas Goirand (zigo) >

1 month, 4 weeks

Re: [devstack][nova][cinder][ironic][glance][swift][neutron][all] Deprecating/removing non-uWSGI deployment mechanisms?

by Stephen Finucane

On Wed, 2024-10-16 at 14:16 +0200, Tobias Urdin wrote: > Hello, > > This is an interesting topic and IMO has been in a weird state for years > and years and I can only deduct that this has been because of, yet > again, the broad spectrum of ways that OpenStack can be deployed. > > Every guide, every deployment tool, every project has their own ways; > supported way, and recommend way, of deploying and multiple projects has > either bugs with one but no the other or the other way around making > deployment even harder. > > Specially on this topic though, this drops testing of one of those ways, > is the plan/should we interpret this as a guideline on what developers > need to test their features? Should we take this as advice on how to deploy > the services and start dropping any usage of mod_wsgi in deployment > tools? If no, should we expect mod_wsgi to break when eventlet is phased > out because it will not be tested? > > I think this topic really need to be a specification (and perhaps a community > goal) around what deliverables should support. > > If we want to take measures, perhaps it's time to consolidate and minimize the headache for deployers and just drop everything except for one way, the right way (that we decide and then stick with). > > I don't want this to sound like throwing shade at this effort, the > opposite I want it to move along, but in a clear way, and just perhaps > we should just drop everything and use oslo.service and go back to how > it was before mod_wsgi was recommended now that we're getting rid of > eventlet and just deploy more services instead of using something like > mod_wsgi to handle N copies of services. In theory, so long as services support WSGI then it shouldn't matter what we WSGI HTTP server we use: they all speak WSGI. Sure, the configuration files are going to look different and the performance and feature sets will vary, but IMO that's a deployment problem/decision rather than something the services (or our CIs) need to worry about. I certainly don't feel *I* am in a position to state that e.g. gunicorn should be preferred to a long-standing alternative like uWSGI or mod_wsgi, an ASGI-first implementation like Daphne or uvicorn, or a newcomer like nginx unit, and this is a position apparently also shared by both the Django [1] or Flask [2] communities (to pick two well-known examples). Maybe deployers could share feedback on their chosen solution and put guidance in the deployment docs but again, that's out of scope for a DevStack change. In addition, I should highlight that we are not currently testing against anything but uWSGI and, in rare cases, eventlet server. A grok for the various config options I mentioned reveals nothing of interest [3][4] (for example, the only override of 'WSGI_MODE' I can find is in monasca-api which, based on recent attempts to push stuff to monasca projects, is either dead or on life support). Nothing is changing here and for the most part we're just removing dead code and pushing a few long-running efforts forwards towards their conclusion. This is also true for the removal of support for deploying with eventlet server: as I noted, neither Cinder nor Nova actually test these paths any more, and Neutron is well on the way to doing the same. The only services that appear to be long- term interested in testing with eventlet server are not those interested in using eventlet for their API service but rather those that need to support a standalone mode, with multiple servers provided by a single binary. Again, as I noted below, this list includes Glance, Ironic, and possibly Swift (???). There's definitely a discussion to be had around what the replacement for eventlet server is for these projects, but that's being tackled elsewhere [5] and is not something I want to wade into here. I hope this clarifies my suggestions here, Stephen [1] https://docs.djangoproject.com/en/5.1/howto/deployment/wsgi/ [2] https://flask.palletsprojects.com/en/2.3.x/deploying/ [3] https://codesearch.opendev.org/?q=ENABLE_HTTPD_MOD_WSGI_SERVICES [4] https://codesearch.opendev.org/?q=WSGI_MODE%3D [5] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… > > /Tobias > > On Wed, Oct 16, 2024 at 11:12:03AM UTC, Stephen Finucane wrote: > > o/ > > > > tl;dr: I'd like to remove the ability to deploy services under mod_wsgi from > > DevStack. I'd also like to remove the ability to deploy services under eventlet, > > where this feature is not currently being tested, and deprecate it where it is. > > > > I took a foray through DevStack last week and realised that, depending on the > > service in question, there are currently 3 possible ways an API can be deployed > > by DevStack: > > > > * Eventlet server > > * Apache + uWSGI > > * Apache + mod_wsgi > > > > In addition, there are a number of overlapping config options that could be set > > to modify which of these was in use: > > > > * ENABLE_HTTPD_MOD_WSGI_SERVICES > > * WSGI_MODE > > * Service-specific options like NOVA_USE_MOD_WSGI, CINDER_USE_MOD_WSGI etc. > > > > Potential confusion aside, there are two issues I see with this. Firstly, and > > most obviously, we are working on eventlet removal, which will necessarily > > involve eventlet server removal [1]. In addition, there is a goal [2][3] to > > remove use of '.wsgi' scripts in favour of Python module paths. Quite a few > > services have migrated to this already, and those that haven't are going to need > > to do so by time pip 25.0 comes out (since that removes the deprecated legacy > > editable install flow [4] meaning these files will not be generated by pbr and > > DevStack deployments will start failing...you have been warned). However, > > mod_wsgi need .wsgi scripts. I didn't realise we cared about mod_wsgi in > > DevStack still before last week, so to complete the wsgi script -> module > > migration for services that can be deployed with mod_wsgi (e.g. Keystone), we're > > going to have to either (a) package a .wsgi script in DevStack or (b) remove the > > ability to deploy with mod_wsgi. > > > > My immediate thought on this was to simply remove all non-uWSGI deployment paths > > immediately and deprecate the config options for removal to force DevStack > > plugins to adapt. However, some services still want to test eventlet server for > > their API while they stabilise support for standard WSGI-based deployments > > (neutron), while others explicitly rely on evenlet to generate a single > > standalone binary (ironic, glance, ...). That means we can't just go remove > > everything now and must instead take a phased approach. I would therefore like > > to propose we do the following: > > > > * Remove the ability to deploy services under mod_wsgi from DevStack and > > deprecate/remove the WSGI_MODE flag. To be clear, other deployment tools can > > continue to use mod_wsgi but DevStack won't. > > * Entirely remove the ability to deploy under eventlet from services that are > > not currently testing this. This includes Nova and Cinder. > > * Invert the default for services that are testing eventlet, so that they test > > with uWSGI by default now and jobs will need to opt-in to eventlet rather > > than opt-in to uWSGI. I think this just affects Neutron. > > * Optionally, and likely only one Neutron is no longer testing eventlet (next > > cycle?), deprecate ENABLE_HTTPD_MOD_WSGI_SERVICES and {SERVICE}_USE_MOD_WSGI > > config options entirely and replace it with {SERVICE}_STANDALONE options like > > Glance has. This will affect Ironic and maybe more. > > > > I have proposed a series that will do much of this. The WIP patches indicate the > > things that I am less clear about, such as Swift's readiness for deploying under > > uWSGI. I would be grateful if folks from the individual service teams could take > > a look and let me know what they think: > > > > https://review.opendev.org/c/openstack/devstack/+/932193/1 > > > > Cheers, > > > > Stephen > > > > [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… > > [2] https://review.opendev.org/c/openstack/governance/+/902807 > > [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack… > > [4] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/ > > >

9 months, 3 weeks

[neutron] bug deputy report for 2025w10

by Bence Romsics

Hi Neutron Team! Last week we had the following new bug reports: Gate failures: * https://bugs.launchpad.net/neutron/+bug/2100776 test_securitygroup(ovs-openflow) fullstack test random failure with Time out exception unassigned * https://bugs.launchpad.net/neutron/+bug/2101165 Random failure in test_established_tcp_session_after_re_attachinging_sg unassigned * https://bugs.launchpad.net/neutron/+bug/2101166 Random failure in dvr ovs grenade jobs to ping vm post upgrade unassigned * https://bugs.launchpad.net/neutron/+bug/2100806 Intermittent test_batch_notifier failure fix proposed: https://review.opendev.org/c/openstack/neutron/+/943212 High: * https://bugs.launchpad.net/neutron/+bug/2101150 User who is not owner of the SG can create/delete rules in the shared SG assigned to Slawek * https://bugs.launchpad.net/neutron/+bug/2101063 [OVN] min-bw QoS policy cannot be defined alone, exception raised fix proposed: https://review.opendev.org/c/openstack/neutron/+/943664 Medium: * https://bugs.launchpad.net/neutron/+bug/2101140 [OVN] Make LSP up/down events processing resilient to IDL disconnections fix proposed: https://review.opendev.org/c/openstack/neutron/+/943404 * https://bugs.launchpad.net/neutron/+bug/2101162 [OVN] Use HA_Chassis_Group for tunnelled gateway network LRPs WIP fix: https://review.opendev.org/c/openstack/neutron/+/938134 * https://bugs.launchpad.net/neutron/+bug/2100938 Assign HA_Chassis_Group to all LRPs for tunnelled gateway assigned to Rodolfo Eventlet removal: * https://bugs.launchpad.net/neutron/+bug/2101076 [eventlet-removal] Remove the usage of eventlet in the MacVTap agent unassigned RFEs: * https://bugs.launchpad.net/neutron/+bug/2100770 [RFE] Make alembic_migration scripts in neutron idempotent * https://bugs.launchpad.net/neutron/+bug/2073254 nova-compute optional wait for vif-plugged in ovn usecase Further triage needed: * https://bugs.launchpad.net/neutron/+bug/2101077 neutron-bgp-agent dies with debug log + json Info requested from reporter on reproduction method * https://bugs.launchpad.net/neutron/+bug/2100752 Severe Packet Loss During Live Migration Due to Premature Route Advertisement in Neutron BGP Dynamic Routing Edit Lajos promised to triage this Cheers, Bence

5 months

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-9)

by Goutham Pacha Ravi

Hello Stackers, We're a month away from the feature freeze date for the 2025.2 ("Flamingo") release cycle [1], scheduled for 2025-08-28. OpenStack project teams are observing various internal deadlines prior to that important date to ensure a complete and timely release. As we look ahead, OpenStack's election officials will begin accepting self-nominations for the upcoming Project Team Lead roles and four seats on the OpenStack Technical Committee starting 2025-08-06 [2]. The nomination deadline is two weeks later, on 2025-08-20. You are welcome to submit an early nomination [3] now if you anticipate being away during the nomination window. To be a Project Team Lead candidate, you must be an Active Project Contributor—i.e., someone who has contributed to the project you wish to lead in the past 12 months (counted up to 2025-08-20). Any individual member of the OpenInfra Foundation, regardless of contributor status, may propose their candidacy for the OpenStack Technical Committee. To vote in these elections, you must be a member of the OpenInfra Foundation [4] and an active contributor—either through merged contributions to official projects or via nomination by a project’s PTL, liaisons, any Special Interest Group, or the OpenStack Technical Committee [5]. To receive your ballot, please ensure your email address is added in Gerrit and that you've opted in to receive emails from CIVS before 2025-08-20: https://civs1.civs.us/cgi-bin/opt_in.pl In the past week, the TC approved revised milestones for the cross-project goal of removing the use of Eventlet [6]. We still aim to drop Eventlet usage completely in the 2027.2 ("J") release. The "aetos" project now has a canonical service type called "metric-storage" [7]. Several other governance changes are currently under community review. === Weekly Meeting === The last weekly meeting of the OpenStack TC took place [8] on 2025-07-22, with a smaller attendance than usual. We expect this trend to continue through the summer break in the northern hemisphere. Key updates included changes to the Eventlet-removal goal timeline and a deeper look into the prolonged inactivity of the Monasca project team. While volunteers have consistently expressed interest in maintaining the project, it hasn’t met reactivation milestones since its last active cycle (2024.1). However, some improvements have been merged across the team’s repositories. The TC agreed to enforce a more formal and transparent process: projects must explicitly request extensions to inactive status [9], and future documentation will include retirement deadlines that compel TC votes on any extensions. We'll continue discussing the next steps for Monasca through governance proposals and this mailing list. Opendev administrators informed us that they completed the transition off Nodepool, and Debian Trixie images are nearly ready for testing. This is also useful for transitioning the Ceph CI job to Debian to enable faster fixes in the client software. In open discussion, the group revisited the definition of corporate affiliation in the TC charter. A proposal will be made to clarify what affiliation diversity means and how/why it should be disclosed during elections. The next meeting of the OpenStack Technical Committee is today, 2025-07-29. It will be hosted on the #openstack-tc channel on OFTC at 1700 UTC. Please find the meeting's agenda and other details on its wiki page [10]. I hope you'll be able to join us. === Governance Proposals === === Merged === - Make Eventlet removal deadlines more acceptable for operators | https://review.opendev.org/c/openstack/governance/+/952903 - Retire networking-midonet | https://review.opendev.org/c/openstack/governance/+/955364 === Open for review === - Define "affiliation" within the context of the TC | https://review.opendev.org/c/openstack/governance/+/956024 - Add series names to runtimes doc | https://review.opendev.org/c/openstack/governance/+/955810 - Remove Monasca from active project | https://review.opendev.org/c/openstack/governance/+/953671 - os-test-images never releases | https://review.opendev.org/c/openstack/governance/+/954249 - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 === Upcoming Events === - 2025-08-06: Nominations open for OpenStack elections: https://governance.openstack.org/election/ - 2025-08-20: Nominations close for OpenStack elections - 2025-08-26: OpenInfra Days, Korea: https://2025.openinfradays.kr/ - 2025-08-28: Feature Freeze deadline, Milestone-3 of the Flamingo release [1] - 2025-08-29: Colombia OpenInfra User Group Meetup: https://www.meetup.com/colombia-openinfra-user-group/events/307096751/ - 2025-10-17: OpenInfra Summit, Paris-Saclay: https://summit2025.openinfra.org/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair [1] 2025.2 "Flamingo" Release Schedule: https://releases.openstack.org/flamingo/schedule.html [2] 2026.1 OpenStack Elections: https://governance.openstack.org/election/ [3] Submitting your candidacy: https://governance.openstack.org/election/#how-to-submit-a-candidacy [4] Join the OpenInfra Foundation: https://openinfra.org/join/individual/ [5] Who is an Active Contributor: https://governance.openstack.org/tc/reference/charter.html#voters-for-tc-se… [6] Eventlet removal timeline: https://governance.openstack.org/tc/goals/selected/remove-eventlet.html#com… [7] OpenStack service types: https://specs.openstack.org/openstack/service-types-authority/#service-data [8] TC Meeting IRC Log 2025-07-22: https://meetings.opendev.org/meetings/tc/2025/tc.2025-07-22-17.00.html [9] Emerging and inactive projects: https://governance.openstack.org/tc/reference/emerging-technology-and-inact… [10] TC Meeting Agenda, 2025-07-29: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Next_Meeting

2 weeks, 1 day

Re: [oslo.messaging] Heartbeat in pthread

by Takashi Kajinami

On 10/1/24 23:31, Arnaud Morin wrote: > Hey, > > I totally agree about the fact that heartbeat_in_pthread and the > oslo.log PipeMutex are technical debt that we need to get rid of, > as well as eventlet. > > However, despite the fact that it seems purely cosmetic on your side, > we believe it's not. > I can't prove / reproduce the issue on a small infra, but definetely, > at large scale, having those tcp connections to be dropped by rabbitmq > and recreated in a loop by agents is affecting the cluster. > > I know all the pain that these settings introduced in the past, but now > I feel we are in a stable situation regarding this, that's why I am > surprised about deprecating heartbeat_in_pthread now. > > Can we, as least, make sure we keep all of this until we switch off > eventlet? > In other words, can we get rid of eventlet, then remove this params? > and not the opposite? That's the plan. We deprecated the parameter because it is no longer useful *ONCE* we get rid of eventlet completely. The parameter will be removed ONLY AFTER the eventlet removal is down. > > Regards, > > Arnaud > > > On 01.10.24 - 11:38, smooney(a)redhat.com wrote: >> im glad you managed to make it work but form a nova perspective we >> do not recommend using heartbeat_in_pthread=true with nova-compute to the >> point that i woudl cosndier that config unsupported. >> >> we also dont recommend using it with nova-api even when running via a wsgi server such as mod_wsgi >> or uwsgi. >> >> the only thing this has ever done is remove a cosmetic waring in the rabbit/nova logs >> due to the heartbeat timing out. This has never fix any functional bug that >> we were aware of but has resulted in several real bugs. >> >> the most recent we hit was https://launchpad.net/bugs/1983863 which was mitigated by >> https://review.opendev.org/c/openstack/oslo.log/+/852443 however that uses a unsafe debug >> option in eventlet eventlet.debug.hub_prevent_multiple_readers(False) >> >> while you may be able to make heartbeat_in_pthread work with a lot of work >> as Takashi noted this will eventually go away when we remove evently and to enable that removal >> we need to replace the PipeMutex that currently fixes logging in a native thread so >> heartbeat_in_pthread is part of the technial debt we need to remvoe to evenrally allow >> us to move away form eventlet entirly. >> >> On Tue, 2024-10-01 at 09:13 +0000, Arnaud Morin wrote: >>> Yes, I agree that it used to be broken, but since the bug was reported, >>> we merged the following fixes: >>> >>> https://review.opendev.org/c/openstack/oslo.messaging/+/894731 >>> https://review.opendev.org/c/openstack/oslo.messaging/+/875615 >>> https://review.opendev.org/c/openstack/oslo.messaging/+/876318 >>> >>> That's why I believe everything should be fine now :) >>> >>> >>> On 01.10.24 - 17:20, Takashi Kajinami wrote: >>>> I was too fast to push Send button. >>>> >>>> It's still interesting to see that you enabled the feature for eventlet services, >>>> such as nova-compute. In the past we got a few bugs caused by that feature, >>>> which made us eventually revert the default value to False. >>>> https://bugs.launchpad.net/oslo.messaging/+bug/1934937 >>>> https://bugs.launchpad.net/oslo.messaging/+bug/1949964 >>>> https://bugs.launchpad.net/oslo.messaging/+bug/1949964 >>>> >>>> You might need to check if the reported problem is reproduced in your env. >>>> >>>> On 10/1/24 17:15, Takashi Kajinami wrote: >>>>> Setting heartbeat_in_pthread is known to break services using eventlet >>>>> so it SHOULD NOT be enabled by default. We tried to enable it by default >>>>> in the past but eventually reverted it after seeing multiple problems. >>>>> >>>>> You can selectively disable it for services not using eventlet (api >>>>> services run by http + mod_wsgi or uwsgi) but should keep it False for >>>>> the other services. >>>>> >>>>> Once we get rid of eventlet then we no longer use eventlet thread for >>>>> heartbeat so we no longer need that option (because the behavior would >>>>> be equivalent to one with heartbeat_in_pthread=True). But until that point >>>>> we can't change the default, unless someone is willing to dig into >>>>> the past problems to make the feature completely work with eventlet (which >>>>> I don't think worth paying effort for at this stage). >>>>> >>>>> On 10/1/24 16:34, Arnaud Morin wrote: >>>>>> Hello, >>>>>> >>>>>> I completely miss the deprecation of heartbeat_in_pthread in >>>>>> oslo.messaging [1]. >>>>>> >>>>>> We heavily rely on this parameter downstream and our opinion is that it >>>>>> should be set to True by default. We use it for both wsgi services and >>>>>> agents (nova-compute, neutron agents, etc.). >>>>>> >>>>>> I understand that eventlet will be dropped in the future, but should we >>>>>> set heartbeat_in_pthread to True by default until then? >>>>>> >>>>>> Regards, >>>>>> >>>>>> Arnaud. >>>>>> >>>>>> >>>>>> [1] https://review.opendev.org/c/openstack/oslo.messaging/+/925778 >>> >>

10 months, 1 week

[tc][all] OpenStack Technical Committee Weekly Summary and Meeting Agenda (2025.2/R-14)

by Goutham Pacha Ravi

Hello Stackers, We're fourteen weeks away from the coordinated 2025.2 "Flamingo" release of OpenStack [1], and Milestone-2 for this release is on 2025-07-03. In the past week, the Call for Papers for the upcoming OpenInfra Summit in Paris-Saclay ended; however, the deadline to propose Forum sessions and Project Updates sessions is 2025-07-08 at 23:59 PST [2]. This time at the summit, there's also a "New Contributor Showcase" inviting presentations on projects that leverage or support open infrastructure. Whether it’s your first contribution, an open source tool, or a collaboration with a community, we want to hear about it, so please consider applying [3]. In the past week, the OpenStack Technical Committee did not merge any new governance changes. A few change proposals are open for the community's input. OpenDev Infra administrators and OpenStack's Release Management team have posted several changes to begin enforcing the Developer Certificate of Origin in lieu of the Contributor License Agreement from July 1, 2025 [4]. If you haven't already begun using "git commit -s" to sign off your commits in adherence to the DCO, please do so. As a reminder, from July 1st, OpenDev's Gerrit system (https://review.opendev.org) will reject any changes that do not include a "Signed-off-by" line in the commit message. Simultaneously, the existing CLAs will no longer be available for new contributors to sign. === Weekly Meeting === The OpenStack Technical Committee met on OFTC's #openstack-tc channel on 2025-06-17 [5]. The team confirmed plans to transition to DCO (Developer Certificate of Origin) by July 1st, approving updates to the contributor guide and discussing the ongoing process of integrating DCO across tools and translations. We then focused on improving the contributor experience, with analyses of review metrics being shared with various project teams like Nova and Cinder through weekly IRC meetings. This data collection aims to identify pain points and best practices to inform future strategies. We plan to drive collective attention to this effort during the PTG in October. We also discussed the level of activity in the Cyborg project. We have had issues contacting the core maintainers to get some critical fixes merged, and the TC agreed to begin the process to mark the project "inactive." This decision aligns with the upcoming M-2 release deadline. The discussion also prompted a health check on several project teams. We'll be discussing this in further detail on proposals directly on Gerrit, or in a future weekly meeting. OpenDev Infra administrators completed a transition from Nodepool to Zuul-launcher and pleasantly have not seen any negative impact on the CI. During an Open Discussion, concerns were raised about the timeline specified by the Eventlet Removal TC goal [6]. Nova maintainers, in particular, expressed the need for more time to ensure a stable transition to threading and adequate operator testing, suggesting a revised target of 2028.1 for full Eventlet removal, a point that will be further debated on the mailing list [7] and a Gerrit proposal [8]. The next meeting of the OpenStack Technical Committee is on 2025-06-24. It will be hosted over IRC in OFTC's #openstack-tc channel at 1700 UTC. Please find the agenda and other details on the meeting's wiki page [9]. I hope you'll be able to join us. === Governance Proposals === ==== Open for review ==== - Require declaration of affiliation from TC Candidates | https://review.opendev.org/c/openstack/governance/+/949432 - Make Eventlet removal deadlines more acceptable for operators | https://review.opendev.org/c/openstack/governance/+/952903 - Mark Cyborg inactive | https://review.opendev.org/c/openstack/governance/+/952798 === Upcoming Events === - 2025-06-28: OpenInfra+Cloud Native Day, Vietnam: https://www.vietopeninfra.org/void2025 - 2025-07-03: OpenStack's 15th Birthday, Colombia User Group: https://www.meetup.com/colombia-openinfra-user-group/events/308383244 - 2025-07-08: OpenInfra Board meeting: https://board.openinfra.org/ - 2025-07-19: OpenInfra Days, Indonesia: https://2025.openinfra.id/ Thank you very much for reading! On behalf of the OpenStack TC, Goutham Pacha Ravi (gouthamr) OpenStack TC Chair

1 month, 3 weeks

openstack-discuss search results for query "#eventlet-removal"