Hey folks, FYI, we think that his topic overlaps a bit with some aspects of the Eventlet Removal Initiative. Indeed, removing Eventlet means removing our usage of the WSGI features offered by Eventlet. https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.... I invite you to read this parallel discussion and I wanted to share with you the HTTP SGI working group => https://wiki.openstack.org/wiki/Eventlet-removal#The_HTTP_SGI_Working_Group IMO we could capitalize from each other by having cross discussion. Choosing the next SGI candidate will dictate how we deal with HTTP for the coming decade, so I think we have to put our focus on it ASAP. We created comparisons between the alternatives at our disposal, here is an example with the HTTP/SGI comparison https://wiki.openstack.org/wiki/SGI-servers-comparison Please find all the useful details in the official wiki: https://wiki.openstack.org/wiki/Eventlet-removal We strongly encourage a collaborative decision, to avoid having multiple ways to manage things. This specialized working group would be responsible for producing a unified approach that will simplify maintenance and enhance user experience. Thanks for your time and consideration. Le jeu. 17 oct. 2024 à 13:06, Stephen Finucane <stephenfin@redhat.com> a écrit :
On Wed, 2024-10-16 at 14:16 +0200, Tobias Urdin wrote:
Hello,
This is an interesting topic and IMO has been in a weird state for years and years and I can only deduct that this has been because of, yet again, the broad spectrum of ways that OpenStack can be deployed.
Every guide, every deployment tool, every project has their own ways; supported way, and recommend way, of deploying and multiple projects has either bugs with one but no the other or the other way around making deployment even harder.
Specially on this topic though, this drops testing of one of those ways, is the plan/should we interpret this as a guideline on what developers need to test their features? Should we take this as advice on how to deploy the services and start dropping any usage of mod_wsgi in deployment tools? If no, should we expect mod_wsgi to break when eventlet is phased out because it will not be tested?
I think this topic really need to be a specification (and perhaps a community goal) around what deliverables should support.
If we want to take measures, perhaps it's time to consolidate and minimize the headache for deployers and just drop everything except for one way, the right way (that we decide and then stick with).
I don't want this to sound like throwing shade at this effort, the opposite I want it to move along, but in a clear way, and just perhaps we should just drop everything and use oslo.service and go back to how it was before mod_wsgi was recommended now that we're getting rid of eventlet and just deploy more services instead of using something like mod_wsgi to handle N copies of services.
In theory, so long as services support WSGI then it shouldn't matter what we WSGI HTTP server we use: they all speak WSGI. Sure, the configuration files are going to look different and the performance and feature sets will vary, but IMO that's a deployment problem/decision rather than something the services (or our CIs) need to worry about. I certainly don't feel *I* am in a position to state that e.g. gunicorn should be preferred to a long-standing alternative like uWSGI or mod_wsgi, an ASGI-first implementation like Daphne or uvicorn, or a newcomer like nginx unit, and this is a position apparently also shared by both the Django [1] or Flask [2] communities (to pick two well-known examples). Maybe deployers could share feedback on their chosen solution and put guidance in the deployment docs but again, that's out of scope for a DevStack change.
In addition, I should highlight that we are not currently testing against anything but uWSGI and, in rare cases, eventlet server. A grok for the various config options I mentioned reveals nothing of interest [3][4] (for example, the only override of 'WSGI_MODE' I can find is in monasca-api which, based on recent attempts to push stuff to monasca projects, is either dead or on life support). Nothing is changing here and for the most part we're just removing dead code and pushing a few long-running efforts forwards towards their conclusion. This is also true for the removal of support for deploying with eventlet server: as I noted, neither Cinder nor Nova actually test these paths any more, and Neutron is well on the way to doing the same. The only services that appear to be long- term interested in testing with eventlet server are not those interested in using eventlet for their API service but rather those that need to support a standalone mode, with multiple servers provided by a single binary. Again, as I noted below, this list includes Glance, Ironic, and possibly Swift (???). There's definitely a discussion to be had around what the replacement for eventlet server is for these projects, but that's being tackled elsewhere [5] and is not something I want to wade into here.
I hope this clarifies my suggestions here, Stephen
[1] https://docs.djangoproject.com/en/5.1/howto/deployment/wsgi/ [2] https://flask.palletsprojects.com/en/2.3.x/deploying/ [3] https://codesearch.opendev.org/?q=ENABLE_HTTPD_MOD_WSGI_SERVICES [4] https://codesearch.opendev.org/?q=WSGI_MODE%3D [5] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack....
/Tobias
On Wed, Oct 16, 2024 at 11:12:03AM UTC, Stephen Finucane wrote:
o/
tl;dr: I'd like to remove the ability to deploy services under
DevStack. I'd also like to remove the ability to deploy services under eventlet, where this feature is not currently being tested, and deprecate it where it is.
I took a foray through DevStack last week and realised that, depending on the service in question, there are currently 3 possible ways an API can be deployed by DevStack:
* Eventlet server * Apache + uWSGI * Apache + mod_wsgi
In addition, there are a number of overlapping config options that could be set to modify which of these was in use:
* ENABLE_HTTPD_MOD_WSGI_SERVICES * WSGI_MODE * Service-specific options like NOVA_USE_MOD_WSGI, CINDER_USE_MOD_WSGI etc.
Potential confusion aside, there are two issues I see with this. Firstly, and most obviously, we are working on eventlet removal, which will necessarily involve eventlet server removal [1]. In addition, there is a goal [2][3] to remove use of '.wsgi' scripts in favour of Python module paths. Quite a few services have migrated to this already, and those that haven't are going to need to do so by time pip 25.0 comes out (since that removes the deprecated legacy editable install flow [4] meaning these files will not be generated by
DevStack deployments will start failing...you have been warned). However, mod_wsgi need .wsgi scripts. I didn't realise we cared about mod_wsgi in DevStack still before last week, so to complete the wsgi script -> module migration for services that can be deployed with mod_wsgi (e.g. Keystone), we're going to have to either (a) package a .wsgi script in DevStack or (b) remove the ability to deploy with mod_wsgi.
My immediate thought on this was to simply remove all non-uWSGI deployment paths immediately and deprecate the config options for removal to force DevStack plugins to adapt. However, some services still want to test eventlet server for their API while they stabilise support for standard WSGI-based deployments (neutron), while others explicitly rely on evenlet to generate a single standalone binary (ironic, glance, ...). That means we can't just go remove everything now and must instead take a phased approach. I would
to propose we do the following:
* Remove the ability to deploy services under mod_wsgi from DevStack and deprecate/remove the WSGI_MODE flag. To be clear, other deployment tools can continue to use mod_wsgi but DevStack won't. * Entirely remove the ability to deploy under eventlet from services
not currently testing this. This includes Nova and Cinder. * Invert the default for services that are testing eventlet, so that
with uWSGI by default now and jobs will need to opt-in to eventlet rather than opt-in to uWSGI. I think this just affects Neutron. * Optionally, and likely only one Neutron is no longer testing eventlet (next cycle?), deprecate ENABLE_HTTPD_MOD_WSGI_SERVICES and {SERVICE}_USE_MOD_WSGI config options entirely and replace it with {SERVICE}_STANDALONE
mod_wsgi from pbr and therefore like that are they test options like
Glance has. This will affect Ironic and maybe more.
I have proposed a series that will do much of this. The WIP patches indicate the things that I am less clear about, such as Swift's readiness for deploying under uWSGI. I would be grateful if folks from the individual service teams could take a look and let me know what they think:
https://review.opendev.org/c/openstack/devstack/+/932193/1
Cheers,
Stephen
[1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.... [2] https://review.opendev.org/c/openstack/governance/+/902807 [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.... [4] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/
-- Hervé Beraud Senior Software Engineer at Red Hat irc: hberaud https://github.com/4383/