On Wed, 2024-10-16 at 14:16 +0200, Tobias Urdin wrote:
> Hello,
>
> This is an interesting topic and IMO has been in a weird state for years
> and years and I can only deduct that this has been because of, yet
> again, the broad spectrum of ways that OpenStack can be deployed.
>
> Every guide, every deployment tool, every project has their own ways;
> supported way, and recommend way, of deploying and multiple projects has
> either bugs with one but no the other or the other way around making
> deployment even harder.
>
> Specially on this topic though, this drops testing of one of those ways,
> is the plan/should we interpret this as a guideline on what developers
> need to test their features? Should we take this as advice on how to deploy
> the services and start dropping any usage of mod_wsgi in deployment
> tools? If no, should we expect mod_wsgi to break when eventlet is phased
> out because it will not be tested?
>
> I think this topic really need to be a specification (and perhaps a community
> goal) around what deliverables should support.
>
> If we want to take measures, perhaps it's time to consolidate and minimize the headache for deployers and just drop everything except for one way, the right way (that we decide and then stick with).
>
> I don't want this to sound like throwing shade at this effort, the
> opposite I want it to move along, but in a clear way, and just perhaps
> we should just drop everything and use oslo.service and go back to how
> it was before mod_wsgi was recommended now that we're getting rid of
> eventlet and just deploy more services instead of using something like
> mod_wsgi to handle N copies of services.

In theory, so long as services support WSGI then it shouldn't matter what we
WSGI HTTP server we use: they all speak WSGI. Sure, the configuration files are
going to look different and the performance and feature sets will vary, but IMO
that's a deployment problem/decision rather than something the services (or our
CIs) need to worry about. I certainly don't feel *I* am in a position to state
that e.g. gunicorn should be preferred to a long-standing alternative like uWSGI
or mod_wsgi, an ASGI-first implementation like Daphne or uvicorn, or a newcomer
like nginx unit, and this is a position apparently also shared by both the
Django [1] or Flask [2] communities (to pick two well-known examples). Maybe
deployers could share feedback on their chosen solution and put guidance in the
deployment docs but again, that's out of scope for a DevStack change.

In addition, I should highlight that we are not currently testing against
anything but uWSGI and, in rare cases, eventlet server. A grok for the various
config options I mentioned reveals nothing of interest [3][4] (for example, the
only override of 'WSGI_MODE' I can find is in monasca-api which, based on recent
attempts to push stuff to monasca projects, is either dead or on life support).
Nothing is changing here and for the most part we're just removing dead code and
pushing a few long-running efforts forwards towards their conclusion. This is
also true for the removal of support for deploying with eventlet server: as I
noted, neither Cinder nor Nova actually test these paths any more, and Neutron
is well on the way to doing the same. The only services that appear to be long-
term interested in testing with eventlet server are not those interested in
using eventlet for their API service but rather those that need to support a
standalone mode, with multiple servers provided by a single binary. Again, as I
noted below, this list includes Glance, Ironic, and possibly Swift (???).
There's definitely a discussion to be had around what the replacement for
eventlet server is for these projects, but that's being tackled elsewhere [5]
and is not something I want to wade into here.

I hope this clarifies my suggestions here,
Stephen

[1] https://docs.djangoproject.com/en/5.1/howto/deployment/wsgi/
[2] https://flask.palletsprojects.com/en/2.3.x/deploying/
[3] https://codesearch.opendev.org/?q=ENABLE_HTTPD_MOD_WSGI_SERVICES
[4] https://codesearch.opendev.org/?q=WSGI_MODE%3D
[5] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/PY6NUNYFU63XOIZHDDBONJDHRXJMZF5J/

>
> /Tobias
>
> On Wed, Oct 16, 2024 at 11:12:03AM UTC, Stephen Finucane wrote:
> > o/
> >
> > tl;dr: I'd like to remove the ability to deploy services under mod_wsgi from
> > DevStack. I'd also like to remove the ability to deploy services under eventlet,
> > where this feature is not currently being tested, and deprecate it where it is.
> >
> > I took a foray through DevStack last week and realised that, depending on the
> > service in question, there are currently 3 possible ways an API can be deployed
> > by DevStack:
> >
> > * Eventlet server
> > * Apache + uWSGI
> > * Apache + mod_wsgi
> >
> > In addition, there are a number of overlapping config options that could be set
> > to modify which of these was in use:
> >
> > * ENABLE_HTTPD_MOD_WSGI_SERVICES
> > * WSGI_MODE
> > * Service-specific options like NOVA_USE_MOD_WSGI, CINDER_USE_MOD_WSGI etc.
> >
> > Potential confusion aside, there are two issues I see with this. Firstly, and
> > most obviously, we are working on eventlet removal, which will necessarily
> > involve eventlet server removal [1]. In addition, there is a goal [2][3] to
> > remove use of '.wsgi' scripts in favour of Python module paths. Quite a few
> > services have migrated to this already, and those that haven't are going to need
> > to do so by time pip 25.0 comes out (since that removes the deprecated legacy
> > editable install flow [4] meaning these files will not be generated by pbr and
> > DevStack deployments will start failing...you have been warned). However,
> > mod_wsgi need .wsgi scripts. I didn't realise we cared about mod_wsgi in
> > DevStack still before last week, so to complete the wsgi script -> module
> > migration for services that can be deployed with mod_wsgi (e.g. Keystone), we're
> > going to have to either (a) package a .wsgi script in DevStack or (b) remove the
> > ability to deploy with mod_wsgi.
> >
> > My immediate thought on this was to simply remove all non-uWSGI deployment paths
> > immediately and deprecate the config options for removal to force DevStack
> > plugins to adapt. However, some services still want to test eventlet server for
> > their API while they stabilise support for standard WSGI-based deployments
> > (neutron), while others explicitly rely on evenlet to generate a single
> > standalone binary (ironic, glance, ...). That means we can't just go remove
> > everything now and must instead take a phased approach. I would therefore like
> > to propose we do the following:
> >
> >  * Remove the ability to deploy services under mod_wsgi from DevStack and
> >    deprecate/remove the WSGI_MODE flag. To be clear, other deployment tools can
> >    continue to use mod_wsgi but DevStack won't.
> >  * Entirely remove the ability to deploy under eventlet from services that are
> >    not currently testing this. This includes Nova and Cinder.
> >  * Invert the default for services that are testing eventlet, so that they test
> >    with uWSGI by default now and jobs will need to opt-in to eventlet rather
> >    than opt-in to uWSGI. I think this just affects Neutron.
> >  * Optionally, and likely only one Neutron is no longer testing eventlet (next
> >    cycle?), deprecate ENABLE_HTTPD_MOD_WSGI_SERVICES and {SERVICE}_USE_MOD_WSGI
> >    config options entirely and replace it with {SERVICE}_STANDALONE options like
> >    Glance has. This will affect Ironic and maybe more.
> >
> > I have proposed a series that will do much of this. The WIP patches indicate the
> > things that I am less clear about, such as Swift's readiness for deploying under
> > uWSGI. I would be grateful if folks from the individual service teams could take
> > a look and let me know what they think:
> >
> > https://review.opendev.org/c/openstack/devstack/+/932193/1
> >
> > Cheers,
> >
> > Stephen
> >
> > [1] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/PY6NUNYFU63XOIZHDDBONJDHRXJMZF5J/
> > [2] https://review.opendev.org/c/openstack/governance/+/902807
> > [3] https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/5RQGZZMCKKHAVH4OLSJCLBQNIHYDPGZD/
> > [4] https://ichard26.github.io/blog/2024/08/whats-new-in-pip-24.2/
> >
>