Open Stack

On Wed, 2019-03-20 at 02:22 +0000, Monty Taylor wrote:
> > we could proably add the concurrent.futures executor interface to privsep if we wanted its only 3 function after all
> > https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor
> > but since there is a python 2.7 backport we may be better served just using it.
> 
> It's also worth pointing out the futurist library, which is part of oslo:
> 
> https://opendev.org/openstack/futurist
> 
> It's similar to (and uses) concurrent.futures - except it also has a 
> bunch of different backend impls - so you can have tasks run in threads, 
> or it can use eventlet, or processes - or work synchronously.

thanks i was looking for this.
i was aware that oslo.messaging to name buy one lib supported
a configurable executor.

i had expected to find those executos in oslo.concurancy but
looking a futurist it makes sense.

i see the executors defiend in https://opendev.org/openstack/futurist/src/branch/master/futurist/_futures.py
do indeed derive form the concurent future executor abstract base class.

the map function is in the executor base class is internally in terms of the submit function so the futuist
executors are compatiable with the standard lib varients from python3.

i wonder if it would make sense to crete an optional dependency on privsep from futurist and add a privsep executor?

pivoting back to melenie's original question regarding what alternative we have for nova we could change
our explict use of eventlets to the GreenThreadPoolExecutor
https://opendev.org/openstack/futurist/src/branch/master/futurist/_futures.py#L322
this would still require and use eventlets under the hood so the perfromace would be the same as our current explict
use of the eventlet event loop but would allow us to change the eventloop via config the same way oslo.messaging
allows changing for the eventlet executor to the threading executor.

if we removed all explcit use of eventlet this way we could then start looking at our implit use of eventlets which
is largely for db acess, rest api calls to other services and some virt dirver specific calls.

anyway i had assumed that revisiting dropping eventlets form services would be a U conversation at the earliest
but if we swap to using the futurist executors for the multi cell scater gather list operation we could stop
monkeypatching under wsgi again and instead use the ThreadPoolExecutor or ProcessPoolExecutor in that case.
when running the api without wsgi we can revert to the GreenThreadPoolExecutor.

> 
> > privsep is effectivly just a ThreadPoolExecutor that layers on the privlage escalation/descalation logic
> > via either fork and drop privalages or sudo + rootwrap/privsep helper to gain privileges