Open Stack

On Wed, Aug 23, 2023, at 10:33 AM, Ghanshyam Mann wrote:
> ---- On Wed, 23 Aug 2023 04:07:36 -0700    wrote --- 
>  > On Wed, 2023-08-23 at 09:10 +0200, Thomas Goirand wrote:
>  > 
>  ...
> ...
>  > 
>  > having said that we did just recently merge the devstack venv support
>  > 
>  > and tools like https://github.com/pyenv/pyenv exist.
>  > 
>  > so if adding debian unstable or testing is not an option and peopel 
> were willing to develop a job
>  > 
>  > (devstack or tox) that worked with pyenv on say ubuntu 22.04 we 
> could in theory test any python version
>  > 
>  > we wanted without the need to mirror fast moving repo to afs.
>  > 
>
> I think zigo asking is valid ask here to upstream and honestly saying I 
> am not seeing
> much challenges to do that. If we have image/nodepool in infra (even 
> with unstable image
> like we do for CentOS stream) then running next python version job as 
> non voting or
> periodic should be ok and should not cost us much. Even it give win-win 
> situation to upstream
> as well as distro/packages maintainers.

The challenge is in adding and maintaining the image and associated mirror content. Yes, if you ignore that then this becomes simple you just tell Zuul to run nodes on the appropriate nodeset and when things break you can either debug them (ideal) or ignore them.

I think Sean's suggestion is a good one since it can be PoC'd without much effort by pushing a change to devstack. Similarly you could hack up unittest jobs to use pyenv and supply a specific newer python on those platforms.

Once upon a time we had OpenSUSE Tumbleweed images. The idea behind them was that Tumbleweed is a bleeding edge rolling release distro and any jobs running there would get new versions of all the things. Very few people took advantage though because things were regularly breaking. I would suggest starting with an existing stable base then adding the specific components on top as in Sean's pyenv suggestion. You can also do similar with container images potentially. This should ideally avoid the problems we've run into trying to run more up to date systems in CI.

Something else we did long ago that I think was actually beneficial is use the beginning of OpenStack release cycles to intentionally disrupt things that need updates. For example linter rules would get updated globally and everyone would need to adjust their linter rules within the projects or adjust the code to pass. OpenStack still isn't running python 3.11 jobs as far as I can tell despite it being available on multiple images today. Maybe OpenStack should be more aggressive at the beginning of cycles and explicitly introduce potentially problematic updates early when the cost is lower and then make things work. It doesn't make a whole lot of sense to me to worry about pre release Python versions when we aren't even up to date with released versions. Solve that problem first, then continue to optimize for Python next.

>
> -gmann