Open Stack

Hello Everyone,

Please find the week R-15 updates on 'Ubuntu Focal migration' community goal.

Tracking: https://storyboard.openstack.org/#!/story/2007865

Progress:
=======
* I have prepared the patched to migrate the unit/functional/doc/cover tox jobs to focal which are WIP till
we finish the project side testing. This and its base patches - https://review.opendev.org/#/c/738328/

* devstack and tempest base patches are changed with Depends-On on 738328 tox job patch. This way we can test the
complete gate (integration + unit  +functional  + doc + cover  +pep8 + lower-constraint) jobs with a single
testing patch by doing Depends-On:  https://review.opendev.org/#/c/734700/ (or devstack base patch or tox one if
you do not have tempest jobs to test)

* I have started a few more project testing and found bugs on the incompatible deps versions for Focal. Please
refer to the 'Bugs Report' section for details.

Bugs Report:
==========
Summary: Total 4 (1 fixed, 3 in-progress).

1. Bug#1882521. (IN-PROGRESS)
There is open bug for nova/cinder where three tempest tests are failing for
volume detach operation. There is no clear root cause found yet
-https://bugs.launchpad.net/cinder/+bug/1882521
We have skipped the tests in tempest base patch to proceed with the other
projects testing but this is blocking things for the migration.

2. We encountered the nodeset name conflict with x/tobiko. (FIXED)
nodeset conflict is resolved now and devstack provides all focal nodes now.

3. Bug#1886296. (IN-PROGRESS)
pyflakes till 2.1.0 is not compatible with python 3.8 which is the default python version
on ubuntu focal[1]. With pep8 job running on focal faces the issue and fail. We need to bump
the pyflakes to 2.1.1 as min version to run pep8 jobs on py3.8.
As of now, many projects are using old hacking version so I am explicitly adding pyflakes>=2.1.1
on the project side[2] but for the long term easy maintenance, I am doing it in 'hacking' requirements.txt[3]
nd will release a new hacking version. After that project can move to new hacking and do not need
to maintain pyflakes version compatibility. 

4. Bug#1886298. (IN-PROGRESS)
'Markupsafe' 1.0 is not compatible with the latest version of setuptools[4],
We need to bump the lower-constraint for Markupsafe to 1.1.1 to make it work.
There are a few more issues[5] with lower-constraint jobs which I am debugging.

What work to be done on the project side:
================================
This goal is more of testing the jobs on focal and fixing bugs if any otherwise
migrate jobs by switching the nodeset to focal node sets defined in devstack.

1. Start a patch in your repo by making depends-on on either of below:
devstack base patch if you are using only devstack base jobs not tempest:
https://review.opendev.org/#/c/731207/
OR
tempest base patch if you are using the tempest base job (like devstack-tempest):
https://review.opendev.org/#/c/734700/
Example: https://review.opendev.org/#/c/738126/

2. If none of your project jobs override the nodeset then above patch will be
testing patch(do not merge) otherwise change the nodeset to focal.
Example: https://review.opendev.org/#/c/737370/

3. If the jobs are defined in branchless repo and override the nodeset then you need to override the branches
variant to adjust the nodeset  so that those jobs run on Focal on victoria onwards only. If no nodeset
is overridden then devstack being branched and stable base job using bionic/xenial will take care of
this.

Once we finish the testing on projects side and no failure then we will merge the devstack and tempest
base patches.

Important things to note:
===================
* Do not forgot to add the story and task link to your patch so that we can track it smoothly.
* Use gerrit topic 'migrate-to-focal'
* Do not backport any of the patches.

References:
=========
Goal doc: https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal.html
Storyboard tracking: https://storyboard.openstack.org/#!/story/2007865

[1] https://github.com/PyCQA/pyflakes/issues/367
[2] https://review.opendev.org/#/c/739315/
[3] https://review.opendev.org/#/c/739334/
[4] https://github.com/pallets/markupsafe/issues/116
[5] https://zuul.opendev.org/t/openstack/build/7ecd9cf100194bc99b3b70fa1e6de032

-gmann