Open Stack

This is week 1 of the roll-out of the "Run under Python 3 by default"
goal (https://governance.openstack.org/tc/goals/stein/python3-first.html).

I intend to send a summary message like this roughly weekly to
report progress on the Zuul setting migrations, because that portion
of the goal needs to be coordinated.

I will also be watching for changes to
https://wiki.openstack.org/wiki/Python3 and will try to report on
progress with functional or unit test jobs as teams log the information
there.

This email is a bit longer than I hope they will usually be, because
it is the first and has some planning information.

== The Plan ==

The completion criteria from the goal are:

1. The Zuul settings to attach jobs have been moved from the
   openstack-infra/project-config repository into each project
   repository.
2. Documentation build and publish jobs use python 3.
3. Release notes build and publish jobs use python 3.
4. Source code linter jobs (pep8, pylint, bandit, etc.) should use
   python 3.
5. Release artifact build and publish jobs use python 3 and publish to
   PyPI.
6. There are functional test jobs running under python 3.
7. The wiki tracking page is up to date for the project.
8. Projects are running python 3.6 unit test jobs.

The goal champions will start by preparing the patches for steps
1-5, and 8. That will give project teams more time to focus on
reviewing and on completing steps 6 and 7, which will need the
expertise of someone on the team.

PLEASE do not write the Zuul configuration migration patches
yourselves.  The rules for which things need to move ended up a bit
complicated, it's all automated, and we do not want to start them
before all of the stable/rocky branches are created for a team
(otherwise we risk misconfiguring the stable branch). Sit tight
and be ready to review the patches when we send them your way.

We are waiting to start most teams until the final releases for
Rocky are done, but we're going to start some of the teams less
affected by that deadline early (like Oslo, Infra, and possibly
Documentation).  Project teams should be prepared to assist if there
are any issues or questions about jobs with branch-specifiers.

While the job migration is under way, changes to project-config for
the team's repositories will be locked. As soon as the patches to
import the job settings are landed for *all* of the repositories
owned by a team we will update the project-config repo to remove
those settings and then the job settings for the team will be
unfrozen again. So, please review zuul configuration changes quickly,
but carefully. :-)

Step 6 involves adding more test jobs and may also require code
changes or tox settings updates, so that work will be left to the
project team.

Step 7, updating the wiki tracking page to indicate what level of
testing is being done and where any gaps might be, is the responsibility
of the project team.

Step 8 starts with a simple patch to add the unit test jobs, which
will come as part of the rest of the zuul migration patches. If the
tests do not immediately work, We will need the project teams to
fix things up in the code or tox.ini files so the tests pass.

== Tracking Progress ==

Because teams potentially need to do something in each repository,
I have created separate stories for each team in storyboard, with
one task per repo. Those stories are all tagged with 'goal-python3-first'
and appear on the tracking board at
https://storyboard.openstack.org/#!/board/104 The tasks on these
stories are for teams to track their work. Feel free to add more
tasks or otherwise elaborate on the stories as needed. If you add
more stories, please use the same tag so they will show up on the
board.

Because not all teams have migrated to storyboard, some of the tasks
are associated with the openstack/governance repo instead of a team
repository. If your team wants to track work using a different tool,
please come back and update the storyboard tasks as part of completing
your goal so anyone tracking progress can find it all in one place.
PTLs should update the stories based on the instructions in the
goal procedures
(https://governance.openstack.org/tc/goals/index.html#team-acknowledgment-of-goals
). If you include the story and task information in commit messages,
some of that updating will happen for free (this is one of the
reasons we're using storyboard).

The goal champions will use a separate story, with one task per
team, to track the work we are going to do with the zuul configuration
migration. See https://storyboard.openstack.org/#!/story/2002586
for details.

== Ongoing work ==

We are making good progress with migrating the zuul settings for
the Oslo team as a final test of the full process. We will start
with other regular teams after the final release candidates for
Rocky are done (after 24 Aug), and the cycle-trailing projects after
their release deadline has passed (after 31 Aug).

== Next Steps ==

If your team is interested in being one of the first to have your
zuul settings migrated, please let us know by following up to this
email. We will start with the volunteers, and then start working
our way through the other teams.

After the final Rocky release, I will propose the change to
project-config to change all of the packaging jobs to use the new
publish-to-pypi-python3 template with the intent of having that
change in place before the first milestone for Stein so that we
have an opportunity to test it. Besides standardizing all of the
Python package building jobs, this change will add a new check job
to try to build a package and verify that it could be published by
running "setup.py check". This should not cause any problems for
projects that were released during Rocky, because we added the same
check to the release validation job in openstack/releases. However,
it may cause problems for projects that were not released during
Rocky. Let the release know if you run into trouble and we will
help figure out what's wrong. See
https://governance.openstack.org/tc/goals/stein/python3-first.html#release-artifact-publishing
for more details.

== How can you help? ==

The best way for most people to help is to review the patches for
the zuul changes and to work on the functional test jobs.

We have scripts to prepare the zuul patches, but they're a bit
finicky and it's easier if we have a small team using them and
fixing them as we go. The champions cannot do the review work or
the functional test job work, so that is *really* the best way to
help.

Another way to help is to pick up a patch that has failing tests
and help fix it. For example, the patches to change the documentation
jobs or add python 3.6 unit test jobs may need some code changes
before they will work. The champions are signed up to write those
patches, but won't necessarily be able to fix everything that is
wrong. You'll find a list of goal-related patches failing tests at
https://review.openstack.org/#/q/topic:python3-first+status:open+(+label:Verified-1+OR+label:Verified-2+)

== How can you ask for help? ==

If you have any questions, please post them here to the -dev list
with the topic tag [python3] in the subject line. Posting questions
here will give the widest audience the chance to see the answers.

== Reference Material ==

Goal description: https://governance.openstack.org/tc/goals/stein/python3-first.html 
Open patches needing reviews: https://review.openstack.org/#/q/topic:python3-first+is:open
Storyboard: https://storyboard.openstack.org/#!/board/104
Zuul migration notes: https://etherpad.openstack.org/p/python3-first
Zuul migration tracking: https://storyboard.openstack.org/#!/story/2002586
Python 3 Wiki page: https://wiki.openstack.org/wiki/Python3