Open Stack

Resending from an address that is subscribed to the list.  Apologies to 
those of you who get this twice.

On 04/30/2018 10:06 AM, Doug Hellmann wrote:
> It would be useful to have more input from PTLs on this issue, so I'm
> CCing all of them to get their attention.
> 
> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
>> It's time to talk about the next steps in our migration from python
>> 2 to python 3.
>>
>> Up to this point we have mostly focused on reaching a state where
>> we support both versions of the language. We are not quite there
>> with all projects, as you can see by reviewing the test coverage
>> status information at
>> https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
>>
>> Still, we need to press on to the next phase of the migration, which
>> I have been calling "Python 3 first". This is where we use python
>> 3 as the default, for everything, and set up the exceptions we need
>> for anything that still requires python 2.
>>
>> To reach that stage, we need to:
>>
>> 1. Change the documentation and release notes jobs to use python 3.
>>     (The Oslo team recently completed this, and found that we did
>>     need to make a few small code changes to get them to work.)
>> 2. Change (or duplicate) all functional test jobs to run under
>>     python 3.
>> 3. Change the packaging jobs to use python 3.
>> 4. Update devstack to use 3 by default and require setting a flag to
>>     use 2. (This may trigger other job changes.)
>>
>> At that point, all of our deliverables will be produced using python
>> 3, and we can be relatively confident that if we no longer had
>> access to python 2 we could still continue operating. We could also
>> start updating deployment tools to use either python 3 or 2, so
>> that users could actually deploy using the python 3 versions of
>> services.
>>
>> Somewhere in that time frame our third-party CI systems will need
>> to ensure they have python 3 support as well.
>>
>> After the "Python 3 first" phase is completed we should release
>> one series using the packages built with python 3. Perhaps Stein?
>> Or is that too ambitious?
>>
>> Next, we will be ready to address the prerequisites for "Python 3
>> only," which will allow us to drop Python 2 support.
>>
>> We need to wait to drop python 2 support as a community, rather
>> than going one project at a time, to avoid doubling the work of
>> downstream consumers such as distros and independent deployers. We
>> don't want them to have to package all (or even a large number) of
>> the dependencies of OpenStack twice because they have to install
>> some services running under python 2 and others under 3. Ideally
>> they would be able to upgrade all of the services on a node together
>> as part of their transition to the new version, without ending up
>> with a python 2 version of a dependency along side a python 3 version
>> of the same package.
>>
>> The remaining items could be fixed earlier, but this is the point
>> at which they would block us:
>>
>> 1. Fix oslo.service functional tests -- the Oslo team needs help
>>     maintaining this library. Alternatively, we could move all
>>     services to use cotyledon (https://pypi.org/project/cotyledon/).

For everyone's awareness, we discussed this in the Oslo meeting today 
and our first step is to see how many, if any, services are actually 
relying on the oslo.service functionality that doesn't work in Python 3 
today.  From there we will come up with a plan for how to move forward.

https://bugs.launchpad.net/manila/+bug/1482633 is the original bug.

>>
>> 2. Finish the unit test and functional test ports so that all of
>>     our tests can run under python 3 (this implies that the services
>>     all run under python 3, so there is no more porting to do).

And integration tests?  I know for the initial python 3 goal we said 
just unit and functional, but it seems to me that we can't claim full 
python 3 compatibility until we can run our tempest jobs against python 
3-based OpenStack.

>>
>> Finally, after we have *all* tests running on python 3, we can
>> safely drop python 2.
>>
>> We have previously discussed the end of the T cycle as the point
>> at which we would have all of those tests running, and if that holds
>> true we could reasonably drop python 2 during the beginning of the
>> U cycle, in late 2019 and before the 2020 cut-off point when upstream
>> python 2 support will be dropped.
>>
>> I need some info from the deployment tool teams to understand whether
>> they would be ready to take the plunge during T or U and start
>> deploying only the python 3 version. Are there other upgrade issues
>> that need to be addressed to support moving from 2 to 3? Something
>> that might be part of the platform(s), rather than OpenStack itself?

Alex can probably expand on this, but I know TripleO has some challenges 
in this area.  Specifically the fact that CentOS 7 will only ever 
support Python 2 and CentOS 8 is planned to only support Python 3. Since 
CentOS 8 is not a thing yet and no release dates are announced they're 
having to use Fedora for Python 3 testing, which isn't something that 
will be supported long-term.  That makes things...complicated.

Some more details are in the PTG discussion wrap-up thread: 
http://lists.openstack.org/pipermail/openstack-dev/2018-March/128481.html

That said, I believe the plan is to be testing on Python 3 by T, so I 
guess that's ultimately the answer to your question.

>>
>> What else have I missed in these phases? Other jobs? Other blocking
>> conditions?
>>
>> Doug