Open Stack

On Tue, May 29, 2018 at 03:51:31PM -0400, Doug Hellmann wrote:
> Following up on this topic, at the Forum discussion last week (see
> https://etherpad.openstack.org/p/YVR-python-2-deprecation-timeline) the
> general plan outlined below was acceptable to most of the folks in the
> room with a few small changes (included below).
> 
> Excerpts from Doug Hellmann's message of 2018-04-25 16:54:46 -0400:
> > It's time to talk about the next steps in our migration from python
> > 2 to python 3.
> > 
> > Up to this point we have mostly focused on reaching a state where
> > we support both versions of the language. We are not quite there
> > with all projects, as you can see by reviewing the test coverage
> > status information at
> > https://wiki.openstack.org/wiki/Python3#Python_3_Status_of_OpenStack_projects
> > 
> > Still, we need to press on to the next phase of the migration, which
> > I have been calling "Python 3 first". This is where we use python
> > 3 as the default, for everything, and set up the exceptions we need
> > for anything that still requires python 2.
> > 
> > To reach that stage, we need to:
> > 
> > 1. Change the documentation and release notes jobs to use python 3.
> >    (The Oslo team recently completed this, and found that we did
> >    need to make a few small code changes to get them to work.)
> > 2. Change (or duplicate) all functional test jobs to run under
> >    python 3.
> > 3. Change the packaging jobs to use python 3.
> > 4. Update devstack to use 3 by default and require setting a flag to
> >    use 2. (This may trigger other job changes.)
> 
> Also:
> 
> - Ensure that devstack configures mod_wsgi (or whatever WSGI service) to
>   use Python 3 when deploying API components.

The python 3 dsvm jobs already do this for the most part. All API services that
support running as a wsgi application run under uwsgi with a single apache
redirecting traffic to those. This is the supported model for running wsgi
services on devstack. Currently keystone, glance, nova, placement, and cinder
run their API servers this way. Neutron doesn't run under as wsgi app (I
don't recall why this was never implemented for neutron) and swift doesn't run
in the py3 jobs at all. You can see an example of this here:

http://logs.openstack.org/08/550108/3/gate/tempest-full-py3/df744ef/controller/logs/

For other services it depends on how they implemented their devstack plugin. I
haven't done an inventory on how all the plugins are running things, so I don't
know what the status of each project is there.

> - Test "python version skew" within a service during a rolling upgrade
>   across multiple hosts.
> - Add an integration test job that does not include python2 on the host
>   at all.
> 
> That last item may block us from using other tools, such as ansible,
> that rely on python2. If the point of such a test is to ensure that
> we are properly installing (and running) our tools under python3,
> maybe *that's* what we want to check, instead of forbidding a python2
> package at all? Could we, for example, look at the set of packages
> installed under python2 and report errors if any OpenStack packages end
> up there?
> 
> > 
> > At that point, all of our deliverables will be produced using python
> > 3, and we can be relatively confident that if we no longer had
> > access to python 2 we could still continue operating. We could also
> > start updating deployment tools to use either python 3 or 2, so
> > that users could actually deploy using the python 3 versions of
> > services.
> > 
> > Somewhere in that time frame our third-party CI systems will need
> > to ensure they have python 3 support as well.
> > 
> > After the "Python 3 first" phase is completed we should release
> > one series using the packages built with python 3. Perhaps Stein?
> > Or is that too ambitious?
> > 
> > Next, we will be ready to address the prerequisites for "Python 3
> > only," which will allow us to drop Python 2 support.
> > 
> > We need to wait to drop python 2 support as a community, rather
> > than going one project at a time, to avoid doubling the work of
> > downstream consumers such as distros and independent deployers. We
> > don't want them to have to package all (or even a large number) of
> > the dependencies of OpenStack twice because they have to install
> > some services running under python 2 and others under 3. Ideally
> > they would be able to upgrade all of the services on a node together
> > as part of their transition to the new version, without ending up
> > with a python 2 version of a dependency along side a python 3 version
> > of the same package.
> > 
> > The remaining items could be fixed earlier, but this is the point
> > at which they would block us:
> > 
> > 1. Fix oslo.service functional tests -- the Oslo team needs help
> >    maintaining this library. Alternatively, we could move all
> >    services to use cotyledon (https://pypi.org/project/cotyledon/).
> > 
> > 2. Finish the unit test and functional test ports so that all of
> >    our tests can run under python 3 (this implies that the services
> >    all run under python 3, so there is no more porting to do).
> > 
> > Finally, after we have *all* tests running on python 3, we can
> > safely drop python 2.
> 
> We clarified that we would only drop python 2 support on master.
> That clarification also raised the point that eventually backports
> may become more difficult if master is using python 3 features, but
> Matt and Tony agreed that we could potentially have rebasing issues
> with fixes today so while this is a new source of such issues it
> isn't a completely new problem and our stable backport policies
> already address it.
> 
> > 
> > We have previously discussed the end of the T cycle as the point
> > at which we would have all of those tests running, and if that holds
> > true we could reasonably drop python 2 during the beginning of the
> > U cycle, in late 2019 and before the 2020 cut-off point when upstream
> > python 2 support will be dropped.
> 
> This date as the earliest point at which existing projects could drop
> python 2 seems to be generally acceptable to everyone. I wrote up
> a TC resolution as a more formal documentation for it:
> 
>   https://review.openstack.org/571011
> 
> I also intend to propose a "python 3 first" goal for Stein, but I
> consider those two things independent.
> 
> The last point of significant interest is that we discussed modifying
> Graham's existing governance change to indicate that new projects
> do not need to have python 2 support with the caveat that platforms
> that do not support python 3 fully, yet, are unlikely to package
> those projects. Graham was going to update the patch, IIRC.
> 
> > 
> > I need some info from the deployment tool teams to understand whether
> > they would be ready to take the plunge during T or U and start
> > deploying only the python 3 version. Are there other upgrade issues
> > that need to be addressed to support moving from 2 to 3? Something
> > that might be part of the platform(s), rather than OpenStack itself?
> > 
> > What else have I missed in these phases? Other jobs? Other blocking
> > conditions?
> > 
> > Doug
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20180529/71f6b12b/attachment.sig>