Open Stack

Excerpts from Jim Rollenhagen's message of 2018-09-13 12:08:08 -0600:
> On Wed, Sep 12, 2018 at 2:28 PM, Doug Hellmann <doug at doughellmann.com>
> wrote:
> 
> > Excerpts from Doug Hellmann's message of 2018-09-12 12:04:02 -0600:
> > > Excerpts from Clark Boylan's message of 2018-09-12 10:44:55 -0700:
> > > > On Wed, Sep 12, 2018, at 10:23 AM, Jim Rollenhagen wrote:
> > > > > The process of operators upgrading Python versions across their
> > fleet came
> > > > > up this morning. It's fairly obvious that operators will want to do
> > this in
> > > > > a rolling fashion.
> > > > >
> > > > > Has anyone considered doing this in CI? For example, running
> > multinode
> > > > > grenade with python 2 on one node and python 3 on the other node.
> > > > >
> > > > > Should we (openstack) test this situation, or even care?
> > > > >
> > > >
> > > > This came up in a Vancouver summit session (the python3 one I think).
> > General consensus there seemed to be that we should have grenade jobs that
> > run python2 on the old side and python3 on the new side and test the update
> > from one to another through a release that way. Additionally there was
> > thought that the nova partial job (and similar grenade jobs) could hold the
> > non upgraded node on python2 and that would talk to a python3 control plane.
> > > >
> > > > I haven't seen or heard of anyone working on this yet though.
> > > >
> > > > Clark
> > > >
> > >
> > > IIRC, we also talked about not supporting multiple versions of
> > > python on a given node, so all of the services on a node would need
> > > to be upgraded together.
> > >
> > > Doug
> >
> > I spent a little time talking with the QA team about setting up
> > this job, and Attila pointed out that we should think about what
> > exactly we think would break during a 2-to-3 in-place upgrade like
> > this.
> >
> > Keeping in mind that we are still testing initial installation under
> > both versions and upgrades under python 2, do we have any specific
> > concerns about the python *version* causing upgrade issues?
> >
> 
> A specific example brought up in the ironic room was the way we encode
> exceptions in oslo.messaging for transmitting over RPC. I know that we've
> found encoding bugs in that in the past, and one can imagine that RPC
> between a service running on py2 and a service running on py3 could have
> similar issues.

Mixing python 2 and 3 components of the same service across nodes
does seem like an interesting case. I wonder if it's something we
could build a functional test job in oslo.messaging for, though,
without having to test every service separately. I'd be happy if
someone did that.

> It's definitely edge cases that we'd be catching here (if any), so I'm
> personally fine with assuming it will just work. But I wanted to pose the
> question to the list, as we agreed this isn't only an ironic problem.

Yes, definitely. I think it's likely to be a bit of work to set up the
jobs and run them for all services, which is why I'm trying to
understand if it's really needed. Thinking through the cases on the list
is a good way to get folks to poke holes in any assertions, so I
appreciate that you started the thread and that everyone is
participating.

Doug