Open Stack

On Fri, 2021-08-13 at 23:55 +0530, Abhishek Kekane wrote:
> Now glance is hit with different SQLA related error, We are tracking it here
> [1].

This had the same underlying cause as the previous change. I've pushed a patch
to address this [1]

Stephen

[1] https://review.opendev.org/c/openstack/glance/+/804699

> 
> [1] https://bugs.launchpad.net/glance/+bug/1939716
> 
> Thanks & Best Regards,
> 
> Abhishek Kekane
> 
> 
> On Fri, Aug 13, 2021 at 5:27 AM Mike Bayer <mike_mp at zzzcomputing.com> wrote:
> > didn't see the main docs linked so the story starts at SQLA's migration
> > docs.
> > 
> > starting w/ 1.4 which has all of 2.0's internals ready to go:
> > 
> > https://docs.sqlalchemy.org/en/14/changelog/migration_14.html
> > 
> > then 2.0 which includes 1.x->2.0 migration notes:
> > 
> > https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
> > 
> > we spent a lot of time trying to make this process smooth.  it has both a
> > few ideas from how py2->py3 worked (warnings mode) and also does some things
> > intentionally the opposite of how py2/3 did it (you can run a codebase that
> > is compatible with 1.4 and 2.0 at the same time).
> > 
> > 
> > 
> > 
> > On Thu, Aug 12, 2021, at 12:00 PM, Stephen Finucane wrote:
> > > tl;dr: If you haven't already started looking at SQLAlchemy 2.0
> > > compatibility
> > > and/or are still using sqlalchemy-migrate, you probably have work to do.
> > > 
> > > As you can guess from $subject, there's a new major version of SQLAlchemy
> > > in the
> > > pipeline. The good news: it cleans up a lot of cruft and makes a lot of
> > > things
> > > such as session management more explicit and verbose, with the idea being
> > > that
> > > this should avoid a lot of gotchas and foot-gun moments. The bad news: the
> > > changes are pretty extensive, and virtually anyone that uses SQLAlchemy,
> > > which
> > > is to say every (?) OpenStack service out there, will likely have to make
> > > changes.
> > > 
> > > This change will likely have a couple of impacts for users:
> > > 
> > > # Breaking changes in oslo.db
> > > 
> > > The oslo devs have been steadily working through the many errors that
> > > oslo.db is
> > > currently emitting when SQLAlchemy 2.0 deprecation warnings are turned on.
> > > The
> > > patches (which is not yet complete) to address these can be found here
> > > [1]. As a
> > > result of these changes, a number of oslo.db APIs are likely to start
> > > performing
> > > slightly differently, and many more may be deprecated and eventually
> > > removed. An
> > > example of the kind of change you can expect to see can be found in this
> > > glance
> > > patch [2]. We will work to minimize these changes but some will be
> > > unavoidable
> > > once sqlalchemy 2.0 actually arrives.
> > > 
> > > # Breaking changes in your own project
> > > 
> > > If oslo.db seeing breaking changes because of SQLAlchemy 2.0, you can be
> > > sure
> > > that you're going to see some too. Fortunately, there are ways you can get
> > > ahead
> > > of this. Turning SADeprecationWarning warnings into errors for your own
> > > module,
> > > as we've done for oslo.db [3][4] seems a sensible pattern to weed out
> > > these
> > > issues.
> > > 
> > > As an aside, enabling warnings as errors seems to be a generally good idea
> > > in a
> > > unit test environment. It's usually a lot easier to address these things
> > > as they
> > > pop up that when things are finally removed and stuff explodes. To each
> > > their
> > > own though, of course.
> > > 
> > > # The long prophesied death of sqlalchemy-migrate
> > > 
> > > The lack of maintenance on sqlalchemy-migrate has already been well
> > > documented
> > > [5], however, SQLAlchemy 2.0 is likely to be the thing that finally puts
> > > the
> > > nail in sqlalchemy-migrate's coffin. I've been working on migrating the
> > > last few
> > > laggards still using sqlalchemy-migrate and have successfully removed all
> > > traces
> > > of it from glance (thanks, glance-core!), while the nova and cinder
> > > patches are
> > > making reasonable progress (though I do wish it was faster). The only
> > > remaining
> > > "core" project to be migrated is keystone. This proved a little too
> > > complicated
> > > for me to do in the limited time I have to work on these things, and I'm
> > > in the
> > > middle of a role change in my day job so my time to work on upstream
> > > OpenStack
> > > will be decreasing much further going forward (though not to zero, I'm
> > > hoping).
> > > Fortunately, there's quite a bit of prior art available now that people
> > > can
> > > follow when migrating stuff [6].
> > > 
> > > Related to the oslo.db changes: everything related to sqlalchemy-migrate
> > > in
> > > oslo.db should be deprecated by the next oslo.db release, and I suspect
> > > we'll be
> > > pretty aggressive in pruning these. Based on codesearch.o.o, this will
> > > have
> > > impacts for at least keystone and cinder.
> > > 
> > > # A chance to evaluate all things DB
> > > 
> > > One positive out of this is that the changes necessary may be broad enough
> > > to
> > > take the opportunity to re-evaluate decisions made regarding your DB
> > > layer.
> > > We've been doing this in nova, moving lots of things about to clear up the
> > > distinction between the main and API DBs and to reduce lots of tech debt
> > > that
> > > had built up in 'nova.db'. Consider using the opportunity to do the if
> > > possible.
> > > 
> > > ---
> > > 
> > > Hopefully this has been useful. If you have questions about any of the
> > > above,
> > > please reach out and I'll do my best to answer them.
> > > 
> > > Cheers,
> > > Stephen
> > > 
> > > 
> > > [1] 
> > >
> >
> https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+(status:open+OR+status:merged)+project:openstack/oslo.db
> > > [2] https://review.opendev.org/c/openstack/glance/+/804406
> > > [3] 
> > >
> >
> https://github.com/openstack/oslo.db/commit/40bce5a2baf75dc87dd591e0f71a00f221a2ba92
> > > [4] 
> > >
> >
> https://github.com/openstack/oslo.db/commit/4c1eb966c08d29214c1905e74965f4109f41b013
> > > [5] 
> > >
> >
> http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020672.html
> > > [6] 
> > >
> >
> https://review.opendev.org/q/topic:%22bp%252Fremove-sqlalchemy-migrate%22+(status:open%20OR%20status:merged)
> > > 
> > > 
> > > 
> > 
> > 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210816/71ca73f4/attachment-0001.html>