Now glance is hit with different SQLA related error, We are tracking it here [1]. [1] https://bugs.launchpad.net/glance/+bug/1939716 Thanks & Best Regards, Abhishek Kekane On Fri, Aug 13, 2021 at 5:27 AM Mike Bayer <mike_mp@zzzcomputing.com> wrote:
didn't see the main docs linked so the story starts at SQLA's migration docs.
starting w/ 1.4 which has all of 2.0's internals ready to go:
https://docs.sqlalchemy.org/en/14/changelog/migration_14.html
then 2.0 which includes 1.x->2.0 migration notes:
https://docs.sqlalchemy.org/en/14/changelog/migration_20.html
we spent a lot of time trying to make this process smooth. it has both a few ideas from how py2->py3 worked (warnings mode) and also does some things intentionally the opposite of how py2/3 did it (you can run a codebase that is compatible with 1.4 and 2.0 at the same time).
On Thu, Aug 12, 2021, at 12:00 PM, Stephen Finucane wrote:
tl;dr: If you haven't already started looking at SQLAlchemy 2.0 compatibility and/or are still using sqlalchemy-migrate, you probably have work to do.
As you can guess from $subject, there's a new major version of SQLAlchemy in the pipeline. The good news: it cleans up a lot of cruft and makes a lot of things such as session management more explicit and verbose, with the idea being that this should avoid a lot of gotchas and foot-gun moments. The bad news: the changes are pretty extensive, and virtually anyone that uses SQLAlchemy, which is to say every (?) OpenStack service out there, will likely have to make changes.
This change will likely have a couple of impacts for users:
# Breaking changes in oslo.db
The oslo devs have been steadily working through the many errors that oslo.db is currently emitting when SQLAlchemy 2.0 deprecation warnings are turned on. The patches (which is not yet complete) to address these can be found here [1]. As a result of these changes, a number of oslo.db APIs are likely to start performing slightly differently, and many more may be deprecated and eventually removed. An example of the kind of change you can expect to see can be found in this glance patch [2]. We will work to minimize these changes but some will be unavoidable once sqlalchemy 2.0 actually arrives.
# Breaking changes in your own project
If oslo.db seeing breaking changes because of SQLAlchemy 2.0, you can be sure that you're going to see some too. Fortunately, there are ways you can get ahead of this. Turning SADeprecationWarning warnings into errors for your own module, as we've done for oslo.db [3][4] seems a sensible pattern to weed out these issues.
As an aside, enabling warnings as errors seems to be a generally good idea in a unit test environment. It's usually a lot easier to address these things as they pop up that when things are finally removed and stuff explodes. To each their own though, of course.
# The long prophesied death of sqlalchemy-migrate
The lack of maintenance on sqlalchemy-migrate has already been well documented [5], however, SQLAlchemy 2.0 is likely to be the thing that finally puts the nail in sqlalchemy-migrate's coffin. I've been working on migrating the last few laggards still using sqlalchemy-migrate and have successfully removed all traces of it from glance (thanks, glance-core!), while the nova and cinder patches are making reasonable progress (though I do wish it was faster). The only remaining "core" project to be migrated is keystone. This proved a little too complicated for me to do in the limited time I have to work on these things, and I'm in the middle of a role change in my day job so my time to work on upstream OpenStack will be decreasing much further going forward (though not to zero, I'm hoping). Fortunately, there's quite a bit of prior art available now that people can follow when migrating stuff [6].
Related to the oslo.db changes: everything related to sqlalchemy-migrate in oslo.db should be deprecated by the next oslo.db release, and I suspect we'll be pretty aggressive in pruning these. Based on codesearch.o.o, this will have impacts for at least keystone and cinder.
# A chance to evaluate all things DB
One positive out of this is that the changes necessary may be broad enough to take the opportunity to re-evaluate decisions made regarding your DB layer. We've been doing this in nova, moving lots of things about to clear up the distinction between the main and API DBs and to reduce lots of tech debt that had built up in 'nova.db'. Consider using the opportunity to do the if possible.
---
Hopefully this has been useful. If you have questions about any of the above, please reach out and I'll do my best to answer them.
Cheers, Stephen
[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+(status:open+OR+s... [2] https://review.opendev.org/c/openstack/glance/+/804406 [3] https://github.com/openstack/oslo.db/commit/40bce5a2baf75dc87dd591e0f71a00f2... [4] https://github.com/openstack/oslo.db/commit/4c1eb966c08d29214c1905e74965f410... [5] http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020672.... [6] https://review.opendev.org/q/topic:%22bp%252Fremove-sqlalchemy-migrate%22+(s...)