Open Stack

didn't see the main docs linked so the story starts at SQLA's migration docs.

starting w/ 1.4 which has all of 2.0's internals ready to go:

https://docs.sqlalchemy.org/en/14/changelog/migration_14.html

then 2.0 which includes 1.x->2.0 migration notes:

https://docs.sqlalchemy.org/en/14/changelog/migration_20.html

we spent a lot of time trying to make this process smooth.  it has both a few ideas from how py2->py3 worked (warnings mode) and also does some things intentionally the opposite of how py2/3 did it (you can run a codebase that is compatible with 1.4 and 2.0 at the same time).




On Thu, Aug 12, 2021, at 12:00 PM, Stephen Finucane wrote:
> tl;dr: If you haven't already started looking at SQLAlchemy 2.0 compatibility
> and/or are still using sqlalchemy-migrate, you probably have work to do.
> 
> As you can guess from $subject, there's a new major version of SQLAlchemy in the
> pipeline. The good news: it cleans up a lot of cruft and makes a lot of things
> such as session management more explicit and verbose, with the idea being that
> this should avoid a lot of gotchas and foot-gun moments. The bad news: the
> changes are pretty extensive, and virtually anyone that uses SQLAlchemy, which
> is to say every (?) OpenStack service out there, will likely have to make
> changes.
> 
> This change will likely have a couple of impacts for users:
> 
> # Breaking changes in oslo.db
> 
> The oslo devs have been steadily working through the many errors that oslo.db is
> currently emitting when SQLAlchemy 2.0 deprecation warnings are turned on. The
> patches (which is not yet complete) to address these can be found here [1]. As a
> result of these changes, a number of oslo.db APIs are likely to start performing
> slightly differently, and many more may be deprecated and eventually removed. An
> example of the kind of change you can expect to see can be found in this glance
> patch [2]. We will work to minimize these changes but some will be unavoidable
> once sqlalchemy 2.0 actually arrives.
> 
> # Breaking changes in your own project
> 
> If oslo.db seeing breaking changes because of SQLAlchemy 2.0, you can be sure
> that you're going to see some too. Fortunately, there are ways you can get ahead
> of this. Turning SADeprecationWarning warnings into errors for your own module,
> as we've done for oslo.db [3][4] seems a sensible pattern to weed out these
> issues.
> 
> As an aside, enabling warnings as errors seems to be a generally good idea in a
> unit test environment. It's usually a lot easier to address these things as they
> pop up that when things are finally removed and stuff explodes. To each their
> own though, of course.
> 
> # The long prophesied death of sqlalchemy-migrate
> 
> The lack of maintenance on sqlalchemy-migrate has already been well documented
> [5], however, SQLAlchemy 2.0 is likely to be the thing that finally puts the
> nail in sqlalchemy-migrate's coffin. I've been working on migrating the last few
> laggards still using sqlalchemy-migrate and have successfully removed all traces
> of it from glance (thanks, glance-core!), while the nova and cinder patches are
> making reasonable progress (though I do wish it was faster). The only remaining
> "core" project to be migrated is keystone. This proved a little too complicated
> for me to do in the limited time I have to work on these things, and I'm in the
> middle of a role change in my day job so my time to work on upstream OpenStack
> will be decreasing much further going forward (though not to zero, I'm hoping).
> Fortunately, there's quite a bit of prior art available now that people can
> follow when migrating stuff [6].
> 
> Related to the oslo.db changes: everything related to sqlalchemy-migrate in
> oslo.db should be deprecated by the next oslo.db release, and I suspect we'll be
> pretty aggressive in pruning these. Based on codesearch.o.o, this will have
> impacts for at least keystone and cinder.
> 
> # A chance to evaluate all things DB
> 
> One positive out of this is that the changes necessary may be broad enough to
> take the opportunity to re-evaluate decisions made regarding your DB layer.
> We've been doing this in nova, moving lots of things about to clear up the
> distinction between the main and API DBs and to reduce lots of tech debt that
> had built up in 'nova.db'. Consider using the opportunity to do the if possible.
> 
> ---
> 
> Hopefully this has been useful. If you have questions about any of the above,
> please reach out and I'll do my best to answer them.
> 
> Cheers,
> Stephen
> 
> 
> [1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+(status:open+OR+status:merged)+project:openstack/oslo.db
> [2] https://review.opendev.org/c/openstack/glance/+/804406
> [3] https://github.com/openstack/oslo.db/commit/40bce5a2baf75dc87dd591e0f71a00f221a2ba92
> [4] https://github.com/openstack/oslo.db/commit/4c1eb966c08d29214c1905e74965f4109f41b013
> [5] http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020672.html
> [6] https://review.opendev.org/q/topic:%22bp%252Fremove-sqlalchemy-migrate%22+(status:open%20OR%20status:merged)
> 
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210812/fcb31564/attachment-0001.html>