[all] Supporting SQLAlchemy 2.0 in OpenStack
Hi, SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed? Cheers, Thomas Goirand (zigo)
Hi, There is a requirement bump patch with sqlalchemy 2 included: https://review.opendev.org/c/openstack/requirements/+/872065 The job results give a high level view of the current state, I suppose a lot of failures are easy to fix, but yeah a plan would be good. In Neutron and projects around Neutron, Rodolfo started to add jobs to have a picture what we have to fix: https://review.opendev.org/c/openstack/neutron/+/872273 Lajos (lajoskatona) Thomas Goirand <zigo@debian.org> ezt írta (időpont: 2023. febr. 1., Sze, 14:10):
Hi,
SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed?
Cheers,
Thomas Goirand (zigo)
We started talking about SQLAlchemy 2.0 in Openstack most officially in August 2021: https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h... Since then, lots of work has been done for SQLAlchemy 2.0 compatibility. oslo.db master IIUC should be fully free of 2.0 deprecation warnings, this is the topic for that: https://review.opendev.org/q/topic:sqlalchemy-20+repo:openstack/oslo.db however, sqlalchemy-migrate would need to be fully retired from it which hasn't happened yet. For downstream projects, the vast majority of SQLAlchemy-related code, which is lots of session.query() calls, should not have to change except to the degree that string-based notation for object attributes is removed in favor of actual attributes, e.g. joinedload("networks") becomes joinedload(Machine.networks). The SQLAlchemy 2.0 migration process is designed to be gradual and starts with building a 2.0 forwards-compliant application that runs under SQLAlchemy 1.4. This process is documented at https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#the-1-4-2-0-mi... . I think a lot of projects are already through this process and are now trying to run directly on the 2.0 release. On Wed, Feb 1, 2023, at 8:02 AM, Thomas Goirand wrote:
Hi,
SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed?
Cheers,
Thomas Goirand (zigo)
On Wed, 2023-02-01 at 08:58 -0500, Mike Bayer wrote:
We started talking about SQLAlchemy 2.0 in Openstack most officially in August 2021:
https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Since then, lots of work has been done for SQLAlchemy 2.0 compatibility. oslo.db master IIUC should be fully free of 2.0 deprecation warnings, this is the topic for that: https://review.opendev.org/q/topic:sqlalchemy-20+repo:openstack/oslo.db however, sqlalchemy-migrate would need to be fully retired from it which hasn't happened yet. stephen who is alredy on cc has been driving most of the work to remvoign sqlalchmey-migrate nova and several other project move to alembic thanks to there work in zed.
i would think moving nova and placment at least to SQLAlchemy 2.0 in bobcat would not be contoverital given the work that stephen and other have already been doing. this would be good perhasp as a comunity goal
For downstream projects, the vast majority of SQLAlchemy-related code, which is lots of session.query() calls, should not have to change except to the degree that string-based notation for object attributes is removed in favor of actual attributes, e.g. joinedload("networks") becomes joinedload(Machine.networks).
The SQLAlchemy 2.0 migration process is designed to be gradual and starts with building a 2.0 forwards-compliant application that runs under SQLAlchemy 1.4. This process is documented at https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#the-1-4-2-0-mi... . I think a lot of projects are already through this process and are now trying to run directly on the 2.0 release.
ya if we have nto got there with nova/placment yet i think it something we can push for in the next release we do have the deprecation warnign mostly turn on and we try to fix them when they happen so its proably time to actully complete the move.
On Wed, Feb 1, 2023, at 8:02 AM, Thomas Goirand wrote:
Hi,
SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed?
Cheers,
Thomas Goirand (zigo)
Thanks for bringing this topic up! As a release team member I tend to agree with Sean that this is a good candidate to be a community goal of 2023.2 Bobcat as we are too late to add this to 2023.1 Antelope release (given that in exactly 2 weeks we will have Antelope-3 milestone, which is Feature Freeze date). Also, we had a good example with the same thing in Zed in the past, where a last minute change caused some trouble (and resulted conversations like this [1]) and people agreed to avoid this in the future if possible. So I think the best is to *NOT* accept Sqlalchemy 2.0.0 in this cycle (2023.1 Antelope). Thanks, Előd Illés irc: elodilles @ #openstack-release [1] https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030914.... ________________________________ From: Sean Mooney <smooney@redhat.com> Sent: Wednesday, February 1, 2023 5:50 PM To: Mike Bayer <mike_mp@zzzcomputing.com>; openstack-discuss@lists.openstack.org <openstack-discuss@lists.openstack.org> Cc: stephenfin@redhat.com <stephenfin@redhat.com> Subject: Re: [all] Supporting SQLAlchemy 2.0 in OpenStack On Wed, 2023-02-01 at 08:58 -0500, Mike Bayer wrote:
We started talking about SQLAlchemy 2.0 in Openstack most officially in August 2021:
https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Since then, lots of work has been done for SQLAlchemy 2.0 compatibility. oslo.db master IIUC should be fully free of 2.0 deprecation warnings, this is the topic for that: https://review.opendev.org/q/topic:sqlalchemy-20+repo:openstack/oslo.db however, sqlalchemy-migrate would need to be fully retired from it which hasn't happened yet. stephen who is alredy on cc has been driving most of the work to remvoign sqlalchmey-migrate nova and several other project move to alembic thanks to there work in zed.
i would think moving nova and placment at least to SQLAlchemy 2.0 in bobcat would not be contoverital given the work that stephen and other have already been doing. this would be good perhasp as a comunity goal
For downstream projects, the vast majority of SQLAlchemy-related code, which is lots of session.query() calls, should not have to change except to the degree that string-based notation for object attributes is removed in favor of actual attributes, e.g. joinedload("networks") becomes joinedload(Machine.networks).
The SQLAlchemy 2.0 migration process is designed to be gradual and starts with building a 2.0 forwards-compliant application that runs under SQLAlchemy 1.4. This process is documented at https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#the-1-4-2-0-mi... . I think a lot of projects are already through this process and are now trying to run directly on the 2.0 release.
ya if we have nto got there with nova/placment yet i think it something we can push for in the next release we do have the deprecation warnign mostly turn on and we try to fix them when they happen so its proably time to actully complete the move.
On Wed, Feb 1, 2023, at 8:02 AM, Thomas Goirand wrote:
Hi,
SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed?
Cheers,
Thomas Goirand (zigo)
I would absolutely agree that Openstack should not be in a hurry to actually *ship* with SQLAlchemy 2.0. 2.0 just came out, the release is going great, but Openstack would benefit from getting in the 2.0 game for real several months from now when the dust has settled. as far as getting *ready* for SQLAlchemy 2.0 and adding 2.0 runs to testing (at least non-voting testing), that should be happening for all projects right now. the biggest task here would be getting rid of sqlalchemy-migrate in all projects and dropping it from oslo.db. On Thu, Feb 2, 2023, at 11:03 AM, Elõd Illés wrote:
Thanks for bringing this topic up! As a release team member I tend to agree with Sean that this is a good candidate to be a community goal of 2023.2 Bobcat as we are too late to add this to 2023.1 Antelope release (given that in exactly 2 weeks we will have Antelope-3 milestone, which is Feature Freeze date).
Also, we had a good example with the same thing in Zed in the past, where a last minute change caused some trouble (and resulted conversations like this [1]) and people agreed to avoid this in the future if possible. So I think the best is to *NOT* accept Sqlalchemy 2.0.0 in this cycle (2023.1 Antelope).
Thanks,
Előd Illés irc: elodilles @ #openstack-release
[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-October/030914....
*From:* Sean Mooney <smooney@redhat.com> *Sent:* Wednesday, February 1, 2023 5:50 PM *To:* Mike Bayer <mike_mp@zzzcomputing.com>; openstack-discuss@lists.openstack.org <openstack-discuss@lists.openstack.org> *Cc:* stephenfin@redhat.com <stephenfin@redhat.com> *Subject:* Re: [all] Supporting SQLAlchemy 2.0 in OpenStack
On Wed, 2023-02-01 at 08:58 -0500, Mike Bayer wrote:
We started talking about SQLAlchemy 2.0 in Openstack most officially in August 2021:
https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Since then, lots of work has been done for SQLAlchemy 2.0 compatibility. oslo.db master IIUC should be fully free of 2.0 deprecation warnings, this is the topic for that: https://review.opendev.org/q/topic:sqlalchemy-20+repo:openstack/oslo.db however, sqlalchemy-migrate would need to be fully retired from it which hasn't happened yet. stephen who is alredy on cc has been driving most of the work to remvoign sqlalchmey-migrate nova and several other project move to alembic thanks to there work in zed.
i would think moving nova and placment at least to SQLAlchemy 2.0 in bobcat would not be contoverital given the work that stephen and other have already been doing.
this would be good perhasp as a comunity goal
For downstream projects, the vast majority of SQLAlchemy-related code, which is lots of session.query() calls, should not have to change except to the degree that string-based notation for object attributes is removed in favor of actual attributes, e.g. joinedload("networks") becomes joinedload(Machine.networks).
The SQLAlchemy 2.0 migration process is designed to be gradual and starts with building a 2.0 forwards-compliant application that runs under SQLAlchemy 1.4. This process is documented at https://docs.sqlalchemy.org/en/20/changelog/migration_20.html#the-1-4-2-0-mi... . I think a lot of projects are already through this process and are now trying to run directly on the 2.0 release.
ya if we have nto got there with nova/placment yet i think it something we can push for in the next release we do have the deprecation warnign mostly turn on and we try to fix them when they happen so its proably time to actully complete the move.
On Wed, Feb 1, 2023, at 8:02 AM, Thomas Goirand wrote:
Hi,
SQLAlchemy 2.0 was released 5 days ago. IMO, we should start drawing a plan to support it for Bobcat, and possibly backport patches for Antelope. Note that I have no clue what his implies. Maybe Mike, you could write about it? Has a lot of things changed?
Cheers,
Thomas Goirand (zigo)
On 2023-02-02 16:03:01 +0000 (+0000), Elõd Illés wrote: [...]
this is a good candidate to be a community goal of 2023.2 Bobcat [...]
Just a minor point of clarification, be aware that the TC has changed the goal framework such that goals are no longer necessarily tied to any particular development cycle or release: "OpenStack-wide goals are not tied to any release, [but] they can have target dates which can be a particular cycle release date" https://governance.openstack.org/tc/goals/index.html#selecting-goals So you'll want to propose a goal and set, I dunno, probably milestone 3 or release candidate week as the target date. Something like that. -- Jeremy Stanley
participants (6)
-
Elõd Illés
-
Jeremy Stanley
-
Lajos Katona
-
Mike Bayer
-
Sean Mooney
-
Thomas Goirand