[all] SQLAlchemy 2.x support in Bobcat
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
Cheers,
Thomas Goirand (zigo)
Hi, Neutron projects are tested in Bobcat with sqlalchemy 2 (actually with the main branch), so I would not expect big issues (🤞) (bagpipe was the last project with serious failures and it is green in the last week) My understanding was that Openstackwise we bump sqlalchemy with starting Caracal, see: https://review.opendev.org/c/openstack/requirements/+/879743
BUT: based on some discussions ie: https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.202... I have doubts that it will happen.
Lajos
Thomas Goirand zigo@debian.org ezt Ãrta (idÅ‘pont: 2023. szept. 13., Sze, 10:22):
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
Cheers,
Thomas Goirand (zigo)
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Cheers,
Thomas Goirand (zigo)
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks: - freezer-api - trove - cloudkitty - watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
Cheers,
Thomas Goirand (zigo)
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of SQLAlchemy, Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of the curve as usual and also have their own (which is where I got the idea from). The projects you mention above could do with an equivalent job and my guess is that the process of getting there will highlight quite a bit of work that they need to do. They need to start at that asap (and tbh really should have started at it a long time ago as they've had over 2 years of a warning [5]).
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Cheers,
Thomas Goirand (zigo)
On Fri, 15 Sept 2023 at 13:29, Stephen Finucane stephenfin@redhat.com wrote:
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer
of
SQLAlchemy in Debian, even though OpenStack is the biggest consumer
of
it. The current maintainer insists that he wants to upload SQLA 2.x
in
Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there
yet,
and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to
Bobcat
and maintain in the debian/patches folder of each project. However,
the
biggest current annoyance, is that I have no idea where we are at.
Are
we close to such a support? Is there a list of patches to apply on
top
of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast
majority (though
not all) of the patches are available at the sqlalchemy-20 topic in
Gerrit [1].
I've been working on this for almost 2 years now and have most of the
core
projects well on their way but not everything is complete, as you'll
tell from
that list. I have a canary patch [2] that I've been using to spot
missing
services. I plan to pick up the Manila work again early in C, but
could do with
help removing the use of autocommit in Heat and the weird test
failures I'm
seeing in Cinder [3]. We also need reviews of the Masakri series (Is
that
project dead? I can't tell). Once those are addressed, I _think_ we
might be
done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3]
https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of SQLAlchemy, Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of the curve as usual and also have their own (which is where I got the idea from). The projects you mention above could do with an equivalent job and my guess is that the process of getting there will highlight quite a bit of work that they need to do. They need to start at that asap (and tbh really should have started at it a long time ago as they've had over 2 years of a warning [5]).
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Cheers,
Thomas Goirand (zigo)
Thank you for highlighting cloudkitty. I added it as a discussion item for our meeting next week.
Cheers, Pierre
On 9/15/23 13:26, Stephen Finucane wrote:
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of SQLAlchemy, Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of the curve as usual and also have their own (which is where I got the idea from). The projects you mention above could do with an equivalent job and my guess is that the process of getting there will highlight quite a bit of work that they need to do. They need to start at that asap (and tbh really should have started at it a long time ago as they've had over 2 years of a warning [5]).
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
FYI, I also fell into the trap of castellan 4.2.0. I'm writing this here for others *not* to do the same mistake as me: do not upgrade, or Cinder will fail its unit tests, and only 4.1.0 is the upper bound. Switching back to 4.1.0 made Cinder build correctly for me.
The good thing with castellan is, I forgot to upload it to Experimental, so to the contrary of oslo.db, it's okish for me ... :)
I hope that helps others, Cheers,
Thomas Goirand (zigo)
On Fri, 2023-09-15 at 14:11 +0200, Thomas Goirand wrote:
On 9/15/23 13:26, Stephen Finucane wrote:
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of SQLAlchemy, Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of the curve as usual and also have their own (which is where I got the idea from). The projects you mention above could do with an equivalent job and my guess is that the process of getting there will highlight quite a bit of work that they need to do. They need to start at that asap (and tbh really should have started at it a long time ago as they've had over 2 years of a warning [5]).
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
FYI, I also fell into the trap of castellan 4.2.0. I'm writing this here for others *not* to do the same mistake as me: do not upgrade, or Cinder will fail its unit tests, and only 4.1.0 is the upper bound. Switching back to 4.1.0 made Cinder build correctly for me.
That version bump should have been a major version bump, but it's a pity we had to revert them: the fixes look to have been pretty simple.
https://review.opendev.org/c/openstack/cinder/+/896121 https://review.opendev.org/c/openstack/nova/+/896100
Stephen
The good thing with castellan is, I forgot to upload it to Experimental, so to the contrary of oslo.db, it's okish for me ... :)
I hope that helps others, Cheers,
Thomas Goirand (zigo)
On Fri, 2023-09-15 at 12:26 +0100, Stephen Finucane wrote:
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast majority (though not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1]. I've been working on this for almost 2 years now and have most of the core projects well on their way but not everything is complete, as you'll tell from that list. I have a canary patch [2] that I've been using to spot missing services. I plan to pick up the Manila work again early in C, but could do with help removing the use of autocommit in Heat and the weird test failures I'm seeing in Cinder [3]. We also need reviews of the Masakri series (Is that project dead? I can't tell). Once those are addressed, I _think_ we might be done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of SQLAlchemy, Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of the curve as usual and also have their own (which is where I got the idea from). The projects you mention above could do with an equivalent job and my guess is that the process of getting there will highlight quite a bit of work that they need to do. They need to start at that asap (and tbh really should have started at it a long time ago as they've had over 2 years of a warning [5]).
Just closing this out, we reverted some of the removals of oslo.db functionality that and cut a new 14.1.0 release. The 15.0.0 release will come out early in C and re-remove these features. I'd ask that the outstanding projects that are still facing issues with oslo.db merge the patches to address these issues asap.
Heat: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/heat (note: There are intermittent failures due to random test order causing duplicate keys. Will be fixed shortly.)
Aodh: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/aodh (note: Gate is broken. Have proposed a temporary fix [1])
Masakari: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/masakari
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/aodh/+/896125
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Cheers,
Thomas Goirand (zigo)
Hello Stephen,
Is it still the plan to release oslo.db 15.0.0 without these features or has this moved to a future release?
Thanks, Pierre Riteau (priteau)
On Thu, 21 Sept 2023 at 16:55, Stephen Finucane stephenfin@redhat.com wrote:
On Fri, 2023-09-15 at 12:26 +0100, Stephen Finucane wrote:
On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
On 9/13/23 15:39, Stephen Finucane wrote:
On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
Hi,
As you may know, and to my great frustration, I'm not the
maintainer of
SQLAlchemy in Debian, even though OpenStack is the biggest
consumer of
it. The current maintainer insists that he wants to upload SQLA
2.x in
Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there
yet,
and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to
Bobcat
and maintain in the debian/patches folder of each project.
However, the
biggest current annoyance, is that I have no idea where we are at.
Are
we close to such a support? Is there a list of patches to apply on
top
of Bobcat that is maintained somewhere?
Please enlighten me... :)
I think you figured this out on IRC this morning, but the vast
majority (though
not all) of the patches are available at the sqlalchemy-20 topic in
Gerrit [1].
I've been working on this for almost 2 years now and have most of
the core
projects well on their way but not everything is complete, as you'll
tell from
that list. I have a canary patch [2] that I've been using to spot
missing
services. I plan to pick up the Manila work again early in C, but
could do with
help removing the use of autocommit in Heat and the weird test
failures I'm
seeing in Cinder [3]. We also need reviews of the Masakri series (Is
that
project dead? I can't tell). Once those are addressed, I _think_ we
might be
done but who knows what else we'll find...
Cheers, Stephen
[1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open [2] https://review.opendev.org/c/openstack/requirements/+/879743 [3]
https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/c...
Thanks for your work, really! And thanks for the details above.
Now, not sure how much this is related, but I've seen that the new oslo.db 14.0.0 breaks:
- freezer-api
- trove
- cloudkitty
- watcher
Is there any plan to fix oslo.db, or the above projects? Maybe revert some commits in oslo.db? Can someone explain to me what's going on for this as well?
oslo.db has intentionally *not* been bumped to >= 13.0.0 in
upper-constraints
because it introduces many breaking changes that are not compatible with SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible
with
SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various
projects
have been fixed which is the same as what we're seeing with SQLAlchemy
2.x.
Fortunately projects that adopt to I have been pushing patches to various projects that add a "tips" job for testing main/master branches of
SQLAlchemy,
Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well
ahead of
the curve as usual and also have their own (which is where I got the
idea from).
The projects you mention above could do with an equivalent job and my
guess is
that the process of getting there will highlight quite a bit of work
that they
need to do. They need to start at that asap (and tbh really should have
started
at it a long time ago as they've had over 2 years of a warning [5]).
Just closing this out, we reverted some of the removals of oslo.db functionality that and cut a new 14.1.0 release. The 15.0.0 release will come out early in C and re-remove these features. I'd ask that the outstanding projects that are still facing issues with oslo.db merge the patches to address these issues asap.
Heat: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/heat (note: There are intermittent failures due to random test order causing duplicate keys. Will be fixed shortly.)
Aodh: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/aodh (note: Gate is broken. Have proposed a temporary fix [1])
Masakari: https://review.opendev.org/q/topic:sqlalchemy-20+project:openstack/masakari
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/aodh/+/896125
Cheers, Stephen
[1] https://review.opendev.org/c/openstack/barbican/+/888308 [2] https://review.opendev.org/c/openstack/placement/+/886229 [3] https://review.opendev.org/c/openstack/cinder/+/886152 [4] https://review.opendev.org/c/openstack/glance/+/889066 [5]
https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.h...
Cheers,
Thomas Goirand (zigo)
Hello, I did Gnocchi recently and it’s available in the new 4.6.0 release.
I don’t envy you if you have to backport every single patch, those are hundred of patches layered on-top of each other and then probably some standalone fixes here and there across the entire ecosystem.
I hope you will fight the maintainer because that doesn’t sound fun :(
Best regards Tobias
On 13 Sep 2023, at 09:21, Thomas Goirand zigo@debian.org wrote:
Hi,
As you may know, and to my great frustration, I'm not the maintainer of SQLAlchemy in Debian, even though OpenStack is the biggest consumer of it. The current maintainer insists that he wants to upload SQLA 2.x in Unstable, potentially breaking all of OpenStack.
At the present moment, if I understand correctly, we're not there yet, and Bobcat doesn't have such a support. It would be ok for me, *IF* there are patches available on master, that I could backport to Bobcat and maintain in the debian/patches folder of each project. However, the biggest current annoyance, is that I have no idea where we are at. Are we close to such a support? Is there a list of patches to apply on top of Bobcat that is maintained somewhere?
Please enlighten me... :)
Cheers,
Thomas Goirand (zigo)
participants (5)
-
Lajos Katona
-
Pierre Riteau
-
Stephen Finucane
-
Thomas Goirand
-
Tobias Urdin