Open Stack

On Fri, 2023-09-15 at 14:11 +0200, Thomas Goirand wrote:
> On 9/15/23 13:26, Stephen Finucane wrote:
> > On Fri, 2023-09-15 at 13:07 +0200, Thomas Goirand wrote:
> > > On 9/13/23 15:39, Stephen Finucane wrote:
> > > > On Wed, 2023-09-13 at 09:21 +0200, Thomas Goirand wrote:
> > > > > Hi,
> > > > > 
> > > > > As you may know, and to my great frustration, I'm not the maintainer of
> > > > > SQLAlchemy in Debian, even though OpenStack is the biggest consumer of
> > > > > it. The current maintainer insists that he wants to upload SQLA 2.x in
> > > > > Unstable, potentially breaking all of OpenStack.
> > > > > 
> > > > > At the present moment, if I understand correctly, we're not there yet,
> > > > > and Bobcat doesn't have such a support. It would be ok for me, *IF*
> > > > > there are patches available on master, that I could backport to Bobcat
> > > > > and maintain in the debian/patches folder of each project. However, the
> > > > > biggest current annoyance, is that I have no idea where we are at. Are
> > > > > we close to such a support? Is there a list of patches to apply on top
> > > > > of Bobcat that is maintained somewhere?
> > > > > 
> > > > > Please enlighten me... :)
> > > > 
> > > > I think you figured this out on IRC this morning, but the vast majority (though
> > > > not all) of the patches are available at the sqlalchemy-20 topic in Gerrit [1].
> > > > I've been working on this for almost 2 years now and have most of the core
> > > > projects well on their way but not everything is complete, as you'll tell from
> > > > that list. I have a canary patch [2] that I've been using to spot missing
> > > > services. I plan to pick up the Manila work again early in C, but could do with
> > > > help removing the use of autocommit in Heat and the weird test failures I'm
> > > > seeing in Cinder [3]. We also need reviews of the Masakri series (Is that
> > > > project dead? I can't tell). Once those are addressed, I _think_ we might be
> > > > done but who knows what else we'll find...
> > > > 
> > > > Cheers,
> > > > Stephen
> > > > 
> > > > [1] https://review.opendev.org/q/topic:sqlalchemy-20+is:open
> > > > [2] https://review.opendev.org/c/openstack/requirements/+/879743
> > > > [3] https://review.opendev.org/q/topic:sqlalchemy-20+is:open+project:openstack/cinder
> > > 
> > > Thanks for your work, really!
> > > And thanks for the details above.
> > > 
> > > Now, not sure how much this is related, but I've seen that the new
> > > oslo.db 14.0.0 breaks:
> > > - freezer-api
> > > - trove
> > > - cloudkitty
> > > - watcher
> > > 
> > > Is there any plan to fix oslo.db, or the above projects? Maybe revert
> > > some commits in oslo.db? Can someone explain to me what's going on for
> > > this as well?
> > 
> > oslo.db has intentionally *not* been bumped to >= 13.0.0 in upper-constraints
> > because it introduces many breaking changes that are not compatible with
> > SQLAlchemy 2.x. Conversely, oslo.db < 13.0.0 is simply not compatible with
> > SQLAlchemy >= 2.x. As such, we can't bump oslo.db until the various projects
> > have been fixed which is the same as what we're seeing with SQLAlchemy 2.x.
> > 
> > Fortunately projects that adopt to I have been pushing patches to various
> > projects that add a "tips" job for testing main/master branches of SQLAlchemy,
> > Alembic, and oslo.db, e.g. [1][2][3][4]. The Neutron folks were well ahead of
> > the curve as usual and also have their own (which is where I got the idea from).
> > The projects you mention above could do with an equivalent job and my guess is
> > that the process of getting there will highlight quite a bit of work that they
> > need to do. They need to start at that asap (and tbh really should have started
> > at it a long time ago as they've had over 2 years of a warning [5]).
> > 
> > Cheers,
> > Stephen
> > 
> > [1] https://review.opendev.org/c/openstack/barbican/+/888308
> > [2] https://review.opendev.org/c/openstack/placement/+/886229
> > [3] https://review.opendev.org/c/openstack/cinder/+/886152
> > [4] https://review.opendev.org/c/openstack/glance/+/889066
> > [5] https://lists.openstack.org/pipermail/openstack-discuss/2021-August/024122.html
> 
> FYI, I also fell into the trap of castellan 4.2.0. I'm writing this here 
> for others *not* to do the same mistake as me: do not upgrade, or Cinder 
> will fail its unit tests, and only 4.1.0 is the upper bound. Switching 
> back to 4.1.0 made Cinder build correctly for me.

That version bump should have been a major version bump, but it's a pity we had
to revert them: the fixes look to have been pretty simple.

https://review.opendev.org/c/openstack/cinder/+/896121
https://review.opendev.org/c/openstack/nova/+/896100

Stephen

> 
> The good thing with castellan is, I forgot to upload it to Experimental, 
> so to the contrary of oslo.db, it's okish for me ... :)
> 
> I hope that helps others,
> Cheers,
> 
> Thomas Goirand (zigo)
> 
>