On Wed, 2022-06-15 at 00:47 +0200, Thomas Goirand wrote:

On 6/13/22 00:10, Thomas Goirand wrote:

...

Hi,

A few DDs are pushing me to upgrade jsonschema in Debian Unstable. However, OpenStack global requirements are still stuck at 3.2.0. Is there any reason for it, or should we attempt to upgrade to 4.6.0?

I'd really appreciate if someone (else than me) was driving this...

Cheers,

Thomas Goirand (zigo)

FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz

Can someone from the Nova team investigate?

nova is not pinning this https://github.com/openstack/nova/blob/master/requirements.txt#L25=

we test with whatever is allowed by uppper constriats so we need to look a this form a requirements team point of view

anyjson seams to deplped on use2to3 which is breaking the script that updates requiremtns for me locally.

if i remvoe that temporally and run

.venv/bin/generate-constraints -b blacklist.txt -p python3.8 -r global-requirements.txt > upper-constraints.txt i get futher but then fail because i dont have all the bindep of every python lib in gloal-requiremetns install on my laptop

ill try and check that again in a devstack vm which should have many of those deps installed.

but something seams to be broken in the automation that updates the upper-constratis. the zuul periodic is at least sometime passing

https://zuul.openstack.org/builds?job_name=propose-updates&project=opens...

but im not sure how many old deps we have and why the requiremnts repo is still pinning to 3.2.0 form november 2019

Cheers,

Thomas Goirand (zigo)

On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote:

---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand zigo@debian.org wrote ----

...

On 6/13/22 00:10, Thomas Goirand wrote:

...

Hi,

A few DDs are pushing me to upgrade jsonschema in Debian Unstable. However, OpenStack global requirements are still stuck at 3.2.0. Is there any reason for it, or should we attempt to upgrade to 4.6.0?

I'd really appreciate if someone (else than me) was driving this...

Cheers,

Thomas Goirand (zigo)

FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz

Can someone from the Nova team investigate?

Nova failures are due to the error message change (it happens regularly and they change them in most versions) in jsonschema new version. I remember we faced this type of issue previously also and we updated nova tests not to assert the error message but seems like there are a few left which is failing with jsonschema 4.6.0.

Along with these error message failures fixes and using the latest jsonschema, in nova, we use Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check what are backward-incompatible changes in Draft7Validator and bump this too?

well i think the reason this is on 3.2.0 iis nothing to do with nova i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/

The conflict is caused by: The user requested jsonschema===4.0.1 tripleo-common 16.4.0 depends on jsonschema>=3.2.0 rsd-lib 1.2.0 depends on jsonschema>=2.6.0 taskflow 4.7.0 depends on jsonschema>=3.2.0 zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 os-net-config 15.2.0 depends on jsonschema>=3.2.0 task-core 0.2.1 depends on jsonschema>=3.2.0 python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 warlock 1.3.3 depends on jsonschema<4 and >=0.7

https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9

it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0

-gmann

...

Cheers,

Thomas Goirand (zigo)

On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote:

On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote:

...

---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand zigo@debian.org wrote ----

...

On 6/13/22 00:10, Thomas Goirand wrote:

...

Hi,

A few DDs are pushing me to upgrade jsonschema in Debian Unstable. However, OpenStack global requirements are still stuck at 3.2.0. Is there any reason for it, or should we attempt to upgrade to 4.6.0?

I'd really appreciate if someone (else than me) was driving this...

Cheers,

Thomas Goirand (zigo)

FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz

Can someone from the Nova team investigate?

Nova failures are due to the error message change (it happens regularly and they change them in most versions) in jsonschema new version. I remember we faced this type of issue previously also and we updated nova tests not to assert the error message but seems like there are a few left which is failing with jsonschema 4.6.0.

Along with these error message failures fixes and using the latest jsonschema, in nova, we use Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check what are backward-incompatible changes in Draft7Validator and bump this too?

well i think the reason this is on 3.2.0 iis nothing to do with nova i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/

The conflict is caused by: The user requested jsonschema===4.0.1 tripleo-common 16.4.0 depends on jsonschema>=3.2.0 rsd-lib 1.2.0 depends on jsonschema>=2.6.0 taskflow 4.7.0 depends on jsonschema>=3.2.0 zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 os-net-config 15.2.0 depends on jsonschema>=3.2.0 task-core 0.2.1 depends on jsonschema>=3.2.0 python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 warlock 1.3.3 depends on jsonschema<4 and >=0.7

https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9

it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0

glance client seams to be the only real user of this https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&fi... perhaps we could jsut remove the dependcy?

...

-gmann

...

Cheers,

Thomas Goirand (zigo)

On Wed, 2022-06-15 at 01:04 +0100, Sean Mooney wrote:

On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote:

...

On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote:

...

---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand zigo@debian.org wrote ----

...

On 6/13/22 00:10, Thomas Goirand wrote:

...

Hi,

A few DDs are pushing me to upgrade jsonschema in Debian Unstable. However, OpenStack global requirements are still stuck at 3.2.0. Is there any reason for it, or should we attempt to upgrade to 4.6.0?

I'd really appreciate if someone (else than me) was driving this...

Cheers,

Thomas Goirand (zigo)

FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz

Can someone from the Nova team investigate?

Nova failures are due to the error message change (it happens regularly and they change them in most versions) in jsonschema new version. I remember we faced this type of issue previously also and we updated nova tests not to assert the error message but seems like there are a few left which is failing with jsonschema 4.6.0.

Along with these error message failures fixes and using the latest jsonschema, in nova, we use Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check what are backward-incompatible changes in Draft7Validator and bump this too?

well i think the reason this is on 3.2.0 iis nothing to do with nova i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/

The conflict is caused by: The user requested jsonschema===4.0.1 tripleo-common 16.4.0 depends on jsonschema>=3.2.0 rsd-lib 1.2.0 depends on jsonschema>=2.6.0 taskflow 4.7.0 depends on jsonschema>=3.2.0 zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 os-net-config 15.2.0 depends on jsonschema>=3.2.0 task-core 0.2.1 depends on jsonschema>=3.2.0 python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 warlock 1.3.3 depends on jsonschema<4 and >=0.7

https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9

it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0

glance client seams to be the only real user of this https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&fi... perhaps we could jsut remove the dependcy?

I've proposed vendoring this dependency in glanceclient [1]. I've also proposed a change to fix this in warlock [2] but given the lack of activity there, I doubt it'll merge anytime soon so the former sounds like a better option.

Cheers, Stephen

[1] https://review.opendev.org/c/openstack/python-glanceclient/+/846197 [2] https://github.com/bcwaldon/warlock/pull/65

...

...

-gmann

...

Cheers,

Thomas Goirand (zigo)

On 6/16/22 19:53, Stephen Finucane wrote:

On Thu, 2022-06-16 at 17:13 +0100, Stephen Finucane wrote:

...

On Wed, 2022-06-15 at 01:04 +0100, Sean Mooney wrote:

...

On Wed, 2022-06-15 at 00:58 +0100, Sean Mooney wrote:

...

On Tue, 2022-06-14 at 18:49 -0500, Ghanshyam Mann wrote:

...

---- On Tue, 14 Jun 2022 17:47:59 -0500 Thomas Goirand zigo@debian.org wrote ----

...

On 6/13/22 00:10, Thomas Goirand wrote: > Hi, > > A few DDs are pushing me to upgrade jsonschema in Debian Unstable. > However, OpenStack global requirements are still stuck at 3.2.0. Is > there any reason for it, or should we attempt to upgrade to 4.6.0? > > I'd really appreciate if someone (else than me) was driving this... > > Cheers, > > Thomas Goirand (zigo) >

FYI, Nova fails with it: https://ci.debian.net/data/autopkgtest/unstable/amd64/n/nova/22676760/log.gz

Can someone from the Nova team investigate?

Nova failures are due to the error message change (it happens regularly and they change them in most versions) in jsonschema new version. I remember we faced this type of issue previously also and we updated nova tests not to assert the error message but seems like there are a few left which is failing with jsonschema 4.6.0.

Along with these error message failures fixes and using the latest jsonschema, in nova, we use Draft4Validator from jsonschema and the latest validator is Draft7Validator so we should check what are backward-incompatible changes in Draft7Validator and bump this too?

well i think the reason this is on 3.2.0 iis nothing to do with nova i bumpt it manually in https://review.opendev.org/c/openstack/requirements/+/845859/

The conflict is caused by: The user requested jsonschema===4.0.1 tripleo-common 16.4.0 depends on jsonschema>=3.2.0 rsd-lib 1.2.0 depends on jsonschema>=2.6.0 taskflow 4.7.0 depends on jsonschema>=3.2.0 zvmcloudconnector 1.4.1 depends on jsonschema>=2.3.0 os-net-config 15.2.0 depends on jsonschema>=3.2.0 task-core 0.2.1 depends on jsonschema>=3.2.0 python-zaqarclient 2.3.0 depends on jsonschema>=2.6.0 warlock 1.3.3 depends on jsonschema<4 and >=0.7

https://zuul.opendev.org/t/openstack/build/06ed295bb8244c16b48e2698c1049be9

it looks like warlock is clamping it ot less then 4 which is why we are stokc on 3.2.0

glance client seams to be the only real user of this https://codesearch.opendev.org/?q=warlock&i=nope&literal=nope&fi... perhaps we could jsut remove the dependcy?

I've proposed vendoring this dependency in glanceclient [1]. I've also proposed a change to fix this in warlock [2] but given the lack of activity there, I doubt it'll merge anytime soon so the former sounds like a better option.

My efforts to collect *all* the projects continues. Just cut 2.0.0 of warlock so we should see this make it's way through the requirements machinery in the next few days. I'll abandon the glanceclient change now.

Stephen

Hi Stephen,

I hope you don't mind I ping and up this thread.

Thanks a lot for this work. Any more progress here?

I'm being pressed by the Debian community to update jsonschema in Unstable, because 3.2.0 is breaking other software (at least 2 packages). If I do, I know things will break in OpenStack. So this *MUST* be fixed for Zed...

Cheers,

Thomas Goirand (zigo)

On 7/12/22 14:14, Stephen Finucane wrote:

On Mon, 2022-07-11 at 18:33 +0200, Thomas Goirand wrote:

...

Hi Stephen,

I hope you don't mind I ping and up this thread.

Thanks a lot for this work. Any more progress here?

We've uncapped warlock in openstack/requirements [1]. We just need the glance folks to remove their own cap now [2] so that we can raise the version in upper constraint.

Stephen

[1] https://review.opendev.org/c/openstack/requirements/+/849284 [2] https://review.opendev.org/c/openstack/python-glanceclient/+/849285

Hi !

I see these 2 are now merged, so it's job (well) done, right?

Cheers,

Thomas Goirand (zigo)

On 7/14/22 11:02, Stephen Finucane wrote:

On Wed, 2022-07-13 at 18:21 +0200, Thomas Goirand wrote:

...

On 7/12/22 14:14, Stephen Finucane wrote:

...

On Mon, 2022-07-11 at 18:33 +0200, Thomas Goirand wrote:

...

Hi Stephen,

I hope you don't mind I ping and up this thread.

Thanks a lot for this work. Any more progress here?

We've uncapped warlock in openstack/requirements [1]. We just need the glance folks to remove their own cap now [2] so that we can raise the version in upper constraint.

Stephen

[1] https://review.opendev.org/c/openstack/requirements/+/849284 [2] https://review.opendev.org/c/openstack/python-glanceclient/+/849285

Hi !

I see these 2 are now merged, so it's job (well) done, right?

I'd assume so, yes. We just need to wait for the machinery to do its job and bump the upper constraint now.

Stephen

Hi Stephen,

I uploaded a patched version of warlock to Unstable with the test fixed for the new jsonschema. However, when looking at the python-jsonschema pseudo-excuse, I can see that version 4.6.0 is breaking a bunch of other OpenStack projects:

https://release.debian.org/britney/pseudo-excuses-experimental.html#python-j...

This includes: - designate - ironic - nova - sahara

I'll try to see what I can do to fix these, maybe some of the failures are unrelated (I haven't investigated yet).

Cheers,

Thomas Goirand (zigo)

Ironic was not too bad either: https://review.opendev.org/c/openstack/ironic/+/849882 Similar for Nova: https://review.opendev.org/c/openstack/nova/+/849881

On Thu, Jul 14, 2022 at 5:08 PM Stephen Finucane stephenfin@redhat.com wrote:

On Thu, 2022-07-14 at 15:51 +0200, Thomas Goirand wrote:

...

On 7/14/22 11:02, Stephen Finucane wrote:

...

On Wed, 2022-07-13 at 18:21 +0200, Thomas Goirand wrote:

...

On 7/12/22 14:14, Stephen Finucane wrote:

...

On Mon, 2022-07-11 at 18:33 +0200, Thomas Goirand wrote:

...

Hi Stephen,

I hope you don't mind I ping and up this thread.

Thanks a lot for this work. Any more progress here?

We've uncapped warlock in openstack/requirements [1]. We just need

the glance

...

...

...

...

folks to remove their own cap now [2] so that we can raise the

version in upper

...

...

...

...

constraint.

Stephen

[1] https://review.opendev.org/c/openstack/requirements/+/849284 [2]

https://review.opendev.org/c/openstack/python-glanceclient/+/849285

...

...

...

Hi !

I see these 2 are now merged, so it's job (well) done, right?

I'd assume so, yes. We just need to wait for the machinery to do its

job and

...

...

bump the upper constraint now.

Stephen

Hi Stephen,

I uploaded a patched version of warlock to Unstable with the test fixed for the new jsonschema. However, when looking at the python-jsonschema pseudo-excuse, I can see that version 4.6.0 is breaking a bunch of other OpenStack projects:

https://release.debian.org/britney/pseudo-excuses-experimental.html#python-j...

...

This includes:

• designate
• ironic
• nova

The nova fix was trivial enough:

https://review.opendev.org/c/openstack/nova/+/849867

I'll have to let someone else fix the other projects. We'll see these flagged as soon as a patch to bump the upper-constraint of jsonschema hits openstack/requirements (which will happen once the python-glanceclient upper constraint bump to same merges).

Stephen

...

• sahara

I'll try to see what I can do to fix these, maybe some of the failures are unrelated (I haven't investigated yet).

Cheers,

Thomas Goirand (zigo)

On Sat, 2022-07-16 at 11:12 +0200, Thomas Goirand wrote:

Hi there!

On 7/14/22 18:09, Dmitry Tantsur wrote:

...

Ironic was not too bad either: https://review.opendev.org/c/openstack/ironic/+/849882 https://review.opendev.org/c/openstack/ironic/+/849882 Similar for Nova: https://review.opendev.org/c/openstack/nova/+/849881 https://review.opendev.org/c/openstack/nova/+/849881

Thanks, I was able to backport these fixes in the Yoga version of Ironic and Nova, and uploaded them to Debian Unstable (currently I'm finishing to build Ironic that had its unit tests passing already).

...

 > - sahara
 >
 > I'll try to see what I can do to fix these, maybe some of the
failures
 > are unrelated (I haven't investigated yet).

Only Sahara is missing a fix now. I tracked it down to this:

https://github.com/openstack/sahara/blob/master/sahara/utils/api_validator.p...

the error being:

TypeError: __init__() got an unexpected keyword argument 'types'

Looks like "types" has gone away from the parent class. Does anyone know what's going on, and what the replacement is? I first thought it looks like "types" should really be "type_checker", so I tried that, but it didn't work...

Once Sahara is fixed, I'm done for all OpenStack packages, and only 2 other Debian packages will need a fix (but I filed Debian bugs against these packages, so we're good...).

Looks like Sahara didn't have deprecation warnings turned on in CI and missed . [1]. Google leads me to [2]. I guess you want [3]. That's all I've got so far.

Stephen

[1] https://github.com/python-jsonschema/jsonschema/blob/v3.2.0/jsonschema/valid... [2] https://python-jsonschema.readthedocs.io/en/latest/creating/ [3] https://python-jsonschema.readthedocs.io/en/latest/creating/#jsonschema.vali...

Cheers,

Thomas Goirand (zigo)