[sdk][devstack] openstacksdk-functional-devstack job failing on stable branches
openstacksdk-functional-devstack began failing in the stable branches (xena through ussuri) shortly after 2022-01-24 11:53:49 with the error
openstack.exceptions.ResourceNotFound: ResourceNotFound: 404: Client Error for url: https://%7Bip_address%7D:9696/v2.0/qos/policies/69e472c0-37d2-46e0-9b9e-727c..., The resource could not be found.
for these four tests:
openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.TestQoSMinimumPacketRateRule.test_find openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.TestQoSMinimumPacketRateRule.test_get openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.TestQoSMinimumPacketRateRule.test_list openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.TestQoSMinimumPacketRateRule.test_update
job build history:
https://zuul.opendev.org/t/openstack/builds?job_name=openstacksdk-functional...
It's passing in master and stable/{train,stein,rocky}. Anyone know what's up?
Hi,
On środa, 26 stycznia 2022 03:45:08 CET Brian Rosmaita wrote:
openstacksdk-functional-devstack began failing in the stable branches (xena through ussuri) shortly after 2022-01-24 11:53:49 with the error
openstack.exceptions.ResourceNotFound: ResourceNotFound: 404: Client Error for url: https://%7Bip_address%7D:9696/v2.0/qos/policies/
69e472c0-37d2-46e0-9b9e-727c962c5
511/minimum_packet_rate_rules, The resource could not be found.
for these four tests:
openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.TestQ
oSMinimumPacketRateRule.test_find openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.Test QoSMinimumPacketRateRule.test_get openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.Test QoSMinimumPacketRateRule.test_list openstack.tests.functional.network.v2.test_qos_minimum_packet_rate_rule.Test QoSMinimumPacketRateRule.test_update
job build history:
https://zuul.opendev.org/t/openstack/builds?job_name=openstacksdk-functional...
devstack&branch=stable%2Fxena&branch=stable%2Fwallaby&branch=stable%2Fvictori
a&branch=stable%2Fussuri
It's passing in master and stable/{train,stein,rocky}. Anyone know what's up?
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2]. I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[1] https://opendev.org/openstack/openstacksdk/src/branch/master/openstack/ tests/functional/network/v2/test_qos_minimum_packet_rate_rule.py [2] https://opendev.org/openstack/openstacksdk/src/branch/stable/xena/ openstack/tests/functional/network/v2/test_qos_minimum_packet_rate_rule.py
On 2022-01-26 09:00:05 +0100 (+0100), Slawek Kaplonski wrote: [...]
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2].
I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[...]
I find it intriguing that the SDK has stable branches at all. Isn't the goal that the latest version of the SDK be able to interface with multiple versions of OpenStack services, new and old? If we don't test that the current SDK code works with Neutron from Xena, then that opens it up to growing serious problems for users.
Having the SDK or tests work out what features are expected to work sounds like the only sensible solution.
On Wed, 2022-01-26 at 13:26 +0000, Jeremy Stanley wrote:
On 2022-01-26 09:00:05 +0100 (+0100), Slawek Kaplonski wrote: [...]
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2].
I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[...]
I find it intriguing that the SDK has stable branches at all. Isn't the goal that the latest version of the SDK be able to interface with multiple versions of OpenStack services, new and old? If we don't test that the current SDK code works with Neutron from Xena, then that opens it up to growing serious problems for users.
ya that seam odd to me too. i would expect sdk to be release independent and work simialr to tempest where master sdk should be useable with all stable branches.
Having the SDK or tests work out what features are expected to work sounds like the only sensible solution.
Hi
On 26. Jan 2022, at 14:35, Sean Mooney smooney@redhat.com wrote:
On Wed, 2022-01-26 at 13:26 +0000, Jeremy Stanley wrote:
On 2022-01-26 09:00:05 +0100 (+0100), Slawek Kaplonski wrote: [...]
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2].
I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[...]
I find it intriguing that the SDK has stable branches at all. Isn't the goal that the latest version of the SDK be able to interface with multiple versions of OpenStack services, new and old? If we don't test that the current SDK code works with Neutron from Xena, then that opens it up to growing serious problems for users.
ya that seam odd to me too. i would expect sdk to be release independent and work simialr to tempest where master sdk should be useable with all stable branches.
Having the SDK or tests work out what features are expected to work sounds like the only sensible solution.
Well, actually it should be working. Cause of absence of real tests this way I can not guarantee it will work in every case, but we do our best to keep it this way (and most of the tests are checking whether feature is available or not).
Honestly I have no clue why it evolved this way and therefore can’t really comment on that. Maybe (just maybe) there was something from the Debian (or other distro) packaging pov (stable, not-stable, etc) that somehow leaded to this setup. Otherwise there is absolutely no possibility to limit which version of sdk need to go into which older distro. And since i.e. ansible and openstackclient depend on the sdk things are getting even more interesting. Sdk in this sense is comparable with keystoneauth lib (and actually depend on it). So once we say there are no stable branches of sdk anymore we open even worse can of worms.
Since we are currently anyway in a phase of big rework in sdk I would say addressing tests that do not work on older branches can be done as well.
Artem
Le mer. 26 janv. 2022 à 15:28, Artem Goncharov artem.goncharov@gmail.com a écrit :
Hi
On 26. Jan 2022, at 14:35, Sean Mooney smooney@redhat.com wrote:
On Wed, 2022-01-26 at 13:26 +0000, Jeremy Stanley wrote:
On 2022-01-26 09:00:05 +0100 (+0100), Slawek Kaplonski wrote: [...]
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2].
I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[...]
I find it intriguing that the SDK has stable branches at all. Isn't the goal that the latest version of the SDK be able to interface with multiple versions of OpenStack services, new and old? If we don't test that the current SDK code works with Neutron from Xena, then that opens it up to growing serious problems for users.
ya that seam odd to me too. i would expect sdk to be release independent and work simialr to tempest where master sdk should be useable with all stable branches.
Having the SDK or tests work out what features are expected to work sounds like the only sensible solution.
Well, actually it should be working. Cause of absence of real tests this way I can not guarantee it will work in every case, but we do our best to keep it this way (and most of the tests are checking whether feature is available or not).
Honestly I have no clue why it evolved this way and therefore can’t really comment on that. Maybe (just maybe) there was something from the Debian (or other distro) packaging pov (stable, not-stable, etc) that somehow leaded to this setup. Otherwise there is absolutely no possibility to limit which version of sdk need to go into which older distro. And since i.e. ansible and openstackclient depend on the sdk things are getting even more interesting. Sdk in this sense is comparable with keystoneauth lib (and actually depend on it). So once we say there are no stable branches of sdk anymore we open even worse can of worms.
Since we are currently anyway in a phase of big rework in sdk I would say addressing tests that do not work on older branches can be done as well.
FWIW, we recently discussed on #openstack-nova about this [1] and the consensus was that we should try to avoid as much as possible any attempt to test stable releases of Nova against any stable branch of the SDK.
I haven't seen yet any bug report, does anyone know about it ?
As our conclusion was that we would refrain pulling stable branches of the SDK, we'll need to make the job non-voting on our repositories as it holds merging backports.
More to come in the next hours, -Sylvain
[1] https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.202...
Artem
On 28. Jan 2022, at 11:49, Sylvain Bauza sbauza@redhat.com wrote:
Le mer. 26 janv. 2022 à 15:28, Artem Goncharov <artem.goncharov@gmail.com mailto:artem.goncharov@gmail.com> a écrit : Hi
On 26. Jan 2022, at 14:35, Sean Mooney <smooney@redhat.com mailto:smooney@redhat.com> wrote:
On Wed, 2022-01-26 at 13:26 +0000, Jeremy Stanley wrote:
On 2022-01-26 09:00:05 +0100 (+0100), Slawek Kaplonski wrote: [...]
Those tests are available only in master branch: [1] and they not exists in e.g. stable/xena [2].
I'm not sure how those jobs are defined exactly but my guess is that on those stable branches where it fails it runs openstacksdk from master branch. So either it should be changed and proper stable branch of the openstacksdk should be used there or we should skip those tests when API extension in neuron is not present.
[...]
I find it intriguing that the SDK has stable branches at all. Isn't the goal that the latest version of the SDK be able to interface with multiple versions of OpenStack services, new and old? If we don't test that the current SDK code works with Neutron from Xena, then that opens it up to growing serious problems for users.
ya that seam odd to me too. i would expect sdk to be release independent and work simialr to tempest where master sdk should be useable with all stable branches.
Having the SDK or tests work out what features are expected to work sounds like the only sensible solution.
Well, actually it should be working. Cause of absence of real tests this way I can not guarantee it will work in every case, but we do our best to keep it this way (and most of the tests are checking whether feature is available or not).
Honestly I have no clue why it evolved this way and therefore can’t really comment on that. Maybe (just maybe) there was something from the Debian (or other distro) packaging pov (stable, not-stable, etc) that somehow leaded to this setup. Otherwise there is absolutely no possibility to limit which version of sdk need to go into which older distro. And since i.e. ansible and openstackclient depend on the sdk things are getting even more interesting. Sdk in this sense is comparable with keystoneauth lib (and actually depend on it). So once we say there are no stable branches of sdk anymore we open even worse can of worms.
Since we are currently anyway in a phase of big rework in sdk I would say addressing tests that do not work on older branches can be done as well.
FWIW, we recently discussed on #openstack-nova about this [1] and the consensus was that we should try to avoid as much as possible any attempt to test stable releases of Nova against any stable branch of the SDK.
I haven't seen yet any bug report, does anyone know about it ?
As our conclusion was that we would refrain pulling stable branches of the SDK, we'll need to make the job non-voting on our repositories as it holds merging backports.
JFYI: We started working on running our functional tests on old stable branches with latest SDK to enable others using sdk master branch.
More to come in the next hours, -Sylvain
[1] https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.202... https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2022-01-28.log.html#t2022-01-28T10:42:31
Artem
participants (6)
-
Artem Goncharov
-
Brian Rosmaita
-
Jeremy Stanley
-
Sean Mooney
-
Slawek Kaplonski
-
Sylvain Bauza