[nova][keystone][neutron][kuryr][requirements] breaking tests with new library versions
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352 MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265 jsonschema===3.0.2 see https://review.opendev.org/649789 I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated. -- Matthew Thode
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug: https://github.com/testing-cabal/testtools/issues/272 Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest. [1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions. [2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it -Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
On 19-08-19 10:54:37, Matthew Treinish wrote:
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
Thanks for the clarification, now to get the other projects to pay attention :| -- Matthew Thode
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666
Sean Mooney agreed to take a look at this one.
websockify===0.9.0 tempest test failing
This is now known as bug 1840788 [1]. I did some initial investigation and tried to fix it. Turned out my fix was already part of another change. But it also turns out that that fix is only part of the solution. See the bug report for (links to) details. At this point, it needs someone who understands what test_novnc [2] is actually trying to do. Because I don't. Anyone? Thanks, efried [1] https://bugs.launchpad.net/nova/+bug/1840788 [2] https://opendev.org/openstack/tempest/src/branch/master/tempest/api/compute/...
On 8/21/19 3:44 PM, Eric Fried wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666
Sean Mooney agreed to take a look at this one.
websockify===0.9.0 tempest test failing
This is now known as bug 1840788 [1].
I did some initial investigation and tried to fix it. Turned out my fix was already part of another change. But it also turns out that that fix is only part of the solution. See the bug report for (links to) details.
At this point, it needs someone who understands what test_novnc [2] is actually trying to do. Because I don't.
Anyone?
From the test run on your patch [3], it looks like we're going to need a change in nova as well in nova/console/websocketproxy.py [4]: AttributeError: module 'websockify' has no attribute 'WebSocketServer' It looks like the WebSocketServer class got moved from websockify.websocket to websockify.websocketserver in v0.9.0: https://github.com/novnc/websockify/commit/8a697622495fd319582cd1c604e7eb2cc... and thus is no longer accessible via the top level 'websockify' module as a result: https://github.com/novnc/websockify/blob/v0.9.0/websockify/__init__.py Since this is a change to upper-constraints to allow v0.9.0, we will need a way for it to work with both module layouts, yeah? Cheers, -melanie [3] https://review.opendev.org/677798 [4] https://zuul.opendev.org/t/openstack/build/e0a8a19021b64350a0a55cc08f374d02/...
Thanks, efried
[1] https://bugs.launchpad.net/nova/+bug/1840788 [2] https://opendev.org/openstack/tempest/src/branch/master/tempest/api/compute/...
On Wed, Aug 21, 2019, at 4:25 PM, melanie witt wrote: Snip
Since this is a change to upper-constraints to allow v0.9.0, we will need a way for it to work with both module layouts, yeah?
Yup, the constrained version is likely one of many valid versions people can use. The constrained version is just what we test with to prevent issues like these from breaking the gate. Unless you also change the lower bounds of the dep people may still continue to use the old version too. Good idea to support the older code in that case.
Cheers, -melanie
On 8/21/19 4:30 PM, Clark Boylan wrote:
On Wed, Aug 21, 2019, at 4:25 PM, melanie witt wrote:
Snip
Since this is a change to upper-constraints to allow v0.9.0, we will need a way for it to work with both module layouts, yeah?
Yup, the constrained version is likely one of many valid versions people can use. The constrained version is just what we test with to prevent issues like these from breaking the gate. Unless you also change the lower bounds of the dep people may still continue to use the old version too. Good idea to support the older code in that case.
Thanks Clark! I've uploaded a nova change at: https://review.opendev.org/677856 and added it as a Depends-On to: https://review.opendev.org/677798 to see if it solves the problem. Cheers, -melanie
{NOVA: websockify 0.9.0 breaks test_novnc [1]}
I've uploaded a nova change at: https://review.opendev.org/677856
and added it as a Depends-On to: https://review.opendev.org/677798
to see if it solves the problem.
It does; it's now merging. Once it and [2] (tempest) have merged, we should be able to proceed with the u-c bump. Thanks Mel! @prometheanfire, you could now swizzle [3] to un-DNM and Depends-On those two. (Is that how this is done, or do we need to let a bot-generated patch like [4] do it?) efried [1] https://bugs.launchpad.net/nova/+bug/1840788 [2] https://review.opendev.org/674364 [3] https://review.opendev.org/677479 [4] https://review.opendev.org/677619
On 19-08-22 12:45:59, Eric Fried wrote:
{NOVA: websockify 0.9.0 breaks test_novnc [1]}
I've uploaded a nova change at: https://review.opendev.org/677856
and added it as a Depends-On to: https://review.opendev.org/677798
to see if it solves the problem.
It does; it's now merging. Once it and [2] (tempest) have merged, we should be able to proceed with the u-c bump.
Thanks Mel!
@prometheanfire, you could now swizzle [3] to un-DNM and Depends-On those two. (Is that how this is done, or do we need to let a bot-generated patch like [4] do it?)
efried
[1] https://bugs.launchpad.net/nova/+bug/1840788 [2] https://review.opendev.org/674364 [3] https://review.opendev.org/677479 [4] https://review.opendev.org/677619
I've un DNM'd it, thanks for the work -- Matthew Thode
On 19-08-21 16:30:39, Clark Boylan wrote:
On Wed, Aug 21, 2019, at 4:25 PM, melanie witt wrote:
Snip
Since this is a change to upper-constraints to allow v0.9.0, we will need a way for it to work with both module layouts, yeah?
Yup, the constrained version is likely one of many valid versions people can use. The constrained version is just what we test with to prevent issues like these from breaking the gate. Unless you also change the lower bounds of the dep people may still continue to use the old version too. Good idea to support the older code in that case.
Yep, if you do want to set a minimum you still can, but that can also affect co-installability. You'll need to have at least one sha that supports both though, otherwise there'd have to be a force push at some point... So might as well support older stuff too (unless it's bad code wise). -- Matthew Thode
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us. Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like. [1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b... -gmann
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
On Wed, Aug 21, 2019 at 07:21:41PM +0900, Ghanshyam Mann wrote:
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us.
https://github.com/testing-cabal/testtools/pull/277 isn't a short term solution, unittest2 is still needed for python < 3.5 in testtools and testtools has not deprecated support for python 2.7 or 3.4 yet. I probably can rework that PR so that it's conditional and always uses stdlib unittest for python >= 3.5 but then testtools ends up maintaining two separate paths depending on python version. I'd like to continue thinking about that is as a long term solution because I don't know when I'll have the time to keep pushing that PR forward.
Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like.
[1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b...
-gmann
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
---- On Thu, 22 Aug 2019 04:41:44 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Wed, Aug 21, 2019 at 07:21:41PM +0900, Ghanshyam Mann wrote:
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us.
https://github.com/testing-cabal/testtools/pull/277 isn't a short term solution, unittest2 is still needed for python < 3.5 in testtools and testtools has not deprecated support for python 2.7 or 3.4 yet. I probably can rework that PR so that it's conditional and always uses stdlib unittest for python >= 3.5 but then testtools ends up maintaining two separate paths depending on python version. I'd like to continue thinking about that is as a long term solution because I don't know when I'll have the time to keep pushing that PR forward.
Thanks for more details. I understand that might take time. I am in OpenInfra event and after that on vacation till 29th Aug. I will be able to check the workaround on testtools or tempest side after that only. I will check with Matthew about when is the plan to move the stestr to 2.5.0. -gmann
Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like.
[1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b...
-gmann
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
On 19-08-23 20:09:31, Ghanshyam Mann wrote:
---- On Thu, 22 Aug 2019 04:41:44 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Wed, Aug 21, 2019 at 07:21:41PM +0900, Ghanshyam Mann wrote:
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us.
https://github.com/testing-cabal/testtools/pull/277 isn't a short term solution, unittest2 is still needed for python < 3.5 in testtools and testtools has not deprecated support for python 2.7 or 3.4 yet. I probably can rework that PR so that it's conditional and always uses stdlib unittest for python >= 3.5 but then testtools ends up maintaining two separate paths depending on python version. I'd like to continue thinking about that is as a long term solution because I don't know when I'll have the time to keep pushing that PR forward.
Thanks for more details. I understand that might take time. I am in OpenInfra event and after that on vacation till 29th Aug. I will be able to check the workaround on testtools or tempest side after that only. I will check with Matthew about when is the plan to move the stestr to 2.5.0.
-gmann
Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like.
[1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b...
-gmann
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
Any progress on this, at the moment only stestr-2.5.1 is being held back. https://review.opendev.org/680914 -- Matthew Thode
---- On Wed, 11 Sep 2019 00:23:08 +0900 Matthew Thode <mthode@mthode.org> wrote ----
On 19-08-23 20:09:31, Ghanshyam Mann wrote:
---- On Thu, 22 Aug 2019 04:41:44 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Wed, Aug 21, 2019 at 07:21:41PM +0900, Ghanshyam Mann wrote:
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us.
https://github.com/testing-cabal/testtools/pull/277 isn't a short term solution, unittest2 is still needed for python < 3.5 in testtools and testtools has not deprecated support for python 2.7 or 3.4 yet. I probably can rework that PR so that it's conditional and always uses stdlib unittest for python >= 3.5 but then testtools ends up maintaining two separate paths depending on python version. I'd like to continue thinking about that is as a long term solution because I don't know when I'll have the time to keep pushing that PR forward.
Thanks for more details. I understand that might take time. I am in OpenInfra event and after that on vacation till 29th Aug. I will be able to check the workaround on testtools or tempest side after that only. I will check with Matthew about when is the plan to move the stestr to 2.5.0.
-gmann
Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like.
[1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b...
-gmann
-Matt Treinish
jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
Any progress on this, at the moment only stestr-2.5.1 is being held back.
There is no progress on this yet. As unittest2 cannot be dropped from testtools, we need to get some workaround in Tempest. I need more time to try the failure and fix. -gmann
-- Matthew Thode
---- On Wed, 11 Sep 2019 00:42:03 +0900 Ghanshyam Mann <gmann@ghanshyammann.com> wrote ----
---- On Wed, 11 Sep 2019 00:23:08 +0900 Matthew Thode <mthode@mthode.org> wrote ----
On 19-08-23 20:09:31, Ghanshyam Mann wrote:
---- On Thu, 22 Aug 2019 04:41:44 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Wed, Aug 21, 2019 at 07:21:41PM +0900, Ghanshyam Mann wrote:
---- On Mon, 19 Aug 2019 23:54:37 +0900 Matthew Treinish <mtreinish@kortar.org> wrote ----
On Sun, Aug 18, 2019 at 11:16:11AM -0500, Matthew Thode wrote: > NOVA: > lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 > websockify===0.9.0 tempest test failing > > KEYSTONE: > oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393 > > NEUTRON: > tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... > this could be caused by pytest===5.1.0 as well > > KURYR: > kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it > https://review.opendev.org/665352 > > MISC: > tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) > stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265
This actually doesn't fix the underlying issue blocking it here. PR 265 is for fixing a compatibility issue with python 3.4, which we don't officially support in stestr but was a simple fix. The blocker is actually not an stestr issue, it's a testtools bug:
https://github.com/testing-cabal/testtools/issues/272
Where this is coming into play here is that stestr 2.5.0 switched to using an internal test runner built off of stdlib unittest instead of testtools/subunit for python 3. This was done to fix a huge number of compatibility issues people had reported when trying to run stdlib unittest suites using stestr on python >= 3.5 (which were caused by unittest2 and testools). The complication for openstack (more specificially tempest) is that it's built off of testtools not stdlib unittest. So when tempest raises 'self.skipException' as part of it's class level skip checks testtools raises 'unittest2.case.SkipTest' instead of 'unittest.case.SkipTest'. stdlib unittest does not understand what that is and treats it as an unhandled exception which is a test failure, instead of the intended skip result. [1] This is actually a general bug and will come up whenever anyone tries to use stdlib unittest to run tempest. We need to come up with a fix for this problem in testtools [2] or just workaround it in tempest.
[1] skip decorators typically aren't effected by this because they set an attribute that gets checked before the test method is executed instead of relying on an exception, which is why this is mostly only an issue for tempest because it does a lot of run time skips via exceptions.
[2] testtools is mostly unmaintained at this point, I was recently granted merge access but haven't had much free time to actively maintain it
Thanks matt for details. As you know, for Tempest where we need to support py2.7 (including unitest2 use) for stable branches, we are going to use the specific stetsr version/branch(<when you will remove the py27 support) so fixing it on testtools side is good option to me. I think your PR to remove the unittest2 use form testtools make sense to me [1]. A workaround in Tempest can be last option for us.
https://github.com/testing-cabal/testtools/pull/277 isn't a short term solution, unittest2 is still needed for python < 3.5 in testtools and testtools has not deprecated support for python 2.7 or 3.4 yet. I probably can rework that PR so that it's conditional and always uses stdlib unittest for python >= 3.5 but then testtools ends up maintaining two separate paths depending on python version. I'd like to continue thinking about that is as a long term solution because I don't know when I'll have the time to keep pushing that PR forward.
Thanks for more details. I understand that might take time. I am in OpenInfra event and after that on vacation till 29th Aug. I will be able to check the workaround on testtools or tempest side after that only. I will check with Matthew about when is the plan to move the stestr to 2.5.0.
-gmann
Till we fix it and to avoid gate break, can we cap stestr in g-r - stestr<2.5.0 ? I know that is not the options you like.
[1] https://github.com/mtreinish/testtools/commit/38fc9a9e302f68d471d7b097c7327b...
-gmann
-Matt Treinish
> jsonschema===3.0.2 see https://review.opendev.org/649789 > > I'm trying to get this in place as we are getting closer to the > requirements freeze (sept 9th-13th). Any help clearing up these bugs > would be appreciated. > > -- > Matthew Thode
Any progress on this, at the moment only stestr-2.5.1 is being held back.
There is no progress on this yet. As unittest2 cannot be dropped from testtools, we need to get some workaround in Tempest. I need more time to try the failure and fix.
Trying to workaround it in Tempest - https://review.opendev.org/#/c/681340/ but it seems it needs to handle too many cases in Tempest (with py and stestr versions ) Let's see if we can properly do it in Tempest. -gmann
-gmann
-- Matthew Thode
On Sun, 2019-08-18 at 11:16 -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
Alright, I tested and fixed the commit above and I should be able to get it merged today. Thanks for following up on this.
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265 jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
On 19-08-22 09:53:30, Michał Dulko wrote:
On Sun, 2019-08-18 at 11:16 -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
Alright, I tested and fixed the commit above and I should be able to get it merged today. Thanks for following up on this.
Yep, saw it merge. Now all it needs is a release so the version on pypy / upper-constraints does not hold back the version of kubernetes.
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265 jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
-- Matthew Thode
On Thu, 2019-08-22 at 08:17 -0500, Matthew Thode wrote:
On 19-08-22 09:53:30, Michał Dulko wrote:
On Sun, 2019-08-18 at 11:16 -0500, Matthew Thode wrote:
NOVA: lxml===4.4.1 nova tests fail https://bugs.launchpad.net/nova/+bug/1838666 websockify===0.9.0 tempest test failing
KEYSTONE: oauthlib===3.1.0 keystone https://bugs.launchpad.net/keystone/+bug/1839393
NEUTRON: tenacity===5.1.1 https://2c976b5e9e9a7bed9985-82d79a041e998664bd1d0bc4b6e78332.ssl.cf2.rackcd... this could be caused by pytest===5.1.0 as well
KURYR: kubernetes===10.0.1 openshift PINS this, only kuryr-tempest-plugin deps on it https://review.opendev.org/665352
Alright, I tested and fixed the commit above and I should be able to get it merged today. Thanks for following up on this.
Yep, saw it merge. Now all it needs is a release so the version on pypy / upper-constraints does not hold back the version of kubernetes.
Alright, let's see how my release commit [1] will work then. [1] https://review.opendev.org/#/c/678542/
MISC: tornado===5.1.1 salt is cauing this, no eta on fix (same as the last year) stestr===2.5.0 needs merged https://github.com/mtreinish/stestr/pull/265 jsonschema===3.0.2 see https://review.opendev.org/649789
I'm trying to get this in place as we are getting closer to the requirements freeze (sept 9th-13th). Any help clearing up these bugs would be appreciated.
participants (7)
-
Clark Boylan
-
Eric Fried
-
Ghanshyam Mann
-
Matthew Thode
-
Matthew Treinish
-
melanie witt
-
Michał Dulko