[openstack-dev] [QA][Neutron] IPv6 related intermittent test failures
brian.haley at hpe.com
Wed Feb 3 03:33:21 UTC 2016
On 02/02/2016 10:03 PM, Matthew Treinish wrote:
> On Tue, Feb 02, 2016 at 05:09:47PM -0800, Armando M. wrote:
>> We have some IPv6 related bugs [1,2,3] that have been lingering for some
>> time now. They have been hurting the gate (e.g.  the most recent
>> offending failure) and since it looks like they have been without owners
>> nor a plan of action for some time, I made the hard decision of skipping
>> them  ahead of the busy times ahead.
> So TBH I don't think the failure rate for these tests are really at a point
> necessitating a skip:
> (also just a cool side-note, you can see the very obvious performance regression
> caused by the keystonemiddleware release and when we excluded that version in
> Well, test_dualnet_dhcp6_stateless_from_os is kinda there with a ~10% failure
> rate, but the other 2 really aren't. I normally would be -1 on the skip patch
> because of that. We try to save the skips for cases where the bugs are really
> severe and preventing productivity at a large scale.
> But, in this case these ipv6 tests are kinda of out of place in tempest. Having
> all the permutations of possible ip allocation configurations always seemed a
> bit too heavy handed. These tests are also consistently in the top 10 slowest
> for a run. We really should have trimmed down this set a while ago so we're only
> have a single case in tempest. Neutron should own the other possible
> configurations as an in-tree test.
> Brian Haley has a patch up from Dec. that was trying to clean it up:
I just updated that to mark six of the eight tests as "slow" per your previous
comment, such that only the dual-NIC/dual-stack tests are run in the gate, the
others will run in the periodic nightly job.
Will help lessen the impact until we can determine if it's the test or Neutron.
> We probably should revisit that soon, since quite clearly no one is looking at
> these right now.
> -Matt Treinish
>> Now one might argue that skipping them is counterproductive because it may
>> allow other regressions to sneak in, but I am hoping that this
>> controversial action will indeed smoke out the right folks.
>> Comments welcome.
>>  https://bugs.launchpad.net/neutron/+bug/1477192
>>  https://bugs.launchpad.net/neutron/+bug/1509004
>>  https://bugs.launchpad.net/openstack-gate/+bug/1540983
>>  https://review.openstack.org/#/c/275457/
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
More information about the OpenStack-dev