Open Stack

On Tue, Feb 02, 2016 at 05:09:47PM -0800, Armando M. wrote:
> Folks,
> 
> We have some IPv6 related bugs [1,2,3] that have been lingering for some
> time now. They have been hurting the gate (e.g. [4] the most recent
> offending failure) and since it looks like they have been without owners
> nor a plan of action for some time, I made the hard decision of skipping
> them [5] ahead of the busy times ahead.

So TBH I don't think the failure rate for these tests are really at a point
necessitating a skip:

http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os

(also just a cool side-note, you can see the very obvious performance regression
caused by the keystonemiddleware release and when we excluded that version in
requirements)

Well, test_dualnet_dhcp6_stateless_from_os is kinda there with a ~10% failure
rate, but the other 2 really aren't. I normally would be -1 on the skip patch
because of that. We try to save the skips for cases where the bugs are really
severe and preventing productivity at a large scale. 

But, in this case these ipv6 tests are kinda of out of place in tempest. Having
all the permutations of possible ip allocation configurations always seemed a
bit too heavy handed. These tests are also consistently in the top 10 slowest
for a run. We really should have trimmed down this set a while ago so we're only
have a single case in tempest. Neutron should own the other possible
configurations as an in-tree test.

Brian Haley has a patch up from Dec. that was trying to clean it up:

https://review.openstack.org/#/c/239868/

We probably should revisit that soon, since quite clearly no one is looking at
these right now.


-Matt Treinish


> 
> Now one might argue that skipping them is counterproductive because it may
> allow other regressions to sneak in, but I am hoping that this
> controversial action will indeed smoke out the right folks.
> 
> Comments welcome.
> 
> Regards,
> Armando
> 
> [1] https://bugs.launchpad.net/neutron/+bug/1477192
> [2] https://bugs.launchpad.net/neutron/+bug/1509004
> [3] https://bugs.launchpad.net/openstack-gate/+bug/1540983
> [4]
> http://logs.openstack.org/37/264937/5/gate/gate-tempest-dsvm-neutron-full/afeaabd//logs/testr_results.html.gz
> [5] https://review.openstack.org/#/c/275457/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160202/f1af9172/attachment.pgp>