Open Stack

On 3 February 2016 at 18:49, Armando M. <armamig at gmail.com> wrote:

>
>
> On 3 February 2016 at 04:28, Sean Dague <sean at dague.net> wrote:
>
>> On 02/02/2016 10:03 PM, Matthew Treinish wrote:
>> > On Tue, Feb 02, 2016 at 05:09:47PM -0800, Armando M. wrote:
>> >> Folks,
>> >>
>> >> We have some IPv6 related bugs [1,2,3] that have been lingering for
>> some
>> >> time now. They have been hurting the gate (e.g. [4] the most recent
>> >> offending failure) and since it looks like they have been without
>> owners
>> >> nor a plan of action for some time, I made the hard decision of
>> skipping
>> >> them [5] ahead of the busy times ahead.
>> >
>> > So TBH I don't think the failure rate for these tests are really at a
>> point
>> > necessitating a skip:
>> >
>> >
>> http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_multi_prefix_slaac
>> >
>> http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_dualnet_dhcp6_stateless_from_os
>> >
>> http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_network_v6.TestGettingAddress.test_dhcp6_stateless_from_os
>> >
>> > (also just a cool side-note, you can see the very obvious performance
>> regression
>> > caused by the keystonemiddleware release and when we excluded that
>> version in
>> > requirements)
>> >
>> > Well, test_dualnet_dhcp6_stateless_from_os is kinda there with a ~10%
>> failure
>> > rate, but the other 2 really aren't. I normally would be -1 on the skip
>> patch
>> > because of that. We try to save the skips for cases where the bugs are
>> really
>> > severe and preventing productivity at a large scale.
>> >
>> > But, in this case these ipv6 tests are kinda of out of place in
>> tempest. Having
>> > all the permutations of possible ip allocation configurations always
>> seemed a
>> > bit too heavy handed. These tests are also consistently in the top 10
>> slowest
>> > for a run. We really should have trimmed down this set a while ago so
>> we're only
>> > have a single case in tempest. Neutron should own the other possible
>> > configurations as an in-tree test.
>> >
>> > Brian Haley has a patch up from Dec. that was trying to clean it up:
>> >
>> > https://review.openstack.org/#/c/239868/
>> >
>> > We probably should revisit that soon, since quite clearly no one is
>> looking at
>> > these right now.
>>
>> We definitely shouldn't be running all the IPv6 tests.
>>
>> But I also think the assumption that the failure rate is low is not a
>> valid reason to keep a test. Unreliable tests that don't have anyone
>> looking into them should be deleted. They are providing negative value.
>> Because people just recheck past them even if their code made the race
>> worse. So any legitimate issues they are exposing are being ignored.
>>
>> If the neutron PTL wants tests pulled, we should just do it.
>>
>>
> Thanks for the support! Having said, I think it's important to make a
> judgement call on a case by case basis, because removing tests blindly
> might as well backfire.
>
> In this specific instance and all things considered, merging [2] (or even
> better [1]) feel like a net gain.
>
> Cheers,
> Armando
>
> [1] https://review.openstack.org/#/c/239868/
> [2] https://review.openstack.org/#/c/275457/
>
>

Btw I did respin [1], because I am still seeing intermittent failures.

[1] https://review.openstack.org/#/c/275457/

        -Sean
>>
>> --
>> Sean Dague
>> http://dague.net
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160205/fa5935e8/attachment.html>