From 7th December 2018, all of sudden functional-py35
check job is failing on current master. Earlier this job was failing
intermittently (2-3 times from 100). I have tried to dig into the issue
and find out that it is failing obly for ubuntu and working fine with
Fedora, centos etc. Below are some of the findings:
1. Even if I add failure tests [1] in black-list file every time other tests reported as failure. 2.
When I compared failed tests results [1] with passed tests results [2] I
have found that passing tests shows 'Pass 741 Skip 7' whereas failing
tests shows 'Pass 627 Failure 9 Skip 5' (it does not execute all the
tests in case of failure) 3. I have compared version of subunit, oslo.service, oslo.log, tox with failed [3] and passed [4] logs and they are same. 4.
I came across a bug in cinder [5] which talks about excessive logs
might cause this error and I have tried to ignore the deprecation
warning in tox.ini by adding 'PYTHONWARNINGS=ignore::DeprecationWarning' but it still shows DeprecationWarning in the logs. 5. I have tried executing tests by setting '--concurrency' to 1 but still the error persists. 6. Whenver subunit.parser error occurs the tests remains 'inprogress' state and sometimes time-outs.
I have no clues so far to tackle this issue, kindly provide your inputs for the same.