Open Stack

Hi Rossella,

thanks for doing this work!
I think your results are pretty much consistent with what we've been
empirically observing in recent days.

elastic-recheck is flagging a resurgence of bug 1253896, and we need to
have somebody look at it as soon as possib.e
For bug 1291922, I've marked it as a duplicate (so far it's been reported
independently by at least 4 people). A patch for it has already merged.
Similarly, I've marked bug 1291918 as a duplicate of bug 1283522. Looking
at the logs for this failure, I found out that the actual root cause is a
"lock wait timeout". We've seen a spike in this kind of failures in the
last 36 hours; for this reason the severity of bug 1283522 was raised to
"critical".

I am a bit surprised by seeing an high occurrence of bug 1281969, perhaps
because I have no idea how it relates with neutron!
However, pushing Elastic Recheck queries won't be such a bad idea.
Finally, some of the bugs you found do not seem to be exclusive to neutron
(such as bug 1291947). In that case an Elastic Recheck query would be even
more useful.


Thanks again,
Salvatore


On 13 March 2014 13:08, Rossella Sblendido <rsblendido at suse.com> wrote:

> Hello devs,
>
> I wanted the update the analysis performed by Salvatore Orlando few weeks
> ago [1]
> I used the following query for Logstash [2] to detect the failures of the
> last 48 hours.
>
> There were 77 failures (40% of the total).
> I classified them and obtained the following:
>
> 21% due to infra issues
> 16% https://bugs.launchpad.net/tempest/+bug/1253896
> 14% https://bugs.launchpad.net/neutron/+bug/1291922
> 12% https://bugs.launchpad.net/tempest/+bug/1281969
> 10% https://bugs.launchpad.net/neutron/+bug/1291920
> 7% https://bugs.launchpad.net/neutron/+bug/1291918
> 7% https://bugs.launchpad.net/neutron/+bug/1291926
> 5% https://bugs.launchpad.net/neutron/+bug/1291947
> 3% https://bugs.launchpad.net/neutron/+bug/1277439
> 3% https://bugs.launchpad.net/neutron/+bug/1283599
> 2% https://bugs.launchpad.net/nova/+bug/1255627
>
> I had to file 5 new bugs, that are on the previous list and can be viewed
> here [3].
>
> cheers,
>
> Rossella
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2014-
> February/027862.html
> [2] http://logstash.openstack.org/#eyJzZWFyY2giOiJidWlsZF9uYW1lOl
> wiY2hlY2stdGVtcGVzdC1kc3ZtLW5ldXRyb24tZnVsbFwiICBBTkQgcHJvam
> VjdDpcIm9wZW5zdGFjay9uZXV0cm9uXCIgQU5EIG1lc3NhZ2U6XCJGaW5pc2
> hlZDpcIiBBTkQgYnVpbGRfc3RhdHVzOlwiRkFJTFVSRVwiIEFORCBidWlsZF
> 9icmFuY2g6XCJtYXN0ZXJcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidG
> ltZWZyYW1lIjoiMTcyODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIj
> p7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM5NDcwNzAzODk5NywibW
> 9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIifQ==
> [3] https://bugs.launchpad.net/neutron/+bugs?field.tag=neutron-full-job
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140314/eac312d7/attachment.html>