<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jun 9, 2014 at 11:55 PM, Eoghan Glynn <span dir="ltr"><<a href="mailto:eglynn@redhat.com" target="_blank">eglynn@redhat.com</a>></span> wrote:<br>

<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div class=""><br>
<br>
> Over the last 7 days ceilometer unit test jobs have a 18% failure rate in the<br>
> gate queue [0], while we see expect to see some failures in integration<br>
> testing, unit tests should not be failing in the gate with such a high<br>
> frequency (and for so long).<br>
><br>
> It looks like these failures are due to two bugs [1] [2]. I would like to<br>
> propose that until these bugs are resolved, that ceilometer refrain from<br>
> approving patches as to not negatively impact the gate queue, which is<br>
> already in a tenuous state.<br>
<br>
</div>Hi Joe,<br>
<br>
Thanks for raising this.<br>
<br>
We have approved patches addressing both persistent failures<br>
in the verification queue:<br>
<br>
  <a href="https://review.openstack.org/98953" target="_blank">https://review.openstack.org/98953</a><br>
  <a href="https://review.openstack.org/98820" target="_blank">https://review.openstack.org/98820</a><br>
<br></blockquote><div><br></div><div>Thank you for the prompt turn around.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">


BTW these data on per-project unit test failure rates sound<br>
interesting and useful. Are these rates surfaced somewhere<br>
easily consumable (by folks outside of the QA team)?<br></blockquote><div><br></div><div>Excellent question, I have a few answers but I'm not sure if any of them are satisfactory. </div><div><br></div><div>I calculated that failure rate via elastic-search, but we also store a lot of great data in <a href="http://graphite.openstack.org/">http://graphite.openstack.org/</a>.  I have a little page that queries <a href="http://graphite.openstack.org">graphite.openstack.org</a> at <a href="http://jogo.github.io/gate/">http://jogo.github.io/gate/</a>. And I added a third graph at the bottom to show the python27 failure rates in the gate queue (this data is can be somewhat misleading, especially if there isn't enough data for the moving average to really mean something).</div>

<div><br></div><div>Another good place to look is the list of unclassified gate failures, <a href="http://status.openstack.org/elastic-recheck/data/uncategorized.html">http://status.openstack.org/elastic-recheck/data/uncategorized.html</a></div>

<div><br></div><div>best,</div><div>Joe</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">


<br>
Cheers,<br>
Eoghan<br>
<div class=""><br>
> best,<br>
> Joe<br>
><br>
> [0]<br>
> <a href="http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJtZXNzYWdlOlwiRmluaXNoZWQ6XCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIgQU5EIHByb2plY3Q6XCJvcGVuc3RhY2svY2VpbG9tZXRlclwiIEFORCBidWlsZF9xdWV1ZTpcImdhdGVcIiBBTkQgKGJ1aWxkX25hbWU6XCJnYXRlLWNlaWxvbWV0ZXItcHl0aG9uMjdcIiBPUiAgYnVpbGRfbmFtZTpcImdhdGUtY2VpbG9tZXRlci1weXRob24yNlwiKSIsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50Iiwib2Zmc2V0IjowLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQwMjM1NjkyMDE2MCwibW9kZSI6InNjb3JlIiwiYW5hbHl6ZV9maWVsZCI6ImJ1aWxkX3N0YXR1cyJ9" target="_blank">http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJtZXNzYWdlOlwiRmluaXNoZWQ6XCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIgQU5EIHByb2plY3Q6XCJvcGVuc3RhY2svY2VpbG9tZXRlclwiIEFORCBidWlsZF9xdWV1ZTpcImdhdGVcIiBBTkQgKGJ1aWxkX25hbWU6XCJnYXRlLWNlaWxvbWV0ZXItcHl0aG9uMjdcIiBPUiAgYnVpbGRfbmFtZTpcImdhdGUtY2VpbG9tZXRlci1weXRob24yNlwiKSIsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50Iiwib2Zmc2V0IjowLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQwMjM1NjkyMDE2MCwibW9kZSI6InNjb3JlIiwiYW5hbHl6ZV9maWVsZCI6ImJ1aWxkX3N0YXR1cyJ9</a><br>


> [1] <a href="https://bugs.launchpad.net/ceilometer/+bug/1323524" target="_blank">https://bugs.launchpad.net/ceilometer/+bug/1323524</a><br>
> [2] <a href="https://bugs.launchpad.net/ceilometer/+bug/1327344" target="_blank">https://bugs.launchpad.net/ceilometer/+bug/1327344</a><br>
><br>
</div>> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div><br></div></div>