Open Stack

On Mon, Jun 9, 2014 at 11:55 PM, Eoghan Glynn <eglynn at redhat.com> wrote:

>
>
> > Over the last 7 days ceilometer unit test jobs have a 18% failure rate
> in the
> > gate queue [0], while we see expect to see some failures in integration
> > testing, unit tests should not be failing in the gate with such a high
> > frequency (and for so long).
> >
> > It looks like these failures are due to two bugs [1] [2]. I would like to
> > propose that until these bugs are resolved, that ceilometer refrain from
> > approving patches as to not negatively impact the gate queue, which is
> > already in a tenuous state.
>
> Hi Joe,
>
> Thanks for raising this.
>
> We have approved patches addressing both persistent failures
> in the verification queue:
>
>   https://review.openstack.org/98953
>   https://review.openstack.org/98820
>
>
Thank you for the prompt turn around.



> BTW these data on per-project unit test failure rates sound
> interesting and useful. Are these rates surfaced somewhere
> easily consumable (by folks outside of the QA team)?
>

Excellent question, I have a few answers but I'm not sure if any of them
are satisfactory.

I calculated that failure rate via elastic-search, but we also store a lot
of great data in http://graphite.openstack.org/.  I have a little page that
queries graphite.openstack.org at http://jogo.github.io/gate/. And I added
a third graph at the bottom to show the python27 failure rates in the gate
queue (this data is can be somewhat misleading, especially if there isn't
enough data for the moving average to really mean something).

Another good place to look is the list of unclassified gate failures,
http://status.openstack.org/elastic-recheck/data/uncategorized.html

best,
Joe



>
> Cheers,
> Eoghan
>
> > best,
> > Joe
> >
> > [0]
> >
> http://logstash.openstack.org/#eyJmaWVsZHMiOltdLCJzZWFyY2giOiJtZXNzYWdlOlwiRmluaXNoZWQ6XCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIgQU5EIHByb2plY3Q6XCJvcGVuc3RhY2svY2VpbG9tZXRlclwiIEFORCBidWlsZF9xdWV1ZTpcImdhdGVcIiBBTkQgKGJ1aWxkX25hbWU6XCJnYXRlLWNlaWxvbWV0ZXItcHl0aG9uMjdcIiBPUiAgYnVpbGRfbmFtZTpcImdhdGUtY2VpbG9tZXRlci1weXRob24yNlwiKSIsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50Iiwib2Zmc2V0IjowLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQwMjM1NjkyMDE2MCwibW9kZSI6InNjb3JlIiwiYW5hbHl6ZV9maWVsZCI6ImJ1aWxkX3N0YXR1cyJ9
> > [1] https://bugs.launchpad.net/ceilometer/+bug/1323524
> > [2] https://bugs.launchpad.net/ceilometer/+bug/1327344
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140610/923086fa/attachment.html>