[openstack-dev] Gate Status - Friday Edition
sorlando at nicira.com
Fri Jan 24 23:57:51 UTC 2014
I've found out that several jobs are exhibiting failures like bug 1254890
 and bug 1253896  because openvswitch seem to be crashing the kernel.
The kernel trace reports as offending process usually either
neutron-ns-metadata-proxy or dnsmasq, but  seem to clearly point to
254 events observed in the previous 6 days show a similar trace in the logs
This means that while this alone won't explain all the failures observed,
it is however potentially one of the prominent root causes.
>From the logs I have little hints about the kernel running. It seems there
has been no update in the past 7 days, but I can't be sure.
Openvswitch builds are updated periodically. The last build I found not to
trigger failures was the one generated on 2014/01/16 at 01:58:18.
Unfortunately version-wise I always have only 1.4.0, no build number.
I don't know if this will require getting in touch with ubuntu, or if we
can just prep a different image which an OVS build known to work without
 "kernel BUG at /build/buildd/linux-3.2.0/fs/buffer.c:2917" and
On 24 January 2014 21:13, Clay Gerrard <clay.gerrard at gmail.com> wrote:
> OH yeah that's much better. I had found those eventually but had to dig
> through all that other stuff :'(
> Moving forward I think we can keep an eye on that page, open bugs for
> those tests causing issue and dig in.
> Thanks again!
> On Fri, Jan 24, 2014 at 11:37 AM, Sean Dague <sean at dague.net> wrote:
>> On 01/24/2014 02:02 PM, Peter Portante wrote:
>> > Hi Sean,
>> > In the last 7 days I see only 6 python27 based test
>> > failures:
>> > And 4 python26 based test
>> > failures:
>> > Maybe the query you posted captures failures where the job did not even
>> > And only 15 hits (well, 18, but three are within the same job, and some
>> > of the tests are run twice, so it is a combined of 10
>> > hits):
>> > Thanks,
>> So it is true, that the Interupted exceptions (which is when a job is
>> killed because of a reset) are some times being turned into Fail events
>> by the system, which is one of the reasons the graphite data for
>> failures is incorrect, and if you use just the graphite sourcing for
>> fails, your numbers will be overly pessimistic.
>> The following is probably better lists
>> (7 uncategorized fails)
>> (5 uncategorized fails)
>> Sean Dague
>> Samsung Research America
>> sean at dague.net / sean.dague at samsung.com
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev