Open Stack

Short story:
- all runs of the tempest job are failing after ~ 1,5 - 2 hrs with  "FATAL:
java.io.IOException: Unexpected termination of the channel" and jenkins
logs "SEVERE: I/O error in channel d-p-c-local_01-1655 java.io.IOException:
Unexpected termination of the channel"

Long story:
- Host is 4 core Intel Xeon, 32GB RAM, 3x1TB HDD
-- devstack is 2015.1.1, networking is done via n-net
- Jenkins master is a VM, 2 core, 8 GB RAM, 500 GB disk
-- not managed through devstack, but using virsh, networking done manually
via iptables
- test slaves are VMs, m1.large (4 VCPU, 8 GB RAM, trusty), managed by
nodepool

Load on master is usually around 6-7, memory usage around 90%, little
swapping.

I tried increasing ClientAliveInterval/ServerAliveInterval and
ClientAliveCountMax/ServerAliveCountMax on both the Jenkins VM and on the
test vm (using the job to configure), but it still fails with the above
mentioned error.

On the host, dmesg is full of: (not sure if directly related to this)
[1037245.231196] br100: port 2(vnet1) entered disabled state
followed by
[1037259.147672] br100: port 2(vnet1) entered forwarding state

I think it might be something related to networking on the host itself but
i couldn't figure it yet.

Any suggestion on what to try next?

Thanks,
Eduard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150610/8ad61bac/attachment.html>