Open Stack

Can you check dmesg –T to see at least if the messages are not too old?
Are you using port2 as a connection to VM?



Lenny Verkhovsky

From: Eduard Matei [mailto:eduard.matei at cloudfounders.com]
Sent: Wednesday, June 10, 2015 10:34 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] ThirdPartyCI: Tempest job fails with "Unexpected termination of the channel"

Short story:
- all runs of the tempest job are failing after ~ 1,5 - 2 hrs with  "FATAL: java.io.IOException: Unexpected termination of the channel" and jenkins logs "SEVERE: I/O error in channel d-p-c-local_01-1655 java.io.IOException: Unexpected termination of the channel"

Long story:
- Host is 4 core Intel Xeon, 32GB RAM, 3x1TB HDD
-- devstack is 2015.1.1, networking is done via n-net
- Jenkins master is a VM, 2 core, 8 GB RAM, 500 GB disk
-- not managed through devstack, but using virsh, networking done manually via iptables
- test slaves are VMs, m1.large (4 VCPU, 8 GB RAM, trusty), managed by nodepool

Load on master is usually around 6-7, memory usage around 90%, little swapping.

I tried increasing ClientAliveInterval/ServerAliveInterval and ClientAliveCountMax/ServerAliveCountMax on both the Jenkins VM and on the test vm (using the job to configure), but it still fails with the above mentioned error.

On the host, dmesg is full of: (not sure if directly related to this)
[1037245.231196] br100: port 2(vnet1) entered disabled state
followed by
[1037259.147672] br100: port 2(vnet1) entered forwarding state

I think it might be something related to networking on the host itself but i couldn't figure it yet.

Any suggestion on what to try next?

Thanks,
Eduard

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150610/0080d0a8/attachment.html>