<div dir="ltr"><div class="gmail_default" style="font-family:monospace,monospace;font-size:small">here is what i found by using strace. any idea?</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small">( more log please see the attachment )</div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small"><br></div><div class="gmail_default" style="font-family:monospace,monospace;font-size:small"><pre style="color:rgb(0,0,0)">[pid 73] <... close resumed> ) = 0
[pid 67] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=71, si_status=0, si_utime=0, si_stime=0} ---
[pid 73] close(2 <unfinished ...>
[pid 72] brk(0xad4000 <unfinished ...>
[pid 73] <... close resumed> ) = 0
[pid 67] execve(\"/usr/lib64/erlang/erts-5.10.4/bin/beam.smp\", [\"/usr/lib64/erlang/erts-5.10.4/bi\"..., \"--\", \"-root\", \"/usr/lib64/erlang\", \"-progname\", \"erl\", \"--\", \"-home\", \"/var/lib/rabbitmq\", \"-epmd_port\", \"4369\", \"--\", \"-pa\", \"/usr/lib/rabbitmq/lib/rabbitmq_s\"..., \"-boot\", \"start_clean\", ...], [/* 18 vars */] <unfinished ...>
[pid 73] close(3) = -1 EBADF (Bad file descriptor)
[pid 72] <... brk resumed> ) = 0xad4000
[pid 73] close(4 <unfinished ...>
[pid 72] brk(0 <unfinished ...>
[pid 73] <... close resumed> ) = -1 EBADF (Bad file descriptor)
[pid 72] <... brk resumed> ) = 0xad4000
[pid 73] close(5) = -1 EBADF (Bad file descriptor)
[pid 73] close(6) = -1 EBADF (Bad file descriptor)
[pid 73] close(7) = -1 EBADF (Bad file descriptor)</pre></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, May 11, 2016 at 12:51 AM, Steven Dake (stdake) <span dir="ltr"><<a href="mailto:stdake@cisco.com" target="_blank">stdake@cisco.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Paul,<br>
<br>
Please run strace with -f (trace child processes).<br>
<br>
What you have there is not sufficient to do the job. FWIW erlang -16 is<br>
totally busted in CentOS because of logging, and erlang -17 is ready for<br>
testing which fixes the erlang crashes. Erlang-15 introduced ipv6 which<br>
could also be the cause of the crashes seen in the gate since they never<br>
happened on 14 and prior. It could also be a bug introduced in -16, so<br>
testing -17 would be helpful.<br>
<br>
See:<br>
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1324922" rel="noreferrer" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1324922</a><br>
<br>
<br>
Regards<br>
-steve<br>
<div class="HOEnZb"><div class="h5"><br>
On 5/10/16, 8:53 AM, "Paul Bourke" <<a href="mailto:paul.bourke@oracle.com">paul.bourke@oracle.com</a>> wrote:<br>
<br>
>I have a debug job open atm to try and investigate this:<br>
><a href="https://review.openstack.org/#/c/300988/" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/300988/</a><br>
><br>
>If anyone is handy with strace here is a run with the output:<br>
><br>
><a href="http://logs.openstack.org/88/300988/6/check/gate-kolla-dsvm-deploy-centos-" rel="noreferrer" target="_blank">http://logs.openstack.org/88/300988/6/check/gate-kolla-dsvm-deploy-centos-</a><br>
>binary/726c3b1/console.html<br>
><br>
>On 09/05/16 18:23, Hui Kang wrote:<br>
>> The ubuntu gate deploy failed too; that is awkward.<br>
>><br>
>><br>
>><a href="http://logs.openstack.org/05/314205/1/check/gate-kolla-dsvm-deploy-ubuntu" rel="noreferrer" target="_blank">http://logs.openstack.org/05/314205/1/check/gate-kolla-dsvm-deploy-ubuntu</a><br>
>>-source/766ee43/console.html#_2016-05-09_16_58_39_411<br>
>><br>
>> - Hui<br>
>><br>
>> On Mon, May 9, 2016 at 1:48 AM, Jeffrey Zhang <<a href="mailto:zhang.lei.fly@gmail.com">zhang.lei.fly@gmail.com</a>><br>
>>wrote:<br>
>>> I deploy the Kolla by using centos+source on centos host always.<br>
>>> Never see such kinda of issue. So i can not re-produce this<br>
>>> locally, now.<br>
>>><br>
>>> On Mon, May 9, 2016 at 8:31 AM, Hui Kang <<a href="mailto:hkang.sunysb@gmail.com">hkang.sunysb@gmail.com</a>><br>
>>>wrote:<br>
>>>><br>
>>>> Hi, Jeffrey,<br>
>>>> I tried deploying centos binary and source on my ubuntu host, which<br>
>>>> completed successfully. Looking at the VMs on the failed gate, they<br>
>>>> are centos VMs.<br>
>>>><br>
>>>> I think we need to debug this problem locally by deploying centos<br>
>>>> kolla on centos hosts. Is my understanding correct? Thanks.<br>
>>>><br>
>>>> - Hui<br>
>>>><br>
>>>> On Sat, May 7, 2016 at 9:54 AM, Jeffrey Zhang<br>
>>>><<a href="mailto:zhang.lei.fly@gmail.com">zhang.lei.fly@gmail.com</a>><br>
>>>> wrote:<br>
>>>>> Recently, the centos binary and source gate failed due to the<br>
>>>>>rabbitmq<br>
>>>>> container<br>
>>>>> existed. After making some debug. I do not found the root cause.<br>
>>>>><br>
>>>>> does anyone has any idea for this?<br>
>>>>><br>
>>>>> see this PS gate result[0]<br>
>>>>> centos binary gate failed[1]<br>
>>>>> CentOS source gate failed[2]<br>
>>>>><br>
>>>>> [0] <a href="https://review.openstack.org/313838" rel="noreferrer" target="_blank">https://review.openstack.org/313838</a><br>
>>>>> [1]<br>
>>>>><br>
>>>>><br>
>>>>><a href="http://logs.openstack.org/38/313838/1/check/gate-kolla-dsvm-deploy-cen" rel="noreferrer" target="_blank">http://logs.openstack.org/38/313838/1/check/gate-kolla-dsvm-deploy-cen</a><br>
>>>>>tos-binary/ea293fe/<br>
>>>>> [2]<br>
>>>>><br>
>>>>><br>
>>>>><a href="http://logs.openstack.org/38/313838/1/check/gate-kolla-dsvm-deploy-cen" rel="noreferrer" target="_blank">http://logs.openstack.org/38/313838/1/check/gate-kolla-dsvm-deploy-cen</a><br>
>>>>>tos-source/d4cb127/<br>
>>>>><br>
>>>>> --<br>
>>>>> Regards,<br>
>>>>> Jeffrey Zhang<br>
>>>>> Blog: <a href="http://xcodest.me" rel="noreferrer" target="_blank">http://xcodest.me</a><br>
>>>>><br>
>>>>><br>
>>>>><br>
>>>>>______________________________________________________________________<br>
>>>>>____<br>
>>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>>> Unsubscribe:<br>
>>>>> <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>>>><br>
>>>><br>
>>>><br>
>>>>_______________________________________________________________________<br>
>>>>___<br>
>>>> OpenStack Development Mailing List (not for usage questions)<br>
>>>> Unsubscribe:<br>
>>>><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>><br>
>>><br>
>>><br>
>>><br>
>>> --<br>
>>> Regards,<br>
>>> Jeffrey Zhang<br>
>>> Blog: <a href="http://xcodest.me" rel="noreferrer" target="_blank">http://xcodest.me</a><br>
>>><br>
>>><br>
>>>________________________________________________________________________<br>
>>>__<br>
>>> OpenStack Development Mailing List (not for usage questions)<br>
>>> Unsubscribe:<br>
>>><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>>><br>
>><br>
>><br>
>>_________________________________________________________________________<br>
>>_<br>
>> OpenStack Development Mailing List (not for usage questions)<br>
>> Unsubscribe:<br>
>><a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
>><br>
><br>
>__________________________________________________________________________<br>
>OpenStack Development Mailing List (not for usage questions)<br>
>Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Regards,</font></span></div><div><span style="font-size:13px;border-collapse:collapse"><font face="monospace, monospace">Jeffrey Zhang</font></span></div><div><span style="font-family:monospace,monospace;font-size:12.8px">Blog: </span><a href="http://xcodest.me/" style="font-family:monospace,monospace;font-size:12.8px" target="_blank">http://xcodest.me</a><font face="monospace, monospace"><br></font></div></div></div></div></div></div></div></div></div>
</div>