Just got luck to reproduce it again!.<div><br></div><div>All Recv-Q/Send-Q are empty, both controller side and compute node side.</div><div>--</div><div><div>root@10.81.44.230:~# netstat -ant | grep -E 'Recv-Q|5672'                                                </div>

<div>Proto Recv-Q Send-Q Local Address           Foreign Address         State      </div><div>tcp        0      0 <a href="http://0.0.0.0:55672">0.0.0.0:55672</a>           0.0.0.0:*               LISTEN     </div><div>
tcp        0      0 <a href="http://127.0.0.1:57497">127.0.0.1:57497</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>
<div>tcp        0      0 <a href="http://127.0.0.1:57499">127.0.0.1:57499</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56775">127.0.0.1:56775</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp        0      0 <a href="http://127.0.0.1:56877">127.0.0.1:56877</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56866">127.0.0.1:56866</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp        0      0 <a href="http://127.0.0.1:57507">127.0.0.1:57507</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56778">127.0.0.1:56778</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp        0      0 <a href="http://127.0.0.1:56779">127.0.0.1:56779</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56491">127.0.0.1:56491</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp        0      0 <a href="http://127.0.0.1:57508">127.0.0.1:57508</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56777">127.0.0.1:56777</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp        0      0 <a href="http://127.0.0.1:56776">127.0.0.1:56776</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div><div>tcp        0      0 <a href="http://127.0.0.1:56493">127.0.0.1:56493</a>         <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          ESTABLISHED</div>

<div>tcp6       0      0 :::5672                 :::*                    LISTEN     </div><div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.139.31:59842">10.46.139.31:59842</a>      ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.178.20:51570">10.46.178.20:51570</a>      ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:57508">127.0.0.1:57508</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.139.31:59843">10.46.139.31:59843</a>      ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56775">127.0.0.1:56775</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.178.20:59565">10.46.178.20:59565</a>      ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:57497">127.0.0.1:57497</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56779">127.0.0.1:56779</a>         ESTABLISHED</div><div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.139.171:54251">10.46.139.171:54251</a>     ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.178.20:54857">10.46.178.20:54857</a>      ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:57499">127.0.0.1:57499</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:57507">127.0.0.1:57507</a>         ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56877">127.0.0.1:56877</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56866">127.0.0.1:56866</a>         ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56778">127.0.0.1:56778</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56777">127.0.0.1:56777</a>         ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56491">127.0.0.1:56491</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>       <a href="http://10.46.139.171:54250">10.46.139.171:54250</a>     ESTABLISHED</div><div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56493">127.0.0.1:56493</a>         ESTABLISHED</div>

<div>tcp6       0      0 <a href="http://127.0.0.1:5672">127.0.0.1:5672</a>          <a href="http://127.0.0.1:56776">127.0.0.1:56776</a>         ESTABLISHED</div></div><div><br></div><div><br></div><div><div>[<a href="mailto:root@10.46.178.20">root@10.46.178.20</a> log]# netstat -ant | grep -E 'Recv-Q|5672'</div>

<div>Proto Recv-Q Send-Q Local Address               Foreign Address             State      </div><div>tcp        0      0 <a href="http://10.46.178.20:59565">10.46.178.20:59565</a>          <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>           ESTABLISHED </div>

<div>tcp        0      0 <a href="http://10.46.178.20:59441">10.46.178.20:59441</a>          <a href="http://10.81.44.230:5672">10.81.44.230:5672</a>           ESTABLISHED </div></div><div><br></div><div><div>--<br>Qiu Yu<br>

<a href="http://www.unicell.info">http://www.unicell.info</a></div><br>
<br><br><div class="gmail_quote">On Wed, Dec 19, 2012 at 8:59 PM, unicell <span dir="ltr"><<a href="mailto:unicell@gmail.com" target="_blank">unicell@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">

Thank you Wen Jian. I don't possess the environment anymore. :-(  So I'll try to follow your advice whenever it is reproducible.<div><div><br><div>Posting this message here trying to collect more ideas, in case it happens again.<br>


<div><br clear="all"><div>--<br>Qiu Yu<br></div><br><br><div class="gmail_quote">On Wed, Dec 19, 2012 at 8:41 PM, Jian Wen <span dir="ltr"><<a href="mailto:wenjian@canonical.com" target="_blank">wenjian@canonical.com</a>></span> wrote:<br>


<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">, the TCP connection is</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>