<div dir="ltr">Hi Priyanka,<div><br></div><div>Can you post dmesg and boot.log from your host?</div><div><br></div><div>Thank you</div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><br><br>
Best regards,<br>David S.<br>
------------------------------------------------<br>
e. <a href="mailto:david@zeromail.us" target="_blank">david@zeromail.us</a><br>
w. <a href="http://blog.pnyet.web.id" target="_blank">http://blog.pnyet.web.id</a></div></div></div>
<br><div class="gmail_quote">On Wed, Jun 3, 2015 at 10:22 AM, Priyanka <span dir="ltr"><<a href="mailto:ppnaik@cse.iitb.ac.in" target="_blank">ppnaik@cse.iitb.ac.in</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF" text="#000000">
    Hi,<br>
    <br>
    The hardware as well as software configurations are the same on all
    nodes.<br>
    <br>
    Thanks,<br>
    <br>
    Priyanka<div><div class="h5"><br>
    <br>
    <br>
    <div>On Tuesday 02 June 2015 05:49 PM,
      Shinobu Kinjo wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">
        <div>
          <div>Hi,<br>
          </div>
          <div><br>
            When were the node crashed?<br>
            How about other two nodes?<br>
          </div>
          Is there any difference between crashed node and others in
          terms of not only hardware but also software?<br>
          <br>
        </div>
         - Kinjo<br>
        <div>
          <div>
            <div class="gmail_extra"><br>
              <div class="gmail_quote">On Tue, Jun 2, 2015 at 3:31 PM,
                Priyanka <span dir="ltr"><<a href="mailto:ppnaik@cse.iitb.ac.in" target="_blank">ppnaik@cse.iitb.ac.in</a>></span>
                wrote:<br>
                <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sir,<br>
                  <br>
                  uname -r<br>
                  <br>
                  3.10.0-123.13.2.el7.x86_64<br>
                  <br>
                  I have stopped iptables services since the start. So,
                  should I still follow what they have mentioned in the
                  5th comment?<br>
                  <br>
                  Thanks,<br>
                  <br>
                  Priyanka<br>
                  <br>
                  <br>
                  On Tuesday 02 June 2015 11:49 AM, Matt Taylor wrote:<br>
                  <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                    Hi Priyanka,<br>
                    <br>
                    Are you using an old kernel ('uname -r' please)?
                    I've seen issues with unlink_anon_vmas in Fedora, so
                    this is rather interesting.<br>
                    <br>
                    In regards to the 2nd kernel panic, it indicates
                    that it's due to iptables and get_counters.<br>
                    <br>
                    It's possibly related to this:<br>
                    <br>
                    <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1089569" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1089569</a><br>
                    <br>
                    Might be worth trying what was mentioned in the 5th
                    comment.<br>
                    <br>
                    Regards,<br>
                    Matt.<br>
                    <br>
                    On 2/06/2015 15:14, Priyanka wrote:<br>
                    <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                      Hi,<br>
                      <br>
                      I have an Openstack Juno setup with one controller
                      node and 3 compute<br>
                      nodes. I installed it using packstack on centOS 7.
                      One of the compute<br>
                      node crashed twice. First crash was two days back
                      and second one today.<br>
                      It automatically restarts after the crash. The
                      crash log in /var/crash<br>
                      were different in both the instances.<br>
                      <br>
                      Crash 1 log:<br>
                      <br>
                      |CPU:  5  PID:  16617  Comm:  pickupNot
                      tainted3.10.0-123.13.2.el7.x86_64#1<br>
                      [11971.208387]  Hardware  name:  Intel 
                      Corporation S2600CP/S2600CP,  BIOS
                      SE5C600.86B.02.02.0002.122320131210 12/23/2013<br>
                      [11971.208389]   ffff8807de2b1cc800000000a2deb902
                      ffff8807de2b1c80 ffffffff815e232c<br>
                      [11971.208392]   ffff8807de2b1cb8 ffffffff8105dee1
                      ffff8807f8e85f50 ffff8807f8e85f40<br>
                      [11971.208394]   ffff8807f90db0c0 ffff8807f8e85f50
                      ffff8807f90db0c0 ffff8807de2b1d20<br>
                      [11971.208396]  Call  Trace:<br>
                      [11971.208401]   [<ffffffff815e232c>]
                      dump_stack+0x19/0x1b<br>
                      [11971.208405]   [<ffffffff8105dee1>]
                      warn_slowpath_common+0x61/0x80<br>
                      [11971.208408]   [<ffffffff8105df5c>]
                      warn_slowpath_fmt+0x5c/0x80<br>
                      [11971.208410]   [<ffffffff812cff82>]
                      __list_del_entry+0x82/0xd0<br>
                      [11971.208412]   [<ffffffff812cffdd>] 
                      list_del+0xd/0x30<br>
                      [11971.208415]   [<ffffffff811776a3>]
                      unlink_anon_vmas+0x93/0x180<br>
                      [11971.208418]   [<ffffffff81168b88>]
                      free_pgtables+0xa8/0x120<br>
                      [11971.208420]   [<ffffffff81173556>]
                      exit_mmap+0xc6/0x1a0<br>
                      [11971.208422]   [<ffffffff8105b187>] 
                      mmput+0x67/0xf0<br>
                      [11971.208424]   [<ffffffff81063dac>] 
                      do_exit+0x28c/0xa60<br>
                      [11971.208426]   [<ffffffff810645ff>]
                      do_group_exit+0x3f/0xa0<br>
                      [11971.208428]   [<ffffffff81064674>]
                      SyS_exit_group+0x14/0x20<br>
                      [11971.208431]   [<ffffffff815f2a19>]
                      system_call_fastpath+0x16/0x1b<br>
                      [11971.208432]  ---[  end  trace
                      ebed116bce4ce8eb]---<br>
                      [11971.208437]  BUG:  unable to handle kernel NULL
                      pointer dereference at(null)<br>
                      [11971.208471]  IP:  [<ffffffff81177663>]
                      unlink_anon_vmas+0x53/0x180<br>
                      [11971.208493]  PGD0<br>
                      [11971.208502]  Oops:  0000  [#1] SMP|<br>
                      <br>
                      Crash 2 log:<br>
                      <br>
                      |[321808.123092]  BUG:  unable to handle kernel
                      paging request at ffffc90017456008<br>
                      [321808.123122]  IP:  [<ffffffffa03a8521>]
                      get_counters+0x91/0xd0  [ip_tables]<br>
                      [321808.123146]  PGD81d437067  PUD81d4a4067 
                      PMD7ec309067  PTE0<br>
                      [321808.123167]  Oops:  0002  [#1] SMP<br>
                      [321808.123179]  Modules  linkedin:  dummy
                      vhost_net macvtap macvlan tun iptable_nat
                      nf_nat_ipv4 nf_nat iptable_raw iptable_filter
                      ip_tables nf_conntrack_ipv6 nf_defrag_ipv6 xt_mac
                      xt_physdev xt_set ip_set_hash_ip ip_set nfnetlink
                      veth ip6table_filter ip6_tables ebtable_nat
                      ebtables openvswitch vxlan ip_tunnel gre sg
                      ipt_REJECT xt_comment xt_conntrack xt_multiport
                      nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack
                      coretemp kvm_intel kvm crct10dif_pclmul iTCO_wdt
                      iTCO_vendor_support crc32_pclmul crc32c_intel
                      ghash_clmulni_intel sb_edac edac_core pcspkr
                      lpc_ich ioatdma i2c_i801 mfd_core aesni_intel lrw
                      gf128mul glue_helper ablk_helper cryptd mei_me mei
                      shpchp wmi acpi_cpufreq mperf nfsd auth_rpcgss
                      nfs_acl lockd sunrpc bridge stp llc xfs libcrc32c
                      sd_mod crc_t10di<br>
                        f crct10d<br>
                      <br>
                      if_common mgag200 syscopyarea sysfillrect<br>
                      [321808.123452]   sysimgblt drm_kms_helper ttm
                      isci igb ahci drm libsas libahci ptp
                      scsi_transport_sas pps_core dca libata
                      i2c_algo_bit i2c_core dm_mirror dm_region_hash
                      dm_log dm_mod[last  unloaded:  ip_tables]<br>
                      [321808.123521]  CPU:  10  PID:  110268  Comm:
                      iptables-saveNot 
                      tainted3.10.0-123.13.2.el7.x86_64#1<br>
                      [321808.123547]  Hardware  name:  Intel 
                      Corporation S2600CP/S2600CP,  BIOS
                      SE5C600.86B.02.02.0002.122320131210 12/23/2013<br>
                      [321808.123577]  task:  ffff88080a8f38e0 ti: 
                      ffff8807ed398000 task.ti:  ffff8807ed398000<br>
                      [321808.123599]  RIP: 
                      0010:[<ffffffffa03a8521>]
                      [<ffffffffa03a8521>] 
                      get_counters+0x91/0xd0  [ip_tables]<br>
                      [321808.123627]  RSP:  0018:ffff8807ed399da8 
                      EFLAGS:  00010286<br>
                      [321808.123643]  RAX:  ffffc90014ab72e8 RBX: 
                      0000000000010380 RCX:  ffffc90017456000<br>
                      [321808.123664]  RDX:  0000000000000054  RSI: 
                      0000000000000000 RDI:  ffff88081e290380<br>
                      [321808.123685]  RBP:  ffff8807ed399dc8 R08: 
                      0000000000000101 R09:  ffff8807ed96caa0<br>
                      [321808.123706]  R10:  0000000000000000  R11: 
                      ffffffff8117b2e8 R12:  ffffffff819e4aa0<br>
                      [321808.123727]  R13:  ffff8807ed96c800 R14: 
                      ffffc90017455000 R15:  ffff88080c961ba0<br>
                      [321808.123748]  FS:   00007fdcc2078740(0000)
                      GS:ffff88081d940000(0000)  knlGS:0000000000000000<br>
                      [321808.123772]  CS:   0010  DS:  0000  ES:  0000 
                      CR0: 0000000080050033<br>
                      ||[321808.123789] CR2: ffffc90017456008 CR3:
                      00000007ec30e000 CR4: 00000000001427e0<br>
                      [321808.123810] DR0: 0000000000000000 DR1:
                      0000000000000000 DR2: 0000000000000000<br>
                      [321808.123831] DR3: 0000000000000000 DR6:
                      00000000ffff0ff0 DR7: 0000000000000400<br>
                      |<br>
                      <br>
                      I am not able to debug the problem. The setup was
                      working fine till now.<br>
                      I am unable to understand what is cuasing such a
                      behaviour. Please help.<br>
                      <br>
                      <br>
                      Thanks,<br>
                      <br>
                      Priyanka<br>
                      <br>
                      <br>
                      <br>
                      _______________________________________________<br>
                      Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                      Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
                      Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                      <br>
                    </blockquote>
                    <br>
                    _______________________________________________<br>
                    Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                    Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
                    Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                  </blockquote>
                  <br>
                  <br>
                  _______________________________________________<br>
                  Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                  Post to     : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
                  Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
                </blockquote>
              </div>
              <br>
              <br clear="all">
              <br>
              -- <br>
              <div>
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div><a href="http://i-shinobu.hatenablog.com/" target="_blank">Life w/ Linux</a><br>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to     : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>