<div dir="ltr">Can you do a 'neutron port-show' for both of those HA ports to check their status field?</div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Dec 6, 2016 at 2:29 AM, Grant Morley <span dir="ltr"><<a href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <p>Hi Kevin & Neil,</p>
    <p>Many thanks for the reply. I have attached a screen shot showing
      that we cannot ping between the L3 HA nodes on the router name
      spaces. This was previously working fine with Mitaka, and has only
      stopped working since the upgrade to Newton.</p>
    <p>From the packet captures and TCP dumps, the traffic doesn't seem
      to be even leaving the namespace.</p>
    <p>On the attachment, the left hand side shows the state of
      keepalived showing both HA agents as master and the ring hand side
      is the ping attempt.</p>
    <p>Regards,<br>
    </p><div><div class="h5">
    <div class="m_4273622011722304119moz-cite-prefix">On 06/12/16 10:14, Kevin Benton wrote:<br>
    </div>
    <blockquote type="cite">
      <div dir="ltr">Yes, that is a misleading warning. What is
        happening is that it's trying to load the interface driver as an
        alias first, which results in a stevedore warning that you see
        and then it falls back to loading it by the class path, which is
        what you have configured. We will need to see if there is a way
        we can suppress that warning somehow when we make the call to
        load by an alias and it fails.
        <div><br>
        </div>
        <div>If you switch your interface to just 'linuxbridge', that
          should get rid of the warning.</div>
        <div><br>
          <div><br>
          </div>
          <div>For both L3 HA nodes becoming master, we need a little
            more info to figure out the root cause. Can you try
            switching into the router namespace on one of the L3 HA
            nodes and see if you can ping the other router instance
            across the L3 HA network for that router?</div>
        </div>
      </div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Mon, Dec 5, 2016 at 7:54 AM, Neil
          Jerram <span dir="ltr"><<a href="mailto:neil@tigera.io" target="_blank">neil@tigera.io</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div dir="ltr">
              <div>
                <div>
                  <div>I have also recently been seeing 'Could not load
                    <whatever>InterfaceDriver' warnings from the
                    DHCP agent, and haven't yet understood that -
                    although I'm pretty sure that my interface driver is
                    being loaded really - or else none of my networking
                    function would work at all.<br>
                    <br>
                  </div>
                  So it's possible that that part of your report is
                  benign, and just a misleading warning.  That said, I
                  am still worried about it too, and would like to
                  understand it properly.<br>
                  <br>
                </div>
                I'm not aware of seeing the other symptoms you
                mentioned.<br>
                <br>
              </div>
                   Neil<br>
              <br>
            </div>
            <br>
            <div class="gmail_quote">
              <div>
                <div class="m_4273622011722304119h5">
                  <div dir="ltr">On Mon, Dec 5, 2016 at 3:14 PM Grant
                    Morley <<a href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a>>
                    wrote:<br>
                  </div>
                </div>
              </div>
              <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
                <div>
                  <div class="m_4273622011722304119h5">
                    <div text="#000000" bgcolor="#FFFFFF" class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">Hi All,</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">We have
                        just upgraded from Mitaka to Newton. We are
                        running OSA and we seem to have come across some
                        weird networking issues since the upgrade.
                        Basically network access to instances is very
                        intermittent and seems to randomly stop working.
                        <br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                      </p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">We are
                        running neutron in HA and it appears that both
                        of the neutron nodes are now trying to be master
                        and are both trying to bring up the gateway IP
                        addresses which would be causing conflicts. <br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                      </p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">We are
                        also seeing a lot of the following in the
                        "neutron-dhcp-agent" log files:</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">2016-12-05
                        14:42:24.837 2020 WARNING stevedore.named
                        [req-1955d0a1-1453-4c65-a93a-5<wbr>4e8ea39b230
                        1ac995c0729142289f7237222f3358<wbr>06
                        3cc95dbe91c84e3e8ebbb9893ee54d<wbr>20 - - -]
                        Could not load neutron.agent.linux.interface.<wbr>BridgeInterfaceDriver<br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                        2016-12-05 14:42:42.803 2020 INFO
                        neutron.agent.dhcp.agent
                        [req-fad7d2bb-9d3c-4192-868a-0<wbr>164b382aecf
                        1ac995c0729142289f7237222f3358<wbr>06
                        3cc95dbe91c84e3e8ebbb9893ee54d<wbr>20 - - -]
                        Trigger reload_allocations for port
                        admin_state_up=True, allowed_address_pairs=[],
                        binding:host_id=, binding:profile=,
                        binding:vif_details=, binding:vif_type=unbound,
                        binding:vnic_type=normal,
                        created_at=2016-12-05T14:42:42<wbr>Z,
                        description=, device_id=8752effa-2ff2-4ce1-b<wbr>e70-e9f2243612cb,
                        device_owner=network:floatingi<wbr>p,
                        extra_dhcp_opts=[], fixed_ips=[{u'subnet_id':
                        u'4ca7db2d-544a-4a97-b5a4-3cbf<wbr>2467a4b7',
                        u'ip_address': u'XXX.XXX.XXX.XXX'}],
                        id=b3cf223d-8e76-484a-a649-d8a<wbr>7dd435124,
                        mac_address=fa:16:3e:ff:0d:50, name=,
                        network_id=af5db886-0178-4f8d-<wbr>9189-f55f773b37fa,
                        port_security_enabled=False, project_id=,
                        revision_number=4, security_groups=[],
                        status=N/A, tenant_id=,
                        updated_at=2016-12-05T14:42:42<wbr>Z</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">I am a
                        bit concerned about neutron not being able to
                        load the Bridge interface driver.</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">Has
                        anyone else come across this at all or have any
                        pointers? This was working fine in Mitaka it
                        just seems since the upgrade to Newton, we have
                        these issues.</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">I am
                        able to provide more logs if they are needed.</p>
                      <p class="m_4273622011722304119m_-4652561670223368530gmail_msg">Regards,<br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                      </p>
                      <div class="m_4273622011722304119m_-4652561670223368530m_-994859292978489524moz-signature m_4273622011722304119m_-4652561670223368530gmail_msg">-- <br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                        <div style="height:13px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> </div>
                        <div class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <font class="m_4273622011722304119m_-4652561670223368530gmail_msg" face="Lucida Grande, Verdana, Arial,
                            sans-serif" color="#A8D8FF"> <span style="font-size:11px;line-height:15px;float:left;margin-right:10px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <img class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                            </span> </font>
                          <div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:5px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <span class="m_4273622011722304119m_-4652561670223368530gmail_msg">Grant
                              Morley</span> <span style="color:rgb(35,157,255);font-size:15px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> </span>
                            <font class="m_4273622011722304119m_-4652561670223368530gmail_msg" face="Hind, Lucida Grande, Verdana, Arial,
                              sans-serif"> <span style="font-size:15px" class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                              </span> </font> </div>
                          <div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:10px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <font class="m_4273622011722304119m_-4652561670223368530gmail_msg" face="Hind, Lucida Grande, Verdana, Arial,
                              sans-serif"> <span style="font-size:15px" class="m_4273622011722304119m_-4652561670223368530gmail_msg">Cloud
                                Lead</span> </font> </div>
                          <div class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <span style="color:#616161" class="m_4273622011722304119m_-4652561670223368530gmail_msg">Absolute</span>
                            <span style="color:#98a5b0" class="m_4273622011722304119m_-4652561670223368530gmail_msg">DevOps
                              Ltd</span> <br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                            Units H, J & K, Gateway 1000, Whittle
                            Way, Stevenage, Herts, SG1 2FP </div>
                          <div style="font-family:Hind,helvetica,arial;font-size:13px;margin-left:80px" class="m_4273622011722304119m_-4652561670223368530gmail_msg"> <a class="m_4273622011722304119m_-4652561670223368530m_-994859292978489524moz-txt-link-abbreviated m_4273622011722304119m_-4652561670223368530gmail_msg" href="http://www.absolutedevops.io" target="_blank"></a><a class="m_4273622011722304119moz-txt-link-abbreviated" href="http://www.absolutedevops.io" target="_blank">www.absolutedevops.io</a>
                            <a class="m_4273622011722304119m_-4652561670223368530m_-994859292978489524moz-txt-link-abbreviated m_4273622011722304119m_-4652561670223368530gmail_msg" href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a>
                            <a class="m_4273622011722304119m_-4652561670223368530gmail_msg">0845
                              874 0580</a> </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
                ______________________________<wbr>_________________<br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                OpenStack-operators mailing list<br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                <a href="mailto:OpenStack-operators@lists.openstack.org" class="m_4273622011722304119m_-4652561670223368530gmail_msg" target="_blank">OpenStack-operators@lists.open<wbr>stack.org</a><br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
                <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" class="m_4273622011722304119m_-4652561670223368530gmail_msg" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-operators</a><br class="m_4273622011722304119m_-4652561670223368530gmail_msg">
              </blockquote>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            OpenStack-operators mailing list<br>
            <a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.open<wbr>stack.org</a><br>
            <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-operators</a><br>
            <br>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
    <div class="m_4273622011722304119moz-signature">-- <br>
      
      
      <div style="height:13px"> </div>
      <div> <font face="Lucida Grande, Verdana, Arial, sans-serif" color="#A8D8FF"> <span style="font-size:11px;line-height:15px;float:left;margin-right:10px"> <img src="cid:part11.7380B9D3.B0DA9307@absolutedevops.io" style="color:#a8d8ff;font-family:'Lucida Grande',Verdana,Arial,sans-serif;font-size:11px;line-height:15px;float:left;margin-top:0px;margin-right:10px"> </span>
        </font>
        <div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:5px"> <span style="font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;font-size:18px;font-weight:200;color:rgb(35,157,255)">Grant Morley</span> <span style="color:rgb(35,157,255);font-size:15px"> </span> <font face="Hind, Lucida Grande, Verdana, Arial, sans-serif"> <span style="font-size:15px"> </span> </font> </div>
        <div style="margin-left:80px;line-height:15px;color:rgb(109,190,255);margin-bottom:10px"> <font face="Hind,
            Lucida Grande, Verdana, Arial, sans-serif"> <span style="font-size:15px">Cloud Lead</span> </font> </div>
        <div style="font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;font-size:13px;margin-left:80px;line-height:15px;color:rgb(153,153,153)"> <span style="color:#616161">Absolute</span> <span style="color:#98a5b0">DevOps
            Ltd</span> <br>
          Units H, J & K, Gateway 1000, Whittle Way, Stevenage,
          Herts, SG1 2FP
        </div>
        <div style="font-family:Hind,helvetica,arial;font-size:13px;margin-left:80px"> <a href="http://www.absolutedevops.io/" style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px;text-decoration:none" target="_blank"></a><a class="m_4273622011722304119moz-txt-link-abbreviated" href="http://www.absolutedevops.io" target="_blank">www.absolutedevops.io</a> <a href="mailto:grant@absolutedevops.i" style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px;text-decoration:none" target="_blank"></a><a class="m_4273622011722304119moz-txt-link-abbreviated" href="mailto:grant@absolutedevops.io" target="_blank">grant@absolutedevops.io</a> <a style="color:#239dff;font-family:Hind,'Lucida Grande',Verdana,Arial,sans-serif;line-height:15px;margin-right:6px">0845 874 0580</a> </div>
      </div>
    </div>
  </div></div></div>

</blockquote></div><br></div>