<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    are you running RDO / Juju? What is the version?<br>
    <br>
    Thanks <br>
    <br>
    <div class="moz-cite-prefix">On 9/18/17 6:40 PM, Jean-Philippe
      Méthot wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:C6BB780C-D983-4AE9-995F-11E41D2B6FED@planethoster.info">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
      Hi,
      <div class=""><br class="">
      </div>
      <div class="">Thank you for your reply. We did restart all neutron
        services, several times. We also restarted the servers but the
        issue is still there.</div>
      <div class=""><br class="">
      </div>
      <div class="">Best regards,</div>
      <div class=""><br class="">
      </div>
      <div class="">
        <div class="">
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;">Jean-Philippe Méthot</div>
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;">Openstack system
            administrator</div>
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;">Administrateur système
            Openstack</div>
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;">PlanetHoster inc.</div>
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;"><br class="">
          </div>
          <div style="color: rgb(0, 0, 0); font-family: Helvetica;
            font-size: 12px; font-style: normal; font-variant-caps:
            normal; font-weight: normal; letter-spacing: normal;
            orphans: auto; text-align: start; text-indent: 0px;
            text-transform: none; white-space: normal; widows: auto;
            word-spacing: 0px; -webkit-text-size-adjust: auto;
            -webkit-text-stroke-width: 0px;" class=""><br class="">
          </div>
          <br class="Apple-interchange-newline">
        </div>
        <br class="">
        <div>
          <blockquote type="cite" class="">
            <div class="">Le 19 sept. 2017 à 10:01, Remo Mattei <<a
                href="mailto:remo@italy1.com" class=""
                moz-do-not-send="true">remo@italy1.com</a>> a écrit :</div>
            <br class="Apple-interchange-newline">
            <div class="">
              <meta http-equiv="content-type" content="text/html;
                charset=utf-8" class="">
              <div dir="auto" class="">
                <div class="">I saw something similar did you restart
                  all the services after the upgrade? Just wonder. I saw
                  some other issue when I upgraded from 7.3 to 7.4 where
                  it gave me some vif error after all servers reboot the
                  problem has been gone. </div>
                <div class=""><br class="">
                </div>
                <div class="">Let me know. </div>
                <div class=""><br class="">
                  Il giorno 18 set 2017, alle ore 17:02, JP Japan <<a
                    href="mailto:jp.methot@planethoster.info" class=""
                    moz-do-not-send="true">jp.methot@planethoster.info</a>>
                  ha scritto:<br class="">
                  <br class="">
                </div>
                <div class="">
                  <meta http-equiv="Content-Type" content="text/html;
                    charset=utf-8" class="">
                  <div class="">Sorry, I ended up sending the previous
                    email a bit too quickly. Here’s some more info about
                    our setup.</div>
                  <div class=""><br class="">
                  </div>
                  <div class="">-It’s running latest Ocata with
                    Openvswitch and network dedicated nodes.</div>
                  <div class="">-The network nodes are L3HA</div>
                  <div class="">-There’s no DVR here.</div>
                  <br class="">
                  <div class="">
                    <blockquote type="cite" class="">
                      <div class="">Le 19 sept. 2017 à 08:51, JP Japan
                        <<a href="mailto:jp.methot@planethoster.info"
                          class="" moz-do-not-send="true">jp.methot@planethoster.info</a>>
                        a écrit :</div>
                      <br class="Apple-interchange-newline">
                      <div class="">
                        <meta http-equiv="Content-Type"
                          content="text/html; charset=utf-8" class="">
                        <div style="word-wrap: break-word;
                          -webkit-nbsp-mode: space; -webkit-line-break:
                          after-white-space;" class="">Hi,
                          <div class=""><br class="">
                          </div>
                          <div class="">A few days ago, we made two big
                            changes on our production infrastructure: we
                            updated to latest Ocata and we changed the
                            outgoing port on our network node to a lacp
                            port. We made the change by switching the
                            port in br-ex in openvswitch to the new
                            lacp-backed port. Ever since these two
                            things happened right after the other, we’ve
                            ran into two issues, one which has much
                            worse consequences than the other:</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">1.We can’t add floating ips to
                            instances anymore. The interface says the
                            operation completed successfully, the
                            database gets updated, but the IP address
                            doesn’t exist in the network namespace on
                            the network nodes. Strangely enough, the
                            iptables rules in the NAT table do exist.
                            The port just doesn’t receive the new
                            address. Adding the floating ip address
                            manually to the virtual interface with "ip
                            netns exec *qrouter namespace id* ip addr
                            add *ip address* dev *virtual interface*"
                            solves this, but is in no way a permanent
                            solution.</div>
                          <div class=""><br class="">
                          </div>
                          <div class="">2.We’re getting an error message
                            in the L3-agent whenever it starts informing
                            us it was unable to add some rules in
                            iptables because there’s a lock on xtables,
                            while as far as we know, the L3-agent itself
                            is the one holding the lock. Here’s the
                            error: </div>
                          <div class=""><br class="">
                          </div>
                          <div class="">
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager # Generated by
                                iptables_manager</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager *nat</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager -I
                                neutron-l3-agent-PREROUTING 7 -d
                                169.254.169.254/32 -i qr-+ -p tcp -m tcp
                                --dport 80 -j REDIRECT --to-ports 9697</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager COMMIT</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager # Completed by
                                iptables_manager</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager ; Stdout: ;
                                Stderr: Another app is currently holding
                                the xtables lock. Perhaps you want to
                                use the -w option?</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager</span></div>
                            <div style="margin: 0px; line-height:
                              normal; font-family: Monaco;" class=""><span
                                style="font-kerning: none" class="">2017-09-18
                                13:00:55.426 18575 ERROR
                                neutron.callbacks.manager</span><span
                                style="line-height: normal; font-family:
                                Helvetica; -webkit-font-kerning: none;"
                                class=""> </span></div>
                          </div>
                          <div style="margin: 0px; line-height: normal;
                            font-family: Monaco;" class=""><span
                              style="line-height: normal; font-family:
                              Helvetica; -webkit-font-kerning: none;"
                              class=""><br class="">
                            </span></div>
                          <div style="margin: 0px; line-height: normal;
                            font-family: Monaco;" class=""><span
                              style="line-height: normal; font-family:
                              Helvetica; -webkit-font-kerning: none;"
                              class="">It’s not clear exactly how this
                              is affecting the setup, as metadata is
                              still going through properly (most likely
                              through the DHCP) but it’s quite worrying.</span></div>
                        </div>
                        _______________________________________________<br
                          class="">
                        Mailing list: <a
                          href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
                          class="" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br
                          class="">
                        Post to     : <a
                          href="mailto:openstack@lists.openstack.org"
                          class="" moz-do-not-send="true">openstack@lists.openstack.org</a><br
                          class="">
                        Unsubscribe : <a
                          href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
                          class="" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br
                          class="">
                      </div>
                    </blockquote>
                  </div>
                  <br class="">
                  <div class="">Jean-Philippe Méthot</div>
                  <div class="">Openstack system administrator</div>
                  <div class="">PlanetHoster inc.</div>
                </div>
                <div class=""><span class="">_______________________________________________</span><br
                    class="">
                  <span class="">Mailing list: <a
                      href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
                      class="" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a></span><br
                    class="">
                  <span class="">Post to     : <a
                      href="mailto:openstack@lists.openstack.org"
                      class="" moz-do-not-send="true">openstack@lists.openstack.org</a></span><br
                    class="">
                  <span class="">Unsubscribe : <a
                      href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack"
                      class="" moz-do-not-send="true">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a></span><br
                    class="">
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br class="">
      </div>
    </blockquote>
    <br>
  </body>
</html>