<html>
  <head>
    <meta content="text/html; charset=ISO-8859-1"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">Thanks for the explanation. Good to
      know.<br>
      <br>
      Le 11/03/2013 20:13, Samuel Winchenbach a écrit :<br>
    </div>
    <blockquote
cite="mid:CAK9s_NxSRGE-N9OWLFM1c2-8cBQsXe9JJzW8roUNsY63tRHTtw@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div class="gmail_default" style="font-family:'courier
          new',monospace">ugh...  I had an example file called
          "nova.conf" in /root</div>
        <div class="gmail_default" style="font-family:'courier
          new',monospace">
          <br>
        </div>
        <div class="gmail_default" style="font-family:'courier
          new',monospace">Apparently nova-manage looks for that file
          first.  Case closed.  I wish I could get the last week back.</div>
        <div class="gmail_default" style="font-family:'courier
          new',monospace">
          <br>
        </div>
        <div class="gmail_default" style="font-family:'courier
          new',monospace">Sam</div>
      </div>
      <div class="gmail_extra"><br>
        <br>
        <div class="gmail_quote">On Mon, Mar 11, 2013 at 2:37 PM,
          Miller, Mark M (EB SW Cloud - R&D - Corvallis) <span
            dir="ltr"><<a moz-do-not-send="true"
              href="mailto:mark.m.miller@hp.com" target="_blank">mark.m.miller@hp.com</a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div link="blue" vlink="purple" lang="EN-US">
              <div>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I
                    ran into a similar problem with the Grizzly-3
                    Keystone release. I had to specify keystone.conf as
                    the config-file with keystone-manage/ db_sync
                    command otherwise it would not use the mysql
                    statement in the keystone.conf file.</span></p>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Mark</span></p>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span></p>
                <div style="border:none;border-top:solid #b5c4df
                  1.0pt;padding:3.0pt 0in 0in 0in">
                  <p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
                      openstack-bounces+mark.m.miller=<a
                        moz-do-not-send="true"
                        href="mailto:hp.com@lists.launchpad.net"
                        target="_blank">hp.com@lists.launchpad.net</a>
                      [mailto:<a moz-do-not-send="true"
                        href="mailto:openstack-bounces%2Bmark.m.miller"
                        target="_blank">openstack-bounces+mark.m.miller</a>=<a
                        moz-do-not-send="true"
                        href="mailto:hp.com@lists.launchpad.net"
                        target="_blank">hp.com@lists.launchpad.net</a>]
                      <b>On Behalf Of </b>Samuel Winchenbach<br>
                      <b>Sent:</b> Monday, March 11, 2013 11:18 AM<br>
                      <b>To:</b> Sylvain Bauza<br>
                      <b>Cc:</b> <a moz-do-not-send="true"
                        href="mailto:openstack@lists.launchpad.net"
                        target="_blank">openstack@lists.launchpad.net</a><br>
                      <b>Subject:</b> Re: [Openstack] Incredibly odd
                      mysql permission error</span></p>
                </div>
                <div>
                  <div class="h5">
                    <p class="MsoNormal"> </p>
                    <div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New"">OK!!!!
                             Someone on the IRC channel got me closer,
                            but we have no idea why this would happen:</span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New""> </span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New"">this
                            works: <a class="moz-txt-link-rfc2396E" href="mailto:root@test1:~#nova-manage--config-file=/etc/nova/nova.confservicelist">"root@test1:~# nova-manage
                            --config-file=/etc/nova/nova.conf service
                            list"</a></span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New"">Why
                            would I have to specify the config file
                            though?  It is in the standard place.</span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New"">Thanks,</span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"><span
                            style="font-family:"Courier New"">Sam</span></p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                      <div>
                        <p class="MsoNormal"> </p>
                      </div>
                    </div>
                    <div>
                      <p class="MsoNormal" style="margin-bottom:12.0pt"> </p>
                      <div>
                        <p class="MsoNormal">On Mon, Mar 11, 2013 at
                          2:01 PM, Samuel Winchenbach <<a
                            moz-do-not-send="true"
                            href="mailto:swinchen@gmail.com"
                            target="_blank">swinchen@gmail.com</a>>
                          wrote:</p>
                        <div>
                          <div>
                            <p class="MsoNormal"><span
                                style="font-family:"Courier
                                New"">For completeness here the
                                routing table, and ip listing for both
                                test1 and test2.   Doubt this will help
                                much:  <a moz-do-not-send="true"
                                  href="http://paste2.org/p/3117125"
                                  target="_blank"><span
                                    style="font-family:"Arial","sans-serif"">http://paste2.org/p/3117125</span></a></span></p>
                          </div>
                        </div>
                        <div>
                          <div>
                            <div>
                              <p class="MsoNormal"
                                style="margin-bottom:12.0pt"> </p>
                              <div>
                                <p class="MsoNormal">On Mon, Mar 11,
                                  2013 at 1:52 PM, Samuel Winchenbach
                                  <<a moz-do-not-send="true"
                                    href="mailto:swinchen@gmail.com"
                                    target="_blank">swinchen@gmail.com</a>>
                                  wrote:</p>
                                <div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">#1 - No change</span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">#2 - All of grants
                                        are in the ip/mask form such as:
                                        'nova'@'<a
                                          moz-do-not-send="true"
                                          href="http://10.21.0.0/255.255.0.0"
                                          target="_blank">10.21.0.0/255.255.0.0</a>'
                                         I have also tried adding
                                        'nova'@'test1' and
                                        'nova'@'10.21.0.1'.  No change.</span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">#3 - I changed the
                                        SQL connection string over to IP
                                        instead of hostname.  No change.
                                         I didn't restart nova-api
                                        because it isn't running.   If I
                                        understand correctly nova-manage
                                        communicated directly with the
                                        db, bypassing nova-api.   This
                                        would appear true seeing
                                        "nova-manage service list" works
                                        correctly on test2.</span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New""> </span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New""> </span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">:(</span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New""> </span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">Thanks for the help!</span></p>
                                  </div>
                                  <div>
                                    <p class="MsoNormal"><span
                                        style="font-family:"Courier
                                        New"">Sam</span></p>
                                  </div>
                                </div>
                                <div>
                                  <div>
                                    <div>
                                      <p class="MsoNormal"
                                        style="margin-bottom:12.0pt"> </p>
                                      <div>
                                        <p class="MsoNormal">On Mon, Mar
                                          11, 2013 at 12:24 PM, Sylvain
                                          Bauza <<a
                                            moz-do-not-send="true"
                                            href="mailto:sylvain.bauza@digimind.com"
                                            target="_blank">sylvain.bauza@digimind.com</a>>
                                          wrote:</p>
                                        <div>
                                          <div>
                                            <p class="MsoNormal">When
                                              looking at MySQL 5.1
                                              refman (<a
                                                moz-do-not-send="true"
                                                href="http://dev.mysql.com/doc/refman/5.1/en/access-denied.html"
                                                target="_blank">http://dev.mysql.com/doc/refman/5.1/en/access-denied.html</a>
                                              ), I would suggest to
                                              follow the procedure :
                                              <br>
                                               1. 'mysqladmin
                                              flush-hosts'<br>
                                               2. replace DNS entries in
                                              mysql.user table by IP
                                              addresses instead<br>
                                               3. modify
                                              /etc/nova/nova.conf with
                                              IP address of HA Mysql
                                              instead (and restart
                                              nova-api !)<br>
                                              <br>
                                              I wouldn't bet on it, but
                                              I would say this is due to
                                              some name resolution which
                                              is incorrect.
                                              <br>
                                              <br>
                                              -Sylvain<br>
                                              <br>
                                              <br>
                                              Le 11/03/2013 17:00,
                                              Sylvain Bauza a écrit :</p>
                                          </div>
                                          <div>
                                            <div>
                                              <blockquote
                                                style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                <div>
                                                  <p class="MsoNormal">Ok,
                                                    lemme try to
                                                    summarize. <br>
                                                    You do have a DRBD
                                                    setup for MySQL
                                                    bound to a VIP
                                                    10.21.1.1 thanks to
                                                    Pacemaker. <br>
                                                    This setup is
                                                    relying on two
                                                    hosts, test1
                                                    (10.21.0.1) and
                                                    test2 (10.21.0.2).<br>
                                                    Your nova.conf is
                                                    pointing to mysql://<a
moz-do-not-send="true" href="http://10.21.1.1" target="_blank">10.21.1.1</a>
                                                    which is the VIP.<br>
                                                    <br>
                                                    Are you sure your
                                                    my.cnf is actually
                                                    the same in between
                                                    both DRBD nodes ? (I
                                                    would recommend to
                                                    symlink it to a
                                                    physical file hosted
                                                    on the DRBD device).<br>
                                                    <br>
                                                    One thing is hurting
                                                    me : you told me
                                                    that nova is also
                                                    pacemake'd. If so,
                                                    why can I still see
                                                    my_ip=10.21.0.2
                                                    (test2) ? It should
                                                    be pointing to
                                                    nova-ha (assuming
                                                    10.21.2.4 as per
                                                    /etc/hosts).<br>
                                                    <br>
                                                    Also, as per my
                                                    understanding of
                                                    Pacemaker, DRBD
                                                    partition is setup
                                                    by default on test2,
                                                    correct ?<br>
                                                    <br>
                                                    <br>
                                                    Sorry, as per my
                                                    first reading, I
                                                    can't see anything
                                                    obvious. That said,
                                                    I'm not sure this is
                                                    a Nova bug, as the
                                                    tcpdump trace is
                                                    seeing a correct
                                                    MySQL connection
                                                    attempt. But maybe
                                                    I'm wrong ?<br>
                                                    <br>
                                                    Anyway, are you sure
                                                    you only have *one*
                                                    MySQL engine running
                                                    (either on test1 or
                                                    test2) and
                                                    nova-manage trying
                                                    to access this right
                                                    one ?<br>
                                                    <br>
                                                    Perms look good to
                                                    me. As it a test
                                                    setup, you could try
                                                    to unleash the
                                                    grants by deleting
                                                    them and allowing
                                                    nova@'%' to see if
                                                    it's a basic dns
                                                    mapping issue.<br>
                                                    <br>
                                                    -Sylvain<br>
                                                    <br>
                                                    <br>
                                                    <br>
                                                    Le 11/03/2013 16:09,
                                                    Samuel Winchenbach a
                                                    écrit :</p>
                                                </div>
                                                <blockquote
                                                  style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                  <div>
                                                    <p class="MsoNormal">I
                                                    </p>
                                                    <div>
                                                      <p
                                                        class="MsoNormal">enabled
                                                        general_log in
                                                        /etc/mysql/my.cnf
                                                         Here are the
                                                        results of
                                                        connecting from
                                                        "test1", "test2"
                                                        and using the
                                                        client:</p>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""><a
moz-do-not-send="true" href="http://paste2.org/p/3115525"
                                                          target="_blank">http://paste2.org/p/3115525</a></span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <p
                                                        class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">I
                                                          purposefully
                                                          used the real
                                                          password in
                                                          case there is
                                                          a problem with
                                                          it.
                                                        </span></p>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> I
                                                          changed before
                                                          submitting
                                                          post.</span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">here
                                                          is a raw
                                                          packet TCP
                                                          dump (tcpdump
                                                          -w rawdump
                                                          port 3306) of
                                                          an attempted
                                                          "nova-manage
                                                          service list"
                                                          from test1:  </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""><a
moz-do-not-send="true"
                                                          href="https://www.dropbox.com/s/u4cjzxv6w6bwwe6/rawdump"
target="_blank">https://www.dropbox.com/s/u4cjzxv6w6bwwe6/rawdump</a></span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">I
                                                          looked at it
                                                          with wireshark
                                                          and couldn't
                                                          see anything
                                                          that jumped
                                                          out at me as
                                                          incorrect.  I
                                                          have not yet
                                                          tried to
                                                          recreate the
                                                          salted
                                                          password.</span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"> </p>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">Here
                                                          is my
                                                          pacemaker
                                                          configuration
                                                          for mysql.  I
                                                          stripped out
                                                          openstack
                                                          services,
                                                          rabbitmq and
                                                          others for
                                                          clarity.  All
                                                          resources are
                                                          currently
                                                          disabled
                                                          (other than
                                                          MySQL): </span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""><a
moz-do-not-send="true" href="http://paste2.org/p/3115685"
                                                          target="_blank">http://paste2.org/p/3115685</a>
                                                        </span></p>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">Please
                                                          don't yell at
                                                          me for having
                                                          STONITH
                                                          disabled :P
                                                           This is a
                                                          testing
                                                          cluster and I
                                                          am working on
                                                          getting routed
                                                          to the IPMI
                                                          interface.</span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">/etc/hosts: </span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""><a
moz-do-not-send="true" href="http://paste2.org/p/3115713"
                                                          target="_blank">http://paste2.org/p/3115713</a>
                                                        </span></p>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">/etc/nova/nova.conf: </span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""><a
moz-do-not-send="true" href="http://paste2.org/p/3115739"
                                                          target="_blank">http://paste2.org/p/3115739</a>
                                                        </span></p>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <p
                                                        class="MsoNormal"> </p>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">If
                                                          there is
                                                          anything else
                                                          I can provide
                                                          you, please
                                                          let me know!
                                                           I have pulled
                                                          out most of my
                                                          hair at this
                                                          point!</span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New""> </span></p>
                                                      </div>
                                                    </div>
                                                    <div>
                                                      <div>
                                                        <p
                                                          class="MsoNormal"><span
                                                          style="font-family:"Courier
                                                          New"">Sam</span></p>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"> </p>
                                                    </div>
                                                    <div>
                                                      <p
                                                        class="MsoNormal"> </p>
                                                    </div>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"
style="margin-bottom:12.0pt"> </p>
                                                    <div>
                                                      <p
                                                        class="MsoNormal">On
                                                        Mon, Mar 11,
                                                        2013 at 10:11
                                                        AM, Sylvain
                                                        Bauza <<a
                                                          moz-do-not-send="true"
href="mailto:sylvain.bauza@digimind.com" target="_blank">sylvain.bauza@digimind.com</a>>
                                                        wrote:</p>
                                                      <div>
                                                        <div>
                                                          <p
                                                          class="MsoNormal">So
                                                          as to
                                                          reproduce the
                                                          nova-manage
                                                          SQL command, I
                                                          would
                                                          recommand to
                                                          tcpdump -A
                                                          port 3306 on
                                                          the host and
                                                          get the SQL
                                                          trace on
                                                          what's
                                                          failing.<br>
                                                          <br>
                                                          Could you
                                                          please explain
                                                          further what
                                                          is your HA
                                                          config ? Are
                                                          you using
                                                          pacemaker/heartbeat
                                                          or any VIP ?<br>
                                                          <br>
                                                          -Sylvain<br>
                                                          <br>
                                                          Le 11/03/2013
                                                          14:23, Samuel
                                                          Winchenbach a
                                                          écrit :</p>
                                                        </div>
                                                        <div>
                                                          <div>
                                                          <blockquote
                                                          style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">Does
                                                          anyone think
                                                          this could be
                                                          an openstack
                                                          bug?  I just
                                                          want to check
                                                          before
                                                          submitting a
                                                          bug report.</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">Sam</p>
                                                          </div>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"> </p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">On
                                                          Fri, Mar 8,
                                                          2013 at 4:02
                                                          PM, Jay Pipes
                                                          <<a
                                                          moz-do-not-send="true"
href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>
                                                          wrote:</p>
                                                          <p
                                                          class="MsoNormal">Sorry,
                                                          I really can't
                                                          think of
                                                          anything :(</p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><br>
                                                          On 03/08/2013
                                                          03:52 PM,
                                                          Samuel
                                                          Winchenbach
                                                          wrote:<br>
                                                          > I dropped
                                                          those users
                                                          and no change.<br>
                                                          ><br>
                                                          > I also
                                                          set up general
                                                          logging in
                                                          mysql but it
                                                          really doesn't
                                                          provide any<br>
                                                          >
                                                          additional
                                                          information.
                                                           Any idea for
                                                          a next step I
                                                          could take?<br>
                                                          ><br>
                                                          > I am
                                                          almost at the
                                                          point of
                                                          taking a
                                                          tcpdump and
                                                          trying to
                                                          recreate the<br>
                                                          > salted
                                                          password.  :/<br>
                                                          ><br>
                                                          > Thanks
                                                          for the help<br>
                                                          ><br>
                                                          > Sam<br>
                                                          ><br>
                                                          ><br>
                                                          ><br>
                                                          ><br>
                                                          > On Fri,
                                                          Mar 8, 2013 at
                                                          3:38 PM, Jay
                                                          Pipes <<a
                                                          moz-do-not-send="true"
href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>
                                                          wrote:<br>
                                                          ><br>
                                                          >     I'm
                                                          stumped :(
                                                          Looks like
                                                          everything is
                                                          set up
                                                          correctly to
                                                          me. What is<br>
                                                          >    
                                                          interested is
                                                          that your nova
                                                          user access
                                                          works from
                                                          test2, but
                                                          there is<br>
                                                          >     no
                                                          nova@test2
                                                          user in the
                                                          mysql.user
                                                          table. What
                                                          about doing a
                                                          DROP USER<br>
                                                          >    
                                                          nova@test1;
                                                          FLUSH
                                                          PRIVILEGES;
                                                          and then see
                                                          if that fixes
                                                          things...
                                                          since</p>
                                                          </div>
                                                          <p
                                                          class="MsoNormal">>
                                                              the <a
                                                          moz-do-not-send="true"
href="http://nova@10.21.0.0/255.255.0.0" target="_blank">
nova@10.21.0.0/255.255.0.0</a> <<a moz-do-not-send="true"
                                                          href="http://nova@10.21.0.0/255.255.0.0"
target="_blank">http://nova@10.21.0.0/255.255.0.0</a>></p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">>
                                                              user is
                                                          clearly
                                                          working for
                                                          the access<br>
                                                          >     from
                                                          test2.<br>
                                                          ><br>
                                                          >     Also,
                                                          I'd recommend
                                                          highly
                                                          removing the
                                                          nova@% user.<br>
                                                          ><br>
                                                          >     Best,<br>
                                                          >     -jay<br>
                                                          ><br>
                                                          >     On
                                                          03/08/2013
                                                          03:09 PM,
                                                          Samuel
                                                          Winchenbach
                                                          wrote:<br>
                                                          >     ><br>
                                                          >     >
                                                          <a
                                                          moz-do-not-send="true"
href="http://paste2.org/p/3085807" target="_blank">http://paste2.org/p/3085807</a><br>
                                                          >     ><br>
                                                          >     ><br>
                                                          >     >
                                                          On Fri, Mar 8,
                                                          2013 at 2:46
                                                          PM, Jay Pipes
                                                          <<a
                                                          moz-do-not-send="true"
href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>></p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">>
                                                              >
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>
                                                          wrote:<br>
                                                          >     ></p>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">>
                                                              >    
                                                          Please paste
                                                          the results of
                                                          SELECT User,
                                                          Host, Password
                                                          FROM<br>
                                                          >    
                                                          mysql.user<br>
                                                          >     >
                                                              when
                                                          running as
                                                          root...<br>
                                                          >     ><br>
                                                          >     >
                                                              Thanks!<br>
                                                          >     >
                                                              -jay<br>
                                                          >     ><br>
                                                          >     >
                                                              On
                                                          03/08/2013
                                                          02:25 PM,
                                                          Samuel
                                                          Winchenbach
                                                          wrote:<br>
                                                          >     >
                                                              > Here
                                                          are my grants.
                                                           I don't know
                                                          if this helps,
                                                          but I did<br>
                                                          >    
                                                          verify that<br>
                                                          >     >
                                                              > the
                                                          password was
                                                          identical for
                                                          each grant:<br>
                                                          >     >
                                                               <a
                                                          moz-do-not-send="true"
href="http://paste2.org/p/3085361" target="_blank">http://paste2.org/p/3085361</a><br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              > On
                                                          Fri, Mar 8,
                                                          2013 at 2:17
                                                          PM, Samuel
                                                          Winchenbach<br>
                                                          >     >
                                                              <<a
                                                          moz-do-not-send="true"
href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>>><br>
                                                          >     >
                                                              >
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>>>>>
                                                          wrote:<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >    
                                                          root@test1:/var/log#
                                                          mysql
                                                          -hmysql-ha
                                                          -unova<br>
                                                          >     >
                                                              >    
                                                          -p********************************
                                                          -e"SELECT
                                                          User, Host,<br>
                                                          >    
                                                          Password<br>
                                                          >     >
                                                              >    
                                                          FROM
                                                          mysql.user;"<br>
                                                          >     >
                                                              >    
                                                          ERROR 1142
                                                          (42000) at
                                                          line 1: SELECT
                                                          command denied
                                                          to user<br>
                                                          >     >
                                                              >    
                                                          'nova'@'test1'
                                                          for table
                                                          'user'<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >    
                                                          On Fri, Mar 8,
                                                          2013 at 2:06
                                                          PM, Jay Pipes<br>
                                                          >     <<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>></p>
                                                          </div>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">>
                                                              >    
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>>
                                                          wrote:<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >      
                                                            What does
                                                          this show?<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >      
                                                            mysql
                                                          -hmysql-ha
                                                          -unova
                                                          -p<PASS>
                                                          -e"SELECT
                                                          User, Host,<br>
                                                          >     >
                                                              Password
                                                          FROM<br>
                                                          >     >
                                                              >      
                                                            mysql.user"<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >      
                                                            -jay<br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              >      
                                                            On
                                                          03/08/2013
                                                          01:46 PM,
                                                          Samuel
                                                          Winchenbach
                                                          wrote:<br>
                                                          >     >
                                                              >      
                                                            > Sorry,
                                                          that must have
                                                          been a copy
                                                          and paste
                                                          error.<br>
                                                          >      Here<br>
                                                          >     >
                                                              is what I<br>
                                                          >     >
                                                              >      
                                                            >
                                                          actually ran:<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            > <a
                                                          moz-do-not-send="true"
href="http://paste2.org/p/3084996" target="_blank">http://paste2.org/p/3084996</a><br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            > On Fri,
                                                          Mar 8, 2013 at
                                                          12:40 PM, Jay
                                                          Pipes<br>
                                                          >     >
                                                              <<a
                                                          moz-do-not-send="true"
href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>><br>
                                                          >     >
                                                              >      
                                                            <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>><br>
                                                          >     >
                                                              >      
                                                            >
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>>>
                                                          wrote:<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >     On
                                                          03/08/2013
                                                          12:19 PM,
                                                          Samuel
                                                          Winchenbach
                                                          wrote:<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > Hi All,<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > I have
                                                          two nodes
                                                          (test1 and
                                                          test2) that I
                                                          am<br>
                                                          >    
                                                          trying to<br>
                                                          >     >
                                                              >      
                                                            set up in a<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > highly
                                                          available
                                                          configuration.<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > During
                                                          the setup
                                                          process I
                                                          tried running<br>
                                                          >    
                                                          "nova-manage<br>
                                                          >     >
                                                              >      
                                                            service
                                                          list" on<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > both
                                                          nodes.   It
                                                          worked fine on
                                                          test2, but<br>
                                                          >     fails
                                                          on<br>
                                                          >     >
                                                              >      
                                                            test1 even<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          though I<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > can
                                                          connect to the
                                                          database with
                                                          the mysql<br>
                                                          >    
                                                          client from<br>
                                                          >     >
                                                              >      
                                                            test1.<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > Here is a
                                                          screen capture
                                                          that shows the
                                                          setup on<br>
                                                          >     >
                                                              the two<br>
                                                          >     >
                                                              >      
                                                            nodes are<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          > basically
                                                          identical:<br>
                                                          >      <a
                                                          moz-do-not-send="true"
href="http://paste2.org/p/3084223" target="_blank">http://paste2.org/p/3084223</a><br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >     In
                                                          the above
                                                          paste you are
                                                          doing:<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          mysql -unova -
                                                               
                                                          hmysql-ha -u
                                                           root    
                                                           nova<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          -p********************************<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          Note you are
                                                          supplying 2 -u
                                                          arguments, and
                                                          mysql<br>
                                                          >     >
                                                              will take<br>
                                                          >     >
                                                              >      
                                                            the second<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          (root).<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          -jay<br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          _______________________________________________<br>
                                                          >     >
                                                              >      
                                                            >    
                                                          Mailing list:
                                                          <a
                                                          moz-do-not-send="true"
href="https://launchpad.net/%7Eopenstack" target="_blank">
https://launchpad.net/~openstack</a><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          Post to     :
                                                          <a
                                                          moz-do-not-send="true"
href="mailto:openstack@lists.launchpad.net" target="_blank">
openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>>><br>
                                                          >     >
                                                              >      
                                                            <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>>>><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>>><br>
                                                          >     >
                                                              >      
                                                            <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>><br>
                                                          >     >
                                                             
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                          >    
                                                          <mailto:<a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a>>>>><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          Unsubscribe :
                                                          <a
                                                          moz-do-not-send="true"
href="https://launchpad.net/%7Eopenstack" target="_blank">
https://launchpad.net/~openstack</a><br>
                                                          >     >
                                                              >      
                                                            >    
                                                          More help   :
                                                          <a
                                                          moz-do-not-send="true"
href="https://help.launchpad.net/ListHelp" target="_blank">
https://help.launchpad.net/ListHelp</a><br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              >      
                                                            ><br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              ><br>
                                                          >     >
                                                              ><br>
                                                          >     ><br>
                                                          >     ><br>
                                                          ><br>
                                                          ></p>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <p
                                                          class="MsoNormal"
style="margin-bottom:12.0pt"> </p>
                                                          <pre>_______________________________________________</pre>
                                                          <pre>Mailing list: <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a></pre>
                                                          <pre>Post to     : <a moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a></pre>
                                                          <pre>Unsubscribe : <a moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a></pre>
                                                          <pre>More help   : <a moz-do-not-send="true" href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a></pre>
                                                          </blockquote>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                        </div>
                                                      </div>
                                                      <p
                                                        class="MsoNormal"
style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
                                                        Mailing list: <a
moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack"
                                                          target="_blank">https://launchpad.net/~openstack</a><br>
                                                        Post to     : <a
moz-do-not-send="true" href="mailto:openstack@lists.launchpad.net"
                                                          target="_blank">openstack@lists.launchpad.net</a><br>
                                                        Unsubscribe : <a
moz-do-not-send="true" href="https://launchpad.net/%7Eopenstack"
                                                          target="_blank">https://launchpad.net/~openstack</a><br>
                                                        More help   : <a
moz-do-not-send="true" href="https://help.launchpad.net/ListHelp"
                                                          target="_blank">https://help.launchpad.net/ListHelp</a></p>
                                                    </div>
                                                    <p class="MsoNormal"> </p>
                                                  </div>
                                                </blockquote>
                                                <p class="MsoNormal"> </p>
                                              </blockquote>
                                              <p class="MsoNormal"> </p>
                                            </div>
                                          </div>
                                        </div>
                                      </div>
                                      <p class="MsoNormal"> </p>
                                    </div>
                                  </div>
                                </div>
                              </div>
                              <p class="MsoNormal"> </p>
                            </div>
                          </div>
                        </div>
                      </div>
                      <p class="MsoNormal"> </p>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <br>
  </body>
</html>