<div dir="ltr">I<div style="font-family:'courier new',monospace;display:inline" class="gmail_default"> enabled general_log in /etc/mysql/my.cnf Here are the results of connecting from "test1", "test2" and using the client:</div>
<div><div style="display:inline" class="gmail_default"><font face="courier new, monospace"><a href="http://paste2.org/p/3115525">http://paste2.org/p/3115525</a></font></div></div><div><span style="font-family:'courier new',monospace">I purposefully used the real password in case there is a problem with it.<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
I changed before submitting post.</div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"><br></div>
</span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
here is a raw packet TCP dump (tcpdump -w rawdump port 3306) of an attempted "nova-manage service list" from test1: </div></span></div><div><div style="display:inline" class="gmail_default"><font face="courier new, monospace"><a href="https://www.dropbox.com/s/u4cjzxv6w6bwwe6/rawdump">https://www.dropbox.com/s/u4cjzxv6w6bwwe6/rawdump</a></font></div>
<span style="font-family:'courier new',monospace"> </span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
I looked at it with wireshark and couldn't see anything that jumped out at me as incorrect. I have not yet tried to recreate the salted password.</div></span><br></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
<br></div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">Here is my pacemaker configuration for mysql. I stripped out openstack services, rabbitmq and others for clarity. All resources are currently disabled (other than MySQL): </div>
</span><span style="font-family:'courier new',monospace"><a href="http://paste2.org/p/3115685">http://paste2.org/p/3115685</a><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"><br></div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
Please don't yell at me for having STONITH disabled :P This is a testing cluster and I am working on getting routed to the IPMI interface.</div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
<br></div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">/etc/hosts: </div></span><font face="courier new, monospace"><a href="http://paste2.org/p/3115713">http://paste2.org/p/3115713</a><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font></div><div><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">/etc/nova/nova.conf: </div><a href="http://paste2.org/p/3115739">http://paste2.org/p/3115739</a><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font></div><div><span style="font-family:'courier new',monospace"><br></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
If there is anything else I can provide you, please let me know! I have pulled out most of my hair at this point!</div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
<br></div></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">Sam</div></span><br></div><div><font face="courier new, monospace"><br>
</font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Mar 11, 2013 at 10:11 AM, Sylvain Bauza <span dir="ltr"><<a href="mailto:sylvain.bauza@digimind.com" target="_blank">sylvain.bauza@digimind.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<div>So as to reproduce the nova-manage SQL
command, I would recommand to tcpdump -A port 3306 on the host and
get the SQL trace on what's failing.<br>
<br>
Could you please explain further what is your HA config ? Are you
using pacemaker/heartbeat or any VIP ?<br>
<br>
-Sylvain<br>
<br>
Le 11/03/2013 14:23, Samuel Winchenbach a écrit :<br>
</div><div><div class="h5">
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_default">Does anyone think this could be an openstack
bug? I just want to check before submitting a bug report.</div>
<div class="gmail_default">
<br>
</div>
<div class="gmail_default">Sam</div>
</div>
<div class="gmail_extra"><br>
<br>
<div class="gmail_quote">On Fri, Mar 8, 2013 at 4:02 PM, Jay
Pipes <span dir="ltr"><<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Sorry, I
really can't think of anything :(<br>
<div><br>
On 03/08/2013 03:52 PM, Samuel Winchenbach wrote:<br>
> I dropped those users and no change.<br>
><br>
> I also set up general logging in mysql but it really
doesn't provide any<br>
> additional information. Any idea for a next step I
could take?<br>
><br>
> I am almost at the point of taking a tcpdump and
trying to recreate the<br>
> salted password. :/<br>
><br>
> Thanks for the help<br>
><br>
> Sam<br>
><br>
><br>
><br>
><br>
> On Fri, Mar 8, 2013 at 3:38 PM, Jay Pipes <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
</div>
<div>> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>
wrote:<br>
><br>
> I'm stumped :( Looks like everything is set up
correctly to me. What is<br>
> interested is that your nova user access works
from test2, but there is<br>
> no nova@test2 user in the mysql.user table. What
about doing a DROP USER<br>
> nova@test1; FLUSH PRIVILEGES; and then see if
that fixes things... since<br>
</div>
> the <a href="http://nova@10.21.0.0/255.255.0.0" target="_blank">nova@10.21.0.0/255.255.0.0</a>
<<a href="http://nova@10.21.0.0/255.255.0.0" target="_blank">http://nova@10.21.0.0/255.255.0.0</a>><br>
<div>> user is clearly working for
the access<br>
> from test2.<br>
><br>
> Also, I'd recommend highly removing the nova@%
user.<br>
><br>
> Best,<br>
> -jay<br>
><br>
> On 03/08/2013 03:09 PM, Samuel Winchenbach wrote:<br>
> ><br>
> > <a href="http://paste2.org/p/3085807" target="_blank">http://paste2.org/p/3085807</a><br>
> ><br>
> ><br>
> > On Fri, Mar 8, 2013 at 2:46 PM, Jay Pipes
<<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
</div>
<div>> > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>
wrote:<br>
> ><br>
</div>
<div>
<div>> > Please paste the results
of SELECT User, Host, Password FROM<br>
> mysql.user<br>
> > when running as root...<br>
> ><br>
> > Thanks!<br>
> > -jay<br>
> ><br>
> > On 03/08/2013 02:25 PM, Samuel
Winchenbach wrote:<br>
> > > Here are my grants. I don't know
if this helps, but I did<br>
> verify that<br>
> > > the password was identical for
each grant:<br>
> > <a href="http://paste2.org/p/3085361" target="_blank">http://paste2.org/p/3085361</a><br>
> > ><br>
> > ><br>
> > > On Fri, Mar 8, 2013 at 2:17 PM,
Samuel Winchenbach<br>
> > <<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
<mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>><br>
> <mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
<mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>>><br>
> > > <mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
<mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>><br>
> <mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>
<mailto:<a href="mailto:swinchen@gmail.com" target="_blank">swinchen@gmail.com</a>>>>>
wrote:<br>
> > ><br>
> > > root@test1:/var/log# mysql
-hmysql-ha -unova<br>
> > >
-p******************************** -e"SELECT User, Host,<br>
> Password<br>
> > > FROM mysql.user;"<br>
> > > ERROR 1142 (42000) at line 1:
SELECT command denied to user<br>
> > > 'nova'@'test1' for table
'user'<br>
> > ><br>
> > ><br>
> > > On Fri, Mar 8, 2013 at 2:06
PM, Jay Pipes<br>
> <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
> > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>><br>
</div>
</div>
<div>
<div>> > > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>>
wrote:<br>
> > ><br>
> > > What does this show?<br>
> > ><br>
> > > mysql -hmysql-ha -unova
-p<PASS> -e"SELECT User, Host,<br>
> > Password FROM<br>
> > > mysql.user"<br>
> > ><br>
> > > -jay<br>
> > ><br>
> > > On 03/08/2013 01:46 PM,
Samuel Winchenbach wrote:<br>
> > > > Sorry, that must
have been a copy and paste error.<br>
> Here<br>
> > is what I<br>
> > > > actually ran:<br>
> > > ><br>
> > > > <a href="http://paste2.org/p/3084996" target="_blank">http://paste2.org/p/3084996</a><br>
> > > ><br>
> > > ><br>
> > > > On Fri, Mar 8, 2013
at 12:40 PM, Jay Pipes<br>
> > <<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>><br>
> > > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>><br>
> > > > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>><br>
> > <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>><br>
> <mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>
<mailto:<a href="mailto:jaypipes@gmail.com" target="_blank">jaypipes@gmail.com</a>>>>>>
wrote:<br>
> > > ><br>
> > > > On 03/08/2013
12:19 PM, Samuel Winchenbach wrote:<br>
> > > > > Hi All,<br>
> > > > ><br>
> > > > > I have two
nodes (test1 and test2) that I am<br>
> trying to<br>
> > > set up in a<br>
> > > > > highly
available configuration.<br>
> > > > ><br>
> > > > > During the
setup process I tried running<br>
> "nova-manage<br>
> > > service list" on<br>
> > > > > both nodes.
It worked fine on test2, but<br>
> fails on<br>
> > > test1 even<br>
> > > > though I<br>
> > > > > can connect
to the database with the mysql<br>
> client from<br>
> > > test1.<br>
> > > > ><br>
> > > > > Here is a
screen capture that shows the setup on<br>
> > the two<br>
> > > nodes are<br>
> > > > > basically
identical:<br>
> <a href="http://paste2.org/p/3084223" target="_blank">http://paste2.org/p/3084223</a><br>
> > > ><br>
> > > > In the above
paste you are doing:<br>
> > > ><br>
> > > > mysql -unova -
hmysql-ha -u root nova<br>
> > > >
-p********************************<br>
> > > ><br>
> > > > Note you are
supplying 2 -u arguments, and mysql<br>
> > will take<br>
> > > the second<br>
> > > > (root).<br>
> > > ><br>
> > > > -jay<br>
> > > ><br>
> > > >
_______________________________________________<br>
> > > > Mailing list: <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
> > > > Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
> > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>>><br>
> > > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
> > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>>>><br>
> > > > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
> > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>>><br>
> > > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>><br>
> > <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a><br>
> <mailto:<a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>>>>><br>
> > > > Unsubscribe : <a href="https://launchpad.net/%7Eopenstack" target="_blank">https://launchpad.net/~openstack</a><br>
> > > > More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
> > > ><br>
> > > ><br>
> > ><br>
> > ><br>
> > ><br>
> ><br>
> ><br>
><br>
><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
<br>
<fieldset></fieldset>
<br>
<pre>_______________________________________________
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a>
Post to : <a href="mailto:openstack@lists.launchpad.net" target="_blank">openstack@lists.launchpad.net</a>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a>
</pre>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack" target="_blank">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp" target="_blank">https://help.launchpad.net/ListHelp</a><br>
<br></blockquote></div><br></div>