<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body bgcolor="#FFFFFF">
<div>Ed and i are on the same team and we are also working on a reaper, using the databases instance table state and notifications. If enough of the community is behind the idea we can push it upstream. Knowing what to reap can be tricky tho. </div>
<div><br>
Sent from my iPhone</div>
<div><br>
On Aug 18, 2011, at 8:11 PM, "Joshua Harlow" <<a href="mailto:harlowja@yahoo-inc.com">harlowja@yahoo-inc.com</a>> wrote:<br>
<br>
</div>
<div></div>
<blockquote type="cite">
<div><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">Thanks,<br>
<br>
It was along the lines of what I was thinking.<br>
<br>
If messages are made persistent, which I hope is planned, or made a configuration option what would be the effects of them not being made persistent.<br>
<br>
Right now if a message is lost, it seems the DB/other nodes are left in a bad state, is there any plan to have a “reaper” python object that will reap this bad data/instances....<br>
<br>
On 8/18/11 4:54 PM, "Edward "koko" Konetzko" <<a href="konetzed@quixoticagony.com"></a><a href="mailto:konetzed@quixoticagony.com">konetzed@quixoticagony.com</a>> wrote:<br>
<br>
</span></font>
<blockquote><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size:11pt">On 08/16/2011 04:50 PM, Joshua Harlow wrote:<br>
> Are there any good documentations on making openstack fault tolerant or<br>
> exactly how it will handle failures?<br>
><br>
> Like say the mq server dies, can another mq server take over. Similar<br>
> with the database (mysql replication?)....<br>
><br>
> Seems like having that kind of information for corporate users would be<br>
> nice, at least a recommended “guide”.<br>
><br>
> -Josh<br>
><br>
><br>
><br>
> _______________________________________________<br>
> Mailing list: <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
> Post to : <a href="openstack@lists.launchpad.net"></a><a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
> Unsubscribe : <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
> More help : <a href="https://help.launchpad.net/ListHelp"></a><a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>
<br>
Josh<br>
<br>
I have a very bare bones start of a doc on making parts of Nova HA. The<br>
problem is this document is no where near ready for release as I am<br>
probably the only person who can understand it. I will try to point you<br>
in the right direction on things I have done that work pretty well.<br>
<br>
Rabbitmq<br>
<a href="http://www.rabbitmq.com/pacemaker.html"></a><a href="http://www.rabbitmq.com/pacemaker.html">http://www.rabbitmq.com/pacemaker.html</a><br>
<br>
Right now in the version of Nova the team I am working with nothing is<br>
marked 'persistent'. Right now in this use case if a node fails rabbitmq<br>
moves over and all the managers reconnect with no issues but all in<br>
flight messages are lost. Maybe someone here can clarify on the<br>
direction of this. I we are using Ubuntu 10.04 and the version of<br>
Rabbitmq in that release does not have the pacemaker scripts, I just<br>
pulled the current package from <a href="http://rabbitmq.com">rabbitmq.com</a> apt repo after that the<br>
pacemaker setup worked perfect.<br>
<br>
MySQL<br>
For MySQL I just did a simple setup using DRDB to replicate<br>
/var/lib/mysql and setup corosync/pacemaker to manage all the MySQL<br>
resources between two nodes. Again with this situation in failover I<br>
had no issues with clients reconnecting to the vip.<br>
<br>
I hope this points you in the right direction, I know its not exactly<br>
what you wanted. Maybe next week I can clean up my documentation and<br>
send it out to the list.<br>
<br>
Edward Konetzko<br>
<br>
_______________________________________________<br>
Mailing list: <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
Post to : <a href="openstack@lists.launchpad.net"></a><a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>
Unsubscribe : <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
More help : <a href="https://help.launchpad.net/ListHelp"></a><a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>
<br>
</span></font></blockquote>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>Mailing list: <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a></span><br>
<span>Post to : <a href="mailto:openstack@lists.launchpad.net"></a><a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a></span><br>
<span>Unsubscribe : <a href="https://launchpad.net/~openstack"></a><a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a></span><br>
<span>More help : <a href="https://help.launchpad.net/ListHelp"></a><a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a></span><br>
</div>
</blockquote>
<font face="monospace">This email may include confidential information. If you received it in error, please delete it.</font></body>
</html>