<div dir="ltr"><font face="courier new, monospace">Sure... I have undone these settings but I saved a copy:<br><br>two hosts:<br>test1 eth0: <a href="http://10.21.0.1/16">10.21.0.1/16</a> eth1: 130.x.x.x/24<br>test2 eth0: <a href="http://10.21.0.2/16">10.21.0.2/16</a> eth1: 130.x.x.x/24<br>
<br>VIP: 10.21.21.1 (just for testing, later I would add a 130.x.x.x/24 VIP for public APIs<br><br>k<div style="display:inline" class="gmail_default">eystone is bound to 10.21.0.1 on test1 and 10.21.0.2 on test2</div><br>
<br><br>in /etc/sysctl.conf:<br> net.ipv4.conf.all.arp_ignore = 1<br> net.ipv4.conf.eth0.arp_ignore = 1<br> net.ipv4.conf.all.arp_announce = 2<br> net.ipv4.conf.eth0.arp_announce = 2<br><br>root# sysctl -p<br><br>
in /etc/sysctl.conf:<br><div style="display:inline" class="gmail_default"> </div>checktimeout=<div style="display:inline" class="gmail_default">3</div><br><div style="display:inline" class="gmail_default"> </div>
checkinterval=<div style="display:inline" class="gmail_default">5</div><br><div style="display:inline" class="gmail_default"> </div>autoreload=<div style="display:inline" class="gmail_default">yes</div><br><div style="display:inline" class="gmail_default">
</div>logfile="/var/log/ldirectord.log"<br><div style="display:inline" class="gmail_default"> </div>quiescent=no<br><div style="display:inline" class="gmail_default"> </div>virtual=<a href="http://10.21.21.1:5000">10.21.21.1:5000</a></font><div>
<font face="courier new, monospace"><div style="display:inline" class="gmail_default"> </div>real=10.2<div style="display:inline" class="gmail_default">1</div>.0.1:5000 gate</font></div><div><font face="courier new, monospace"><div style="display:inline" class="gmail_default">
</div>real=10.2<div style="display:inline" class="gmail_default">1</div>.0.2:5000 gate</font></div><div><font face="courier new, monospace"> <div style="display:inline" class="gmail_default"></div>scheduler=<div style="display:inline" class="gmail_default">
w</div>rr</font></div><div><font face="courier new, monospace"><div style="display:inline" class="gmail_default"></div> protocol=tcp</font></div><div><font face="courier new, monospace"><div style="display:inline" class="gmail_default">
</div> checktype=connect</font></div><div><div class="gmail_default"><font face="courier new, monospace"> checkport=5000</font></div><font face="courier new, monospace"><div class="gmail_default" style="display:inline">
</div>virtual=<a href="http://10.21.21.1">10.21.21.1</a>:<div style="display:inline" class="gmail_default">35357</div></font><div><font face="courier new, monospace"><div class="gmail_default" style="display:inline">
</div>real=10.2<div style="display:inline" class="gmail_default">1</div>.0.1:<div style="display:inline" class="gmail_default">35357</div> gate</font></div><div><font face="courier new, monospace"><div class="gmail_default" style="display:inline">
</div>real=10.2<div style="display:inline" class="gmail_default">1</div>.0.2:<div style="display:inline" class="gmail_default">35357</div> gate</font></div><div><font face="courier new, monospace"> <div class="gmail_default" style="display:inline">
</div>scheduler=<div class="gmail_default" style="display:inline">w</div>rr</font></div><div><font face="courier new, monospace"><div class="gmail_default" style="display:inline"></div> protocol=tcp</font></div><div>
<font face="courier new, monospace"><div class="gmail_default" style="display:inline"></div> checktype=connect</font></div><div class="gmail_default"><font face="courier new, monospace"> checkport=35357</font></div>
<div class="gmail_default"><font face="courier new, monospace"><br></font></div><div class="gmail_default"><font face="courier new, monospace"><br></font></div><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
crm shell:</div></font></div><div><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"><br></div></font></div><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>primitive <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
p_openstack_</div>ip ocf:heartbeat:IPaddr2 \<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></font><span style="font-family:'courier new',monospace"> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>op monitor interval="60" timeout="20" \<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>params ip=<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
"10.21.21.1</div>" <div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></font><div class="gmail_default" style="font-family:'courier new',monospace;display:inline">
</div><span style="font-family:'courier new',monospace">cidr_netmask="<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">16</div>"<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>lvs_support="true"<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">p</div>rimitive <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
p_openstack_ip_lo</div> ocf:heartbeat:IPaddr2 \<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></font><span style="font-family:'courier new',monospace"> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>op monitor interval="60" timeout="20" \<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>params ip="<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
10.21.21.1</div>" nic="lo" <div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>cidr_netmask="32"<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>primitive <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
p_openstack_</div>lvs ocf:heartbeat:ldirectord \<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div></font><span style="font-family:'courier new',monospace"> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><span style="font-family:'courier new',monospace"> </span><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>op monitor interval="20" timeout="10"<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>group <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
g_openstack_</div>ip<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">_</div>lvs <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
p_openstack_</div>ip <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">p_openstack_</div>lvs<br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>clone <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
c_openstack_ip_lo</div> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">p_openstack_ip_lo</div> meta interleave="true" <br><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
</div></font><span style="font-family:'courier new',monospace"> </span><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default"></div>colocation<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
co_openstack_lo_never_lvs</div> -inf: c<div style="font-family:'courier new',monospace;display:inline" class="gmail_default">_openstack_ip_lo</div> <div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
g_openstack_ip_lvs</div></font><div><font face="courier new, monospace"><br></font></div><div><font face="courier new, monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
Thanks for taking a look at this. </div></font></div><div><span style="font-family:'courier new',monospace"><br></span></div><div><span style="font-family:'courier new',monospace"><div style="font-family:'courier new',monospace;display:inline" class="gmail_default">
Sam</div></span></div><div><div><font face="courier new, monospace"><br><br><br><br>On Fri, Feb 15, 2013 at 3:54 AM, Sébastien Han <<a href="mailto:han.sebastien@gmail.com">han.sebastien@gmail.com</a>> wrote:<br>
><br>> Hum I don't see the problem, it's possible to load-balance VIPs with LVS, there are just IPs... Can I see your conf?<br>><br>> --<br>> Regards,<br>> Sébastien Han.<br>><br>><br>> On Thu, Feb 14, 2013 at 8:34 PM, Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>> wrote:<br>
>><br>>> W<br>>> ell, I think I will have to go with one ip per service and forget about load balancing. It seems as though with LVS routing requests internally through the VIP is difficult (impossible?) at least with LVS-DR. It seems like a shame not to be able to distribute the work among the controller nodes.<br>
>><br>>><br>>> On Thu, Feb 14, 2013 at 9:50 AM, Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>> wrote:<br>>>><br>>>> Hi Sébastien,<br>>>><br>
>>> I have two hosts with public interfaces with a number (~8) compute nodes behind them. I am trying to set the two public nodes in for HA and load balancing, I plan to run all the openstack services on these two nodes in Active/Active where possible. I currently have MySQL and RabbitMQ setup in pacemaker with a drbd backend.<br>
>>><br>>>> That is a quick summary. If there is anything else I can answer about my setup please let me know.<br>>>><br>>>> Thanks,<br>>>> Sam<br>>>><br>>>><br>
>>> On Thu, Feb 14, 2013 at 9:26 AM, Sébastien Han <<a href="mailto:han.sebastien@gmail.com">han.sebastien@gmail.com</a>> wrote:<br>>>>><br>>>>> Well I don't know your setup, if you use LB for API service or if you use an active/passive pacemaker but at the end it's not that much IPs I guess. I dare to say that Keepalived sounds outdated to me...<br>
>>>><br>>>>> If you use pacemaker and want to have the same IP for all the resources simply create a resource group with all the openstack service inside it (it's ugly but if it's what you want :)). Give me more info about your setup and we can go further in the discussion :).<br>
>>>><br>>>>> --<br>>>>> Regards,<br>>>>> Sébastien Han.<br>>>>><br>>>>><br>>>>> On Thu, Feb 14, 2013 at 3:15 PM, Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>> wrote:<br>
>>>>><br>>>>>> T<br>>>>>> he only real problem is that it would consume a lot of IP addresses when exposing the public interfaces. I _think_ I may have the solution in your blog actually: <a href="http://www.sebastien-han.fr/blog/2012/10/19/highly-available-lvs/">http://www.sebastien-han.fr/blog/2012/10/19/highly-available-lvs/</a><br>
>>>>> and<br>>>>>> <a href="http://clusterlabs.org/wiki/Using_ldirectord">http://clusterlabs.org/wiki/Using_ldirectord</a><br>>>>>><br>>>>>> I am trying to weigh the pros and cons of this method vs keepalived/haproxy and just biting the bullet and using one IP per service.<br>
>>>>><br>>>>>><br>>>>>> On Thu, Feb 14, 2013 at 4:17 AM, Sébastien Han <<a href="mailto:han.sebastien@gmail.com">han.sebastien@gmail.com</a>> wrote:<br>>>>>>><br>
>>>>>> What's the problem to have one IP on service pool basis?<br>>>>>>><br>>>>>>> --<br>>>>>>> Regards,<br>>>>>>> Sébastien Han.<br>
>>>>>><br>>>>>>><br>>>>>>> On Wed, Feb 13, 2013 at 8:45 PM, Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>> wrote:<br>>>>>>>><br>
>>>>>>> What if the VIP is created on a different host than keystone is started on? It seems like you either need to set net.ipv4.ip_nonlocal_bind = 1 or create a colocation in pacemaker (which would either require all services to be on the same host, or have an ip-per-service). <br>
>>>>>>><br>>>>>>>><br>>>>>>>><br>>>>>>>><br>>>>>>>> On Wed, Feb 13, 2013 at 2:28 PM, Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>> wrote:<br>
>>>>>>>><br>>>>>>>>> There we go<br>>>>>>>>> <a href="https://review.openstack.org/#/c/21581/">https://review.openstack.org/#/c/21581/</a><br>>>>>>>>><br>
>>>>>>>> Razique Mahroua - Nuage & Co<br>>>>>>>>> <a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a><br>>>>>>>>> Tel : +33 9 72 37 94 15<br>
>>>>>>>><br>>>>>>>>><br>>>>>>>>> Le 13 févr. 2013 à 20:15, Razique Mahroua <<a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a>> a écrit :<br>
>>>>>>>><br>>>>>>>>> I'm currently updating that part of the documentation - indeed it states that two IPs are used, but in fact, you end up with only one VIP for the API service.<br>
>>>>>>>> I'll send the patch tonight<br>>>>>>>>><br>>>>>>>>> Razique Mahroua - Nuage & Co<br>>>>>>>>> <a href="mailto:razique.mahroua@gmail.com">razique.mahroua@gmail.com</a><br>
>>>>>>>> Tel : +33 9 72 37 94 15<br>>>>>>>>><br>>>>>>>>> <NUAGECO-LOGO-Fblan_petit.jpg><br>>>>>>>>><br>>>>>>>>> Le 13 févr. 2013 à 20:05, Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>> a écrit :<br>
>>>>>>>><br>>>>>>>>> In that documentation it looks like each openstack service gets it own IP (keystone is being assigned 192.168.42.103 and glance is getting 192.168.42.104). <br>
>>>>>>>><br>>>>>>>>> I might be missing something too because in the section titled "Configure the VIP" it create a primitive called "p_api-ip" (or p_ip_api if you read the text above it) and then in "Adding Keystone resource to Pacemaker" it creates a group with "p_ip_keystone"???<br>
>>>>>>>><br>>>>>>>>><br>>>>>>>>> Stranger yet, "Configuring OpenStack Services to use High Available Glance API" says: "For Nova, for example, if your Glance API service IP address is 192.168.42.104 as in the configuration explained here, you would use the following line in your nova.conf file : glance_api_servers = 192.168.42.103" But, in the step before it set: "registry_host = 192.168.42.104"?<br>
>>>>>>>><br>>>>>>>>> So I am not sure which ip you would connect to here... <br>>>>>>>>><br>>>>>>>>> Sam<br>>>>>>>>><br>
>>>>>>>><br>>>>>>>>><br>>>>>>>>> On Wed, Feb 13, 2013 at 1:29 PM, JuanFra Rodriguez Cardoso <<a href="mailto:juanfra.rodriguez.cardoso@gmail.com">juanfra.rodriguez.cardoso@gmail.com</a>> wrote:<br>
>>>>>>>>><br>>>>>>>>>> Hi Samuel:<br>>>>>>>>>><br>>>>>>>>>> Yes, it's possible with pacemaker. Look at <a href="http://docs.openstack.org/trunk/openstack-ha/content/ch-intro.html">http://docs.openstack.org/trunk/openstack-ha/content/ch-intro.html</a>.<br>
>>>>>>>>><br>>>>>>>>>> Regards,<br>>>>>>>>>> JuanFra<br>>>>>>>>>><br>>>>>>>>>><br>>>>>>>>>> 2013/2/13 Samuel Winchenbach <<a href="mailto:swinchen@gmail.com">swinchen@gmail.com</a>><br>
>>>>>>>>>><br>>>>>>>>>>> Hi All,<br>>>>>>>>>>><br>>>>>>>>>>> I currently have a HA OpenStack cluster running where the OpenStack services are kept alive with a combination of haproxy and keepalived.<br>
>>>>>>>>>><br>>>>>>>>>>> Is it possible to configure pacemaker so that all the OpenStack services are served by the same IP? With keepalived I have a virtual ip that can move from server to server and haproxy sends the request to a machine that has a "live" service. This allows one (public) ip to handle all incoming requests. I believe it is the combination of VRRP/IPVS that allows this.<br>
>>>>>>>>>><br>>>>>>>>>>><br>>>>>>>>>>> Is it possible to do something similar with pacemaker? I really don't want to have an IP for each service, and I don't want to make it a requirement that all OpenStack services must be running on the same server.<br>
>>>>>>>>>><br>>>>>>>>>>> Thanks... I hope this question is clear, I feel like I sort of butchered the wording a bit.<br>>>>>>>>>>><br>
>>>>>>>>>> Sam<br>>>>>>>>>>><br>>>>>>>>>>> _______________________________________________<br>>>>>>>>>>> Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>>>>> Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>>>>>>>>>>> Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>>>>> More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>>>>>>>>>>><br>>>>>>>>>><br>
>>>>>>>><br>>>>>>>>> _______________________________________________<br>>>>>>>>> Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>>> Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>>>>>>>>> Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>>> More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>>>>>>>>><br>>>>>>>>><br>>>>>>>>><br>
>>>>>>><br>>>>>>>><br>>>>>>>> _______________________________________________<br>>>>>>>> Mailing list: <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>> Post to : <a href="mailto:openstack@lists.launchpad.net">openstack@lists.launchpad.net</a><br>>>>>>>> Unsubscribe : <a href="https://launchpad.net/~openstack">https://launchpad.net/~openstack</a><br>
>>>>>>> More help : <a href="https://help.launchpad.net/ListHelp">https://help.launchpad.net/ListHelp</a><br>>>>>>>><br>>>>>>><br>>>>>><br>>>>><br>
>>><br>>><br>></font><br></div></div></div>