<div dir="ltr"><div>Hi,</div><div><br></div><div></div><div>I think my patchsets are in good shape and ready for merge, just need some reviewers finally :(.</div><div>I am glad that there is this type of question, who knows, maybe this will speed up the process of merge :) </div><div><br></div><div>You can still backport it to your downstream branches, we are using it in this way for several kolla-ansible versions and it works.</div><div><br></div><div>Thanks,</div><div>Michal</div><div><br></div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div style="margin:0px;padding:0px 0px 20px;width:1503px;color:rgb(34,34,34);font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium"><div style="margin:8px 0px 0px;padding:0px"><div dir="ltr"><span><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:small"><span style="color:rgb(136,136,136);vertical-align:top;line-height:18px;font-size:11pt;font-family:Arial,Helvetica,Geneva;font-weight:bold">Michal Arbet<br></span><span style="color:rgb(136,136,136);font-size:10pt;vertical-align:top;line-height:18px;font-family:Arial,Helvetica,Geneva">Openstack Engineer<br><img src="https://www.google.com/a/ultimum.io/images/logo.gif" width="200" height="81"><br>Ultimum Technologies a.s.<br>Na Poříčí 1047/26, 11000 Praha 1<br>Czech Republic<br><br><a value="+420608314961" style="color:rgb(17,85,204)">+420 604 228 897</a> <br><a href="mailto:michal.arbet@ultimum.io" style="color:rgb(17,85,204)" target="_blank">michal.arbet@ultimum.io</a><br></span><font color="#1155cc" face="Arial, Helvetica, Geneva" style="font-size:12.8px"><span style="font-size:13.3333px;line-height:18px"><u><a href="https://ultimum.io/" style="color:rgb(17,85,204)" target="_blank">https://ultimum.io</a></u></span></font><br></div><div dir="ltr" style="font-family:Arial,Helvetica,sans-serif;font-size:small"><font color="#1155cc" face="Arial, Helvetica, Geneva" style="font-size:12.8px"><span style="font-size:13.3333px;line-height:18px"><br></span></font></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small"><font color="#666666"><a href="https://www.linkedin.com/company/ultimum-technologies" style="color:rgb(17,85,204)" target="_blank">LinkedIn</a> | <a href="https://twitter.com/ultimumtech" style="color:rgb(17,85,204)" target="_blank">Twitter</a> | <a href="https://www.facebook.com/ultimumtechnologies/timeline" style="color:rgb(17,85,204)" target="_blank">Facebook</a></font></div></div></div></div></div></div></div></div></span></div></div><div><div></div><div><div></div></div></div><div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">st 8. 6. 2022 v 16:37 odesílatel Norrie, Andrew <<a href="mailto:Andrew.Norrie@cgg.com">Andrew.Norrie@cgg.com</a>> napsal:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_4507977372183256931WordSection1">
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Hi Mark,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Thanks for the reply.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">I’ll look into the HAProxy “custom config” as suggested and using different ports for the cell databases.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Maybe that can be worked in.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">We are expanding our OpenStack/kolla-ansible personnel/knowledge so will look into the possibility for<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">us to contribute as reviewers/testers as a start.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Best regards .... Andrew<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Mark Goddard <<a href="mailto:mark@stackhpc.com" target="_blank">mark@stackhpc.com</a>> <br>
<b>Sent:</b> Monday, June 6, 2022 2:22 AM<br>
<b>To:</b> Norrie, Andrew <<a href="mailto:Andrew.Norrie@cgg.com" target="_blank">Andrew.Norrie@cgg.com</a>><br>
<b>Cc:</b> <a href="mailto:openstack-discuss@lists.openstack.org" target="_blank">openstack-discuss@lists.openstack.org</a><br>
<b>Subject:</b> Re: [kolla-ansible][xena] Cell database HA/redundancy?<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Fri, 3 Jun 2022 at 14:10, Norrie, Andrew <<a href="mailto:Andrew.Norrie@cgg.com" target="_blank">Andrew.Norrie@cgg.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Hi,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">We are currently planning some large OpenStack deployments utilizing kolla-ansible and</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">I’m curious what folks are doing with the cell database HA/redundancy.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">With kolla-ansible (xena) it appears that a loadbalancer setup is only allowed for the</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">default database shard (shard 0)... reference:
<a href="https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.openstack.org%2Fkolla-ansible%2Flatest%2Freference%2Fdatabases%2Fmariadb-guide.html&data=05%7C01%7CAndrew.Norrie%40cgg.com%7Cf493c7380a8342b0eb9408da4795b553%7C307ea68275e14701a1146c42f9ff0d24%7C0%7C0%7C637901005570096176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=QvBz9RObPfwW3PsIeM1whZjaT2LOwrj5Iylgh2wfeTo%3D&reserved=0" target="_blank">
https://docs.openstack.org/kolla-ansible/latest/reference/databases/mariadb-guide.html</a></span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">If we are setting up separate cell database shards with Galera, I’m wondering if there is a convenient work-around or
configuration for implementation of HA for these cell databases.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">In the inventory group_vars directory you can specify the group variables (for each cell database) as:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">nova_cell_database_address:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">nova_cell_database_group:</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">but these aren’t virtual IP hosts/addresses (non default db shard). This works perfectly fine with</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">a single server cell database but not Galera. If that db server goes down the cell instance information is lost.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)"> </span><u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">Hi Andrew,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">You are correct - only the first shard is load balanced. The sharding feature was actually just the first patch in a series intended to support proxysql. I believe this would add the functionality
you are looking for. In fact, the proposed test case for proxysql in our CI is a multi-cell deployment.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">Here is the patch series:<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><a href="https://fra01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.opendev.org%2Fq%2Ftopic%3Aproxysql&data=05%7C01%7CAndrew.Norrie%40cgg.com%7Cf493c7380a8342b0eb9408da4795b553%7C307ea68275e14701a1146c42f9ff0d24%7C0%7C0%7C637901005570096176%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ooeBJE5Wsxuz1yNR%2F94Nk4ZK7wEx8nv4Jztl6Tk%2BbAg%3D&reserved=0" target="_blank">https://review.opendev.org/q/topic:proxysql</a><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">Unfortunately it has been stuck for a while, largely due to core reviewer bandwidth. The author is Michael Arbet, who is often on IRC as kevko. I'd suggest registering your interest in these
patches via gerrit, and at one of the weekly kolla IRC meetings (this week is cancelled due to the summit). If you have time available, then testing and providing reviews could help to get the patches moving.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">In the meantime, you can configure HAProxy to load balance additional services, by placing an HAProxy config snippet in /etc/kolla/config/haproxy/services.d/*.cfg.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">Regards,<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif">Mark<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif"><u></u> <u></u></span></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><span style="font-family:Verdana,sans-serif;color:rgb(49,132,155)">Many thanks ... Andrew</span><u></u><u></u></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal">“This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly
prohibited. If you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately.”
<u></u><u></u></p>
</div>
</blockquote>
</div>
</div>
</div>
<hr>
“This e-mail and any accompanying attachments are confidential. The information is intended solely for the use of the individual to whom it is addressed. Any review, disclosure, copying, distribution, or use of the email by others is strictly prohibited. If
you are not the intended recipient, you must not review, disclose, copy, distribute or use this e-mail; please delete it from your system and notify the sender immediately.”
</div>
</blockquote></div>