<div dir="ltr">K, here's the follow-up.<br><div>I suppose I was wrong saying that "<span style="font-family:arial,sans-serif;font-size:13px">InvalidBDM: Block Device Mapping is Invalid" problem with multiple cinder-scheduler instances is caused by deadlocks. Here's a thread regarding this problem, still without an appropriate answer: <br>
</span><font face="arial, sans-serif"><a href="http://www.gossamer-threads.com/lists/openstack/dev/24998">http://www.gossamer-threads.com/lists/openstack/dev/24998</a><br></font><span style="font-family:arial,sans-serif">The thread is about Folsom. Is it even possible that no one figured out a solution since then?</span><font face="arial, sans-serif"><br>
</font></div><div><font face="arial, sans-serif"><br>BTW, I managed to make multiple nova-conductor instances running by binding MySQL to "management" interfaces on controller nodes and then pointing nova.conf to 127.0.0.1 where haproxy is listening on each node, forwarding queries to these management IP's; but the second controller IP is configured as backup, so all requests from both controllers go to the first one while it is alive, if it fails haproxy switches to the second controller. </font></div>
<div><font face="arial, sans-serif">This is quick&dirty way though, with 0 scalability, so the question is still open.</font><span style="font-family:arial,sans-serif"> </span></div><div><font face="arial, sans-serif"><br>
</font></div><div><font face="arial, sans-serif"><br></font></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-05-15 8:07 GMT+03:00 Chu Duc Minh <span dir="ltr"><<a href="mailto:chu.ducminh@gmail.com" target="_blank">chu.ducminh@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>I hava the same setup & got the same problem like Sergey. <br></div>Not yet figured out a good solution for this (except Pacemaker/Corosync) <br>
</div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">
On Thu, May 15, 2014 at 1:49 AM, Сергей Мотовиловец <span dir="ltr"><<a href="mailto:motovilovets.sergey@gmail.com" target="_blank">motovilovets.sergey@gmail.com</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div class="h5">
<div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px">Hello everyone!</span><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I'm facing some troubles with nova and cinder here.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I have 2 control nodes (active/active) in my testing environment with Percona XtraDB cluster (Galera+xtrabackup) + garbd on a separate node (to avoid split-brain) + OpenStack Icehouse, latest from Ubuntu 14.04 main repo.</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">The problem is horizontal scalability of nova-conductor and cinder-scheduler services, seems like all active instances of these services are trying to execute same MySQL queries they get from Rabbit, which leads to numerous deadlocks in set-up with Galera. </div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">In case when multiple nova-conductor services are running (and using MySQL instances on corresponding control nodes) it appears as "Deadlock found when trying to get lock; try restarting transaction" in log.<br>
</div><div style="font-family:arial,sans-serif;font-size:13px">With cinder-scheduler it leads to "InvalidBDM: Block Device Mapping is Invalid."</div><div style="font-family:arial,sans-serif;font-size:13px"><br>
</div>
<div style="font-family:arial,sans-serif;font-size:13px">Is there any possible way to make multiple instances of these services running simultaneously and not duplicating queries? <br>(I don't really like the idea of handling this with Heartbeat+Pacemaker or other similar stuff, mostly because I'm thinking about equal load distribution across control nodes, but in this case it seems like it has an opposite effect, multiplying load on MySQL)</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Another thing that is extremely annoying: if instance stuck in ERROR state because of deadlock during its termination - it is impossible to terminate instance anymore in Horizon, only via nova-api with reset-state. How can this be handled?</div>
<div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">I'd really appreciate any help/advises/thoughts regarding these problems.</div><div style="font-family:arial,sans-serif;font-size:13px">
<br></div><div style="font-family:arial,sans-serif;font-size:13px"><br></div><div style="font-family:arial,sans-serif;font-size:13px">Best regards,</div><div style="font-family:arial,sans-serif;font-size:13px">Motovilovets Sergey</div>
<div style="font-family:arial,sans-serif;font-size:13px">Software Operation Engineer</div></div>
<br></div></div>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div>