<div dir="ltr">Joshua,<div><br></div><div>We are performing few scaling tests for our solution and see that there are errors as below:</div><div><br></div><div><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">Failed saving logbook 'cc6f5cbd-c2f7-4432-9ca6-fff185cf853b'\n  InternalError: (pymysql.err.InternalError) (1205, u'Lock wait timeout exceeded; try restarting transaction') [SQL: u'UPDATE logbooks SET created_at=%s, updated_at=%s, meta=%s, name=%s, uuid=%s WHERE logbooks.uuid = %s'] [parameters: (datetime.datetime(2016, 3, 18, 18, 16, 40), datetime.datetime(2016, 3, 23, 3, 3, 44, 95395), u'{}', u'test', u'cc6f5cbd-c2f7-4432-9ca6-fff185cf853b', u'cc6f5cbd-c2f7-4432-9ca6-fff185cf853b')]"</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">We have about 800 flows as of now and e<span style="font-family:arial,sans-serif">ach flow is updated in the same logbook in a separate eventlet thread.</span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">Every thread calls save_logbook() on the same logbook record. I think this function is trying to update logbook record even though my usecase needs only flow details to be inserted and it doesn't update any information related to logbook.</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">Probably one of the threads was holding the lock while updating, and others tried for lock and failed after the default interval has elapsed.</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">I can think of few alternatives at the moment:</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><span style="font-family:arial,sans-serif">1. Increase the number of logbooks</span><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black">2. Increase the <span style="color:maroon;font-family:Consolas,Menlo,Monaco,'Lucida Console','Liberation Mono','DejaVu Sans Mono','Bitstream Vera Sans Mono','Courier New',monospace,sans-serif;white-space:inherit;background-color:rgb(238,238,238)">innodb_lock_wait_timeout</span></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word">3. There are some suggestions to make the innodb transaction isolation level to "READ COMMITTED" instead of "REPEATABLE READ", but I am not very familiar of the side effects they can cause</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word">Appreciate your thoughts on given alternatives or probably even better alternative</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word">Thanks,</pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word">Kanthi</pre></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre><pre style="margin-top:0px;margin-bottom:0px;padding:0px;white-space:pre-wrap;word-wrap:break-word;font-size:13px;color:black"><br></pre></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Mar 20, 2016 at 10:00 PM, Joshua Harlow <span dir="ltr"><<a href="mailto:harlowja@fastmail.com" target="_blank">harlowja@fastmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">Lingxian Kong wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Kanthi, sorry for chiming in, I suggest you may have a chance to take<br>
a look at Mistral[1], which is the workflow as a service in<br>
OpenStack(or without OpenStack).<br>
</blockquote>
<br></span>
Out of curiosity, why? Seems the ML post was about 'TaskFlow persistence' not mistral, just saying (unsure how it is relevant to mention mistral in this)...<br>
<br>
Back to getting more coffee...<br>
<br>
-Josh<div class="HOEnZb"><div class="h5"><br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>