<div dir="ltr">Hi Saverio,<div><br></div><div>The issue seems to be related to neutron tuning. We observed the same issue with stable/ocata branch code. When we tuned few neutron parameters it is working fine.</div><div>Thanks for your suggestion.</div><div><br></div><div>Thanks,</div><div>Satya.P</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 22, 2017 at 10:10 AM, Satyanarayana Patibandla <span dir="ltr"><<a href="mailto:satya.patibandla@gmail.com" target="_blank">satya.patibandla@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Saverio,<div><br></div><div>Thanks for your inputs. Will test with statable/ocata branch code and will share the result.</div><div><br></div><div>Thanks,</div><div>Satya.P</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Feb 22, 2017 at 1:54 AM, Saverio Proto <span dir="ltr"><<a href="mailto:zioproto@gmail.com" target="_blank">zioproto@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
I would use at least the stable/ocata branch. If you just use master<br>
that is not supposed to be stable, and also I am not sure if you can<br>
fill a bug against a specific commit in master.<br>
<span class="m_-186719621597207765HOEnZb"><font color="#888888"><br>
Saverio<br>
</font></span><div class="m_-186719621597207765HOEnZb"><div class="m_-186719621597207765h5"><br>
2017-02-21 21:12 GMT+01:00 Satyanarayana Patibandla<br>
<<a href="mailto:satya.patibandla@gmail.com" target="_blank">satya.patibandla@gmail.com</a>>:<br>
> Hi Saverio,<br>
><br>
> We have tried to create 20 VMs each time using heat template. There is 1 sec<br>
> time gap between each VM creation request. When we reached 114 VMs we got<br>
> the error mentioned in the below mail.Heat template will boot instance from<br>
> volume and it assigns floating IP to the instance.<br>
><br>
> Except neutron-server container we restarted all the neutron agent<br>
> containers which are present on all network and compute nodes. We are using<br>
> kolla to deploy openstack services.<br>
><br>
> We are using 1 month old master branch openstack code to deploy our<br>
> services.<br>
><br>
> Please find the error logs in the below link.<br>
> <a href="http://paste.openstack.org/show/599892/" rel="noreferrer" target="_blank">http://paste.openstack.org/sho<wbr>w/599892/</a><br>
><br>
> Thanks,<br>
> Satya.P<br>
><br>
> On Wed, Feb 22, 2017 at 12:21 AM, Saverio Proto <<a href="mailto:zioproto@gmail.com" target="_blank">zioproto@gmail.com</a>> wrote:<br>
>><br>
>> Hello Satya,<br>
>><br>
>> I would fill a bug on launchpad for this issue.<br>
>> 114 VMs is not much. Can you identify how to trigger the issue to<br>
>> reproduce it ? or it just happens randomly ?<br>
>><br>
>> When you say rebooting the network node, do you mean the server<br>
>> running the neutron-server process ?<br>
>><br>
>> what version and distribution of openstack are you using ?<br>
>><br>
>> thank you<br>
>><br>
>> Saverio<br>
>><br>
>><br>
>> 2017-02-21 13:54 GMT+01:00 Satyanarayana Patibandla<br>
>> <<a href="mailto:satya.patibandla@gmail.com" target="_blank">satya.patibandla@gmail.com</a>>:<br>
>> > Hi All,<br>
>> ><br>
>> > We are trying to deploy Openstack in our production environment. For<br>
>> > networking we are using DVR with out L3 HA. We are able to create 114<br>
>> > VMs<br>
>> > with out any issue. After creating 114 VMs we are getting the below<br>
>> > error.<br>
>> ><br>
>> > Error: <html><body><h1>504 Gateway Time-out</h1> The server didn't<br>
>> > respond<br>
>> > in time. </body></html><br>
>> ><br>
>> > Neutron services are getting freezed up due to a persistent lock on the<br>
>> > agents table. it seems one of the network node is holding the lock on<br>
>> > the<br>
>> > table. After rebooting the network node, the Neutron CLI was responsive<br>
>> > again.<br>
>> ><br>
>> > Neutron agent and neutron server is throwing below errors.<br>
>> ><br>
>> > Neutron-server errors:<br>
>> > ERROR oslo_db.sqlalchemy.exc_<wbr>filters "Can't reconnect until invalid<br>
>> > "<br>
>> > ERROR oslo_db.sqlalchemy.exc_filters InvalidRequestError: Can't<br>
>> > reconnect<br>
>> > until invalid transaction is rolled back<br>
>> > ERROR neutron.api.v2.resource [req-24fa6eaa-a9e0-4f55-97e0-5<wbr>9db203e72c6<br>
>> > 3eb776587c9c40569731ebe5c3557b<wbr>c7 f43e8699cd5a46e89ffe39e3cac753<wbr>41 - - -]<br>
>> > index failed: No details.<br>
>> > ERROR neutron.api.v2.resource DBError: Can't reconnect until invalid<br>
>> > transaction is rolled back<br>
>> ><br>
>> ><br>
>> > Neutron agents errors:<br>
>> > MessagingTimeout: Timed out waiting for a reply to message ID<br>
>> > 40638b6bf12c44cd9a404ecaa14a99<wbr>09<br>
>> ><br>
>> > Could you please provide us your valuable inputs or suggestions for<br>
>> > above<br>
>> > errors.<br>
>> ><br>
>> > Thanks,<br>
>> > Satya.P<br>
>> ><br>
>> > ______________________________<wbr>_________________<br>
>> > OpenStack-operators mailing list<br>
>> > <a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.open<wbr>stack.org</a><br>
>> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k-operators</a><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>