<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">That will work until you reboot the system. <div class=""><br class=""></div><div class="">In Red Hat you can modify the /etc/security/limits.conf</div><div class=""><br class=""></div><div class="">Remo </div><div class=""><font color="#333333" face="Consolas, Monaco, Andale Mono, monospace" class=""><span style="font-size: 14.399999618530273px; white-space: nowrap; background-color: rgb(247, 247, 247);" class=""><br class=""></span></font><div class=""><br class=""></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Dec 19, 2016, at 09:57, John Petrini <<a href="mailto:jpetrini@coredial.com" class="">jpetrini@coredial.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Hi Preshant,</div><div class=""><br class=""></div><div class="">You can change the open file limit of the running process by echoing the value to it. For example...</div><div class=""><br class=""></div><div class=""><pre style="white-space:pre-wrap;color:rgb(80,0,80)" class=""><code class=""><span class="gmail-il">echo</span> -n "Max open files=8192:unlimited" > /proc/<PID>/<wbr class=""><span class="gmail-il">limits</span></code></pre><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature"><div dir="ltr" class=""><div dir="ltr" class=""><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><p style="margin-top:8px;margin-bottom:8px;font-family:helvetica,verdana,arial,sans-serif;font-size:0.85em;color:rgb(51,51,51);line-height:1.4" class=""><span style="font-size:0.85em;line-height:1.4" class="">___</span><br class=""></p><p style="margin-top:8px;margin-bottom:8px;font-family:helvetica,verdana,arial,sans-serif;font-size:1.1em;color:rgb(51,51,51);line-height:1.4" class="">John Petrini</p><p style="margin-top:8px;margin-bottom:8px;font-family:helvetica,verdana,arial,sans-serif;font-size:0.85em;color:rgb(51,51,51);line-height:1.6" class=""><br class=""></p></div></div></div></div><a href="http:/" target="_blank" class=""></a></div></div></div></div>
<br class=""><div class="gmail_quote">On Mon, Dec 19, 2016 at 12:21 PM, Prashant Shetty <span dir="ltr" class=""><<a href="mailto:prashantshetty1985@gmail.com" target="_blank" class="">prashantshetty1985@gmail.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div class=""><div class=""><div class="">Hi Arne, <br class="">Thanks for your reply. Currently all these services are running on ubuntu controller under screen.<br class=""></div>Do we have any option to set the file limit option for n-api service in this case?. I am not using systemd in my setup to run these services.<br class=""><br class=""></div>Thanks,<br class=""></div>Prashant<br class=""></div><div class="gmail-HOEnZb"><div class="gmail-h5"><div class="gmail_extra"><br class=""><div class="gmail_quote">On Mon, Dec 19, 2016 at 10:19 PM, Arne Wiebalck <span dir="ltr" class=""><<a href="mailto:Arne.Wiebalck@cern.ch" target="_blank" class="">Arne.Wiebalck@cern.ch</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="word-wrap:break-word" class="">
<div class="">Prashant,</div>
<div class=""><br class="">
</div>
<div class="">If this is for systemd, how about changing the nova-api unit file?</div>
<div class=""><br class="">
</div>
<div class="">Something like</div>
<div class=""><br class="">
</div>
<div class="">—></div>
<div class="">
<div class="">[Service]</div>
<div class="">...</div>
<div class="">LimitNOFILE=65536</div>
</div>
<div class=""><—</div>
<div class=""><br class="">
</div>
<div class="">should do it. </div>
<div class=""><br class="">
</div>
<div class="">Cheers,</div>
<div class=""> Arne</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<br class="">
<div class="">
<blockquote type="cite" class=""><div class=""><div class="gmail-m_2676013432257320070h5">
<div class="">On 19 Dec 2016, at 17:23, Prashant Shetty <<a href="mailto:prashantshetty1985@gmail.com" target="_blank" class="">prashantshetty1985@gmail.com</a>> wrote:</div>
<br class="gmail-m_2676013432257320070m_8343404362413844012Apple-interchange-newline">
</div></div><div class=""><div class=""><div class="gmail-m_2676013432257320070h5">
<div dir="ltr" class=""><font size="2" class=""><span style="font-family:verdana,sans-serif" class="">Team,<br class="">
<br class="">
I have scale setup and metadata requests are seems to fail from instance. Main reason for failure is "Max open files" limit(1024) set on nova-api service.
<br class="">
Though on controller we have set max open file limit of 65k(limit.conf), nova-api always comes up with 1024 limit causing failure.<br class="">
<br class="">
Could someone let me know how can we change the max open files limit of nova-api service?<br class="">
<br class="">
Setup Details:<br class="">
<br class="">
· Single controller<br class="">
· 500 KVM computes<br class="">
· Devstack branch: stable/newton<br class="">
· We have native metadata and dhcp running on platform<br class="">
· 3750 instances<br class="">
<br class="">
<br class="">
stack@controller:/opt/stack/lo<wbr class="">gs$ ps aux | grep nova-api<br class="">
stack 14998 2.2 0.3 272104 121648 pts/8 S+ 09:53 0:14 /usr/bin/python /usr/local/bin/nova-api<br class="">
stack@controller:/opt/stack/lo<wbr class="">gs$<br class="">
stack@controller:/opt/stack/lo<wbr class="">gs$<br class="">
stack@controller:/opt/stack/lo<wbr class="">gs$ cat /proc/14998/limits<br class="">
Limit Soft Limit Hard Limit Units<br class="">
Max cpu time unlimited unlimited seconds<br class="">
Max file size unlimited unlimited bytes<br class="">
Max data size unlimited unlimited bytes<br class="">
Max stack size 8388608 unlimited bytes<br class="">
Max core file size unlimited unlimited bytes<br class="">
Max resident set unlimited unlimited bytes<br class="">
Max processes 128611 128611 processes<br class="">
Max open files 1024 4096 files<br class="">
Max locked memory 65536 65536 bytes<br class="">
Max address space unlimited unlimited bytes<br class="">
Max file locks unlimited unlimited locks<br class="">
Max pending signals 128611 128611 signals<br class="">
Max msgqueue size 819200 819200 bytes<br class="">
Max nice priority 0 0<br class="">
Max realtime priority 0 0<br class="">
Max realtime timeout unlimited unlimited us<br class="">
stack@controller:/opt/stack/lo<wbr class="">gs$<br class="">
<br class="">
n-api:<br class="">
</span></font><br class="">
<div style="margin:0px;padding:0px;border-width:0px;border-style:none;outline:none 0px;font-size:13px;vertical-align:baseline;color:rgb(51,51,51);font-family:monospace;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-image:none;background-color:rgb(255,255,255);background-size:initial;background-origin:initial;background-clip:initial;background-position:0% 0%;background-repeat:repeat" class="gmail-m_2676013432257320070m_8343404362413844012gmail-norm">
2016-11-08 18:44:26.168 30069 INFO nova.metadata.wsgi.server [req-fb4d729b-a1cd-4df1-aaf8-3<wbr class="">f854a739cce - -] (30069) wsgi exited, is_accepting=True<br class="">
Traceback (most recent call last):<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/hubs/hub.<wbr class="">py", line 457, in fire_timers<br class="">
timer()<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/hubs/timer.<wbr class="">py", line 58, in __call__<br class="">
cb(*args, **kw)<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/event.py", line 168, in _do_send<br class="">
waiter.switch(result)<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/greenthread<wbr class="">.py", line 214, in main<br class="">
result = function(*args, **kwargs)<br class="">
File "/opt/stack/nova/nova/utils.py<wbr class="">", line 1066, in context_wrapper<br class="">
return func(*args, **kwargs)<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/wsgi.py", line 865, in server<br class="">
client_socket = sock.accept()<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/greenio/<wbr class="">base.py", line 214, in accept<br class="">
res = socket_accept(fd)<br class="">
File "/usr/local/lib/python2.7/dist<wbr class="">-packages/eventlet/greenio/<wbr class="">base.py", line 56, in socket_accept<br class="">
return descriptor.accept()<br class="">
File "/usr/lib/python2.7/socket.py"<wbr class="">, line 206, in accept<br class="">
sock, addr = self._sock.accept()<br class="">
error: [Errno 24] Too many open files<br class="">
</div>
<span style="color:rgb(51,51,51);font-family:monospace;font-size:13px;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;display:inline;float:none;background-color:rgb(255,255,255)" class=""><br class="">
</span><font size="2" class=""><span style="font-family:verdana,sans-serif" class="">Thanks,<br class="">
Prashant</span></font><br class="">
<br class="">
</div></div></div>
______________________________<wbr class="">_________________<br class="">
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" class="">
http://lists.openstack.org/cgi<wbr class="">-bin/mailman/listinfo/openstac<wbr class="">k</a><br class="">
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank" class="">openstack@lists.openstack.org</a><br class="">
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank" class="">
http://lists.openstack.org/cgi<wbr class="">-bin/mailman/listinfo/openstac<wbr class="">k</a><br class="">
</div>
</blockquote>
</div>
<br class="">
<div class="">--<br class="">
Arne Wiebalck<br class="">
CERN IT </div>
<br class="">
</div>
</blockquote></div><br class=""></div>
</div></div><br class="">______________________________<wbr class="">_________________<br class="">
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank" class="">http://lists.openstack.org/<wbr class="">cgi-bin/mailman/listinfo/<wbr class="">openstack</a><br class="">
Post to : <a href="mailto:openstack@lists.openstack.org" class="">openstack@lists.openstack.org</a><br class="">
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank" class="">http://lists.openstack.org/<wbr class="">cgi-bin/mailman/listinfo/<wbr class="">openstack</a><br class="">
<br class=""></blockquote></div><br class=""></div></div></div>
!DSPAM:1,585821ba284381377620225!
_______________________________________________<br class="">Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br class="">Post to : <a href="mailto:openstack@lists.openstack.org" class="">openstack@lists.openstack.org</a><br class="">Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br class=""><br class=""><br class="">!DSPAM:1,585821ba284381377620225!<br class=""></div></blockquote></div><br class=""></div></div></body></html>