<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Seems that page size option is a client feature that ldap server dont must respect. Look. at [1]</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
I read it a little bit, that the page size only affects the number of data entries to be returned to the client, but the server has to calculate all the entries in the directory.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Can you look in the ldap servers logfile and/or increase the debug_level in your configuration?<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0">
[1] <a href="https://serverfault.com/questions/328671/paging-using-ldapsearch/329027" id="LPlnkOWALinkPreview">
https://serverfault.com/questions/328671/paging-using-ldapsearch/329027</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1">
<div id="LPBorder_GTaHR0cHM6Ly9zZXJ2ZXJmYXVsdC5jb20vcXVlc3Rpb25zLzMyODY3MS9wYWdpbmctdXNpbmctbGRhcHNlYXJjaC8zMjkwMjc." class="LPBorder548478" style="width: 100%; margin-top: 16px; margin-bottom: 16px; position: relative; max-width: 800px; min-width: 424px;">
<table id="LPContainer548478" role="presentation" style="padding: 12px 36px 12px 12px; width: 100%; border-width: 1px; border-style: solid; border-color: rgb(200, 200, 200); border-radius: 2px;">
<tbody>
<tr style="border-spacing: 0px;" valign="top">
<td>
<div id="LPImageContainer548478" style="position: relative; margin-right: 12px; height: 160px; overflow: hidden;">
<a target="_blank" id="LPImageAnchor548478" href="https://serverfault.com/questions/328671/paging-using-ldapsearch/329027"><img id="LPThumbnailImageId548478" alt="" style="display: block;" width="160" height="160" src="https://cdn.sstatic.net/Sites/serverfault/Img/apple-touch-icon@2.png?v=9b1f48ae296b"></a></div>
</td>
<td style="width: 100%;">
<div id="LPTitle548478" style="font-size: 21px; font-weight: 300; margin-right: 8px; font-family: "wf_segoe-ui_light", "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px;">
<a target="_blank" id="LPUrlAnchor548478" href="https://serverfault.com/questions/328671/paging-using-ldapsearch/329027" style="text-decoration: none;">Paging using ldapsearch</a></div>
<div id="LPDescription548478" style="font-size: 14px; max-height: 100px; font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; margin-bottom: 12px; margin-right: 8px; overflow: hidden; color: rgb(102, 102, 102);">
I am searching an LDAP directory that has a much larger number of results than the sizelimit currently set,500, by slapd.conf that for all intents and purposes cannot be changed) My idea was to keep</div>
<div id="LPMetadata548478" style="font-size: 14px; font-weight: 400; font-family: "wf_segoe-ui_normal", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; color: rgb(166, 166, 166);">
serverfault.com</div>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<br>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>Von:</b> Albert Braden <ozzzo@yahoo.com><br>
<b>Gesendet:</b> Mittwoch, 24. Mai 2023 15:23<br>
<b>An:</b> openstack-discuss@lists.openstack.org <openstack-discuss@lists.openstack.org>; Kaster, Jörn <Joern.Kaster@epg.com><br>
<b>Betreff:</b> Re: AW: [kolla] [train] [keystone] Number of User/Group entities returned by LDAP exceeded size limit</font>
<div> </div>
</div>
<div>
<p></p>
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#cc1010; border-width:1px; padding:2pt; font-size:10pt; line-height:12p; font-family:Calibri; color:#cc1010; text-align:left; font-weight:bold">
OUTSIDE-EPG!</div>
<br>
<p></p>
<div>
<div>The Keystone documentation [1] appears to indicate that LDAP limitations can be worked around by enabling paging, using the page_size setting. Am I reading it wrong?<br>
<br>
[1] https://docs.openstack.org/keystone/train/admin/configuration.html#identity-ldap-server-set-up
</div>
<div class="x_yahoo_quoted" style="margin:10px 0px 0px 0.8ex; border-left:1px solid #ccc; padding-left:1ex">
<div style="font-family:'Helvetica Neue',Helvetica,Arial,sans-serif; font-size:13px; color:#26282a">
<div>On Wednesday, May 24, 2023, 02:34:23 AM EDT, Kaster, Jörn <joern.kaster@epg.com> wrote:
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div id="x_yiv6784134135"><style type="text/css">
<!--
#x_yiv6784134135 p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello Albert,</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
have seen your message on monday and think that it was replied personaly in the meantime. Anyway.</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I think this problem is not dedicated to the openstack services. The problem is caused by the ldap server. Which one do you use?</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Look in the documentation of the ldap server to configure a larger size limit.</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br clear="none">
</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
greets from here</div>
<div class="x_yiv6784134135elementToProof" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Jörn<br clear="none">
</div>
<div id="x_yiv6784134135appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_yiv6784134135yqt52794" class="x_yiv6784134135yqt9570179197">
<div dir="ltr" id="x_yiv6784134135divRplyFwdMsg"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Von:</b> Albert Braden <ozzzo@yahoo.com><br clear="none">
<b>Gesendet:</b> Dienstag, 23. Mai 2023 20:35<br clear="none">
<b>An:</b> OpenStack Discuss <openstack-discuss@lists.openstack.org><br clear="none">
<b>Betreff:</b> Re: [kolla] [train] [keystone] Number of User/Group entities returned by LDAP exceeded size limit</font>
<div> </div>
</div>
<div>
<p></p>
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#cc1010; border-width:1px; padding:2pt; font-size:10pt; font-family:Calibri; color:#cc1010; text-align:left; font-weight:bold">
OUTSIDE-EPG!</div>
<br clear="none">
<p></p>
<div>
<div>Nobody replied to this Friday afternoon so I'm trying again:<br clear="none">
<br clear="none">
On Friday, May 19, 2023, 09:29:17 AM EDT, Albert Braden <ozzzo@yahoo.com> wrote:<br clear="none">
<br clear="none">
<br clear="none">
We have 2052 groups in our LDAP server. We recently started getting an error when we try to list groups:<br clear="none">
<br clear="none">
$ os group list --domain AUTH.OURDOMAIN.COM<br clear="none">
Number of User/Group entities returned by LDAP exceeded size limit. Contact your LDAP administrator. (HTTP 500)<br clear="none">
<br clear="none">
I read the "Additional LDAP integration settings" section in [1] and then tried setting various values of page_size (10, 100, 1000) in the [ldap] section of keystone.conf but that didn't make a difference. What am I missing?<br clear="none">
<br clear="none">
[1] https://docs.openstack.org/keystone/train/admin/configuration.html#identity-ldap-server-set-up<br clear="none">
<br clear="none">
Here's the stack trace:<br clear="none">
<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application [req-198741c6-58b2-46b1-8622-bae1fc5c5280 d64c83e1ea954c368e9fe08a5d8450a1 47dc15c280c9436fadac4d41f1d54a64 - default default] Number of User/Group entities returned by LDAP exceeded size limit.
Contact your LDAP administrator.: keystone.exception.LDAPSizeLimitExceeded: Number of User/Group entities returned by LDAP exceeded size limit. Contact your LDAP administrator.<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application Traceback (most recent call last):<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 996, in search_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application attrlist, attrsonly)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 689, in wrapper<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return func(self, conn, *args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 824, in search_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application attrsonly)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 870, in search_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self.search_ext_s(base,scope,filterstr,attrlist,attrsonly,None,None,timeout=self.timeout)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1286, in search_ext_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self._apply_method_s(SimpleLDAPObject.search_ext_s,*args,**kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 1224, in _apply_method_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return func(self,*args,**kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 864, in search_ext_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self.result(msgid,all=1,timeout=timeout)[1]<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 756, in result<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application resp_type, resp_data, resp_msgid = self.result2(msgid,all,timeout)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 760, in result2<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application resp_type, resp_data, resp_msgid, resp_ctrls = self.result3(msgid,all,timeout)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 767, in result3<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application resp_ctrl_classes=resp_ctrl_classes<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 774, in result4<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application ldap_result = self._ldap_call(self._l.result4,msgid,all,timeout,add_ctrls,add_intermediates,add_extop)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 340, in _ldap_call<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application reraise(exc_type, exc_value, exc_traceback)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/compat.py", line 46, in reraise<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application raise exc_value<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib64/python3.6/site-packages/ldap/ldapobject.py", line 324, in _ldap_call<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application result = func(*args,**kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application ldap.SIZELIMIT_EXCEEDED: {'msgtype': 100, 'msgid': 2, 'result': 4, 'desc': 'Size limit exceeded', 'ctrls': []}<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application During handling of the above exception, another exception occurred:<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application Traceback (most recent call last):<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application rv = self.dispatch_request()<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self.view_functions[rule.endpoint](**req.view_args)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 480, in wrapper<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application resp = resource(*args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/flask/views.py", line 88, in view<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self.dispatch_request(*args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/flask_restful/__init__.py", line 595, in dispatch_request<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application resp = meth(*args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/api/groups.py", line 59, in get<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self._list_groups()<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/api/groups.py", line 86, in _list_groups<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application hints=hints)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/common/manager.py", line 116, in wrapped<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 414, in wrapper<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return f(self, *args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 424, in wrapper<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return f(self, *args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/core.py", line 1329, in list_groups<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application ref_list = driver.list_groups(hints)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", line 116, in list_groups<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return self.group.get_all_filtered(hints)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/core.py", line 474, in get_all_filtered<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application for group in self.get_all(query, hints)]<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 1647, in get_all<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application for x in self._ldap_get_all(hints, ldap_filter)]<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/common/driver_hints.py", line 42, in wrapper<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application return f(self, hints, *args, **kwargs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 1600, in _ldap_get_all<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application attrs)<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application File "/usr/lib/python3.6/site-packages/keystone/identity/backends/ldap/common.py", line 998, in search_s<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application raise exception.LDAPSizeLimitExceeded()<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application keystone.exception.LDAPSizeLimitExceeded: Number of User/Group entities returned by LDAP exceeded size limit. Contact your LDAP administrator.<br clear="none">
2023-05-15 20:18:41.932 36 ERROR keystone.server.flask.application </div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>