<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">On 08/07/2013 08:05 PM, Miller, Mark M
(EB SW Cloud - R&D - Corvallis) wrote:<br>
</div>
<blockquote
cite="mid:D6182642CE6D2D4FBFCDF99946E249883B360EF9@G9W0343.americas.hpqcorp.net"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.hoenzb
{mso-style-name:hoenzb;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I
have been thinking about the keystone user lookup GET API
for a split LDAP/SQL backend when you are using a read only
LDAP backend:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
<a moz-do-not-send="true"
href="http://15.253.58.165:35357/v3/auth/tokens">http://15.253.58.165:35357/v3/auth/tokens</a><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">A
suggestion has been made to add additional lookup
constraints via a filter. The problem with read only LDAP
databases is that you are not able to tag the keystone users
with any flags to indicate that they are keystone users. The
current Keystone H-2 LDAP backend code performs the
_ldap_get_all function (which took 1 ½ hours today) and must
then look to see which of those users are in the keystone
database because the REST API call only returned the one
user that I had assigned a project role to. I am thinking
that this logic is backwards. Instead of starting with the
LDAP server, start by querying the keystone SQL database for
LDAP users and then query the LDAP system for those users a
certain number at a time (good use of pagination). By the
way, I am assuming that keystone finds the LDAP users by
looking in the user_project_metadata, user_group_membership_
anduser_domain_metadata tables for user IDs that are not in
the user table.</span></p>
</div>
</blockquote>
<br>
We should probably just drop the list_user functionality from
Keystone, as it probably doens't belong there. listing users in a
project it probably fine, but all users in the system only makes
sense for really trivial systems.<br>
<br>
Most LDAP servers limit the number of records returned. I know in
FreeIPA, we had 200 records, and then you needed a filter to find
what you wanted beyond that. Pagination is a bettersolution,
although I shudder to think of the impact of all those live cursors
on a heavily loaded Enterprise directory.<br>
<br>
<br>
<blockquote
cite="mid:D6182642CE6D2D4FBFCDF99946E249883B360EF9@G9W0343.americas.hpqcorp.net"
type="cite">
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Mark<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span
style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
Dolph Mathews [<a class="moz-txt-link-freetext" href="mailto:dolph.mathews@gmail.com">mailto:dolph.mathews@gmail.com</a>]
<br>
<b>Sent:</b> Wednesday, August 07, 2013 4:40 PM<br>
<b>To:</b> OpenStack Development Mailing List<br>
<b>Cc:</b> Taylor, Monty<br>
<b>Subject:</b> Re: [openstack-dev] Keystone Split Backend
LDAP Hang Problem<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">That's been a "don't do that" for quite a
while, but we might finally have a solution in havana:<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <a moz-do-not-send="true"
href="https://blueprints.launchpad.net/keystone/+spec/pagination-backend-support">https://blueprints.launchpad.net/keystone/+spec/pagination-backend-support</a><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, Aug 7, 2013 at 3:56 PM, Miller,
Mark M (EB SW Cloud - R&D - Corvallis) <<a
moz-do-not-send="true"
href="mailto:mark.m.miller@hp.com" target="_blank">mark.m.miller@hp.com</a>>
wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I
ran into an issue/problem with keystone and it is ok
to simply tell me to “don’t do that”, but I am
wondering how others approach this problem. </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I
have the keystone H-2 split backend code connected
the HP Enterprise Directory which is humongous in
size. From that directory I have only one user
configured with a project role in keystone. When I
performed the following REST API call:</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">GET:
<a moz-do-not-send="true"
href="http://15.253.58.141:35357/v3/users"
target="_blank">http://15.253.58.141:35357/v3/users</a></span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">The
keystone server took almost an hour and a half to
process my request before responding with the
correct information:</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
08:54:24 DEBUG [keystone.common.ldap.core] LDAP
bind: dn=cn=CloudOSKeystoneDev, ou=Applications, o=<a
moz-do-not-send="true" href="http://hp.com"
target="_blank">hp.com</a></span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
08:54:25 DEBUG [keystone.common.ldap.core] In
get_connection 6 user: cn=CloudOSKeystoneDev,
ou=Applications, o=<a moz-do-not-send="true"
href="http://hp.com" target="_blank">hp.com</a></span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
08:54:25 DEBUG [keystone.common.ldap.core] MY
query in _ldap_get_all filter: None, query:
(&(objectClass=hpPerson))</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
<span style="background:yellow">08:54:25</span>
DEBUG [keystone.common.ldap.core] LDAP search:
dn=ou=People,o=<a moz-do-not-send="true"
href="http://hp.com" target="_blank">hp.com</a>,
scope=2, query=(&(objectClass=hpPerson)),
attrs=['None', 'userPassword', 'hpStatus', 'mail',
'cn']</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
<span style="background:yellow">10:20:10</span>
INFO [access] 15.253.57.88 - - [28/Jul/2013:17:20:10
+0000] "GET
<a moz-do-not-send="true"
href="http://15.253.58.141:35357/v3/users"
target="_blank">http://15.253.58.141:35357/v3/users</a>
HTTP/1.0" 200 87832184</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">2013-07-28
10:20:25 DEBUG [eventlet.wsgi.server]
15.253.57.88 - - [28/Jul/2013 10:20:25] "GET
/v3/users HTTP/1.1" 200 87832342 5160.268039</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">REST
API response:</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">{</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"user": {</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"name": "<a moz-do-not-send="true"
href="mailto:mark.m.miller@hp.com" target="_blank">mark.m.miller@hp.com</a>",</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"links": {</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"self": "<a moz-do-not-send="true"
href="http://localhost:5000/v3/users/mark.m.miller@hp.com"
target="_blank">http://localhost:5000/v3/users/mark.m.miller@hp.com</a>"</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
},</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"enabled": "Active",</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"domain_id": "default",</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"email": "<a moz-do-not-send="true"
href="mailto:mark_m_miller@hp.com" target="_blank">mark_m_miller@hp.com</a>",</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
"id": "<a moz-do-not-send="true"
href="mailto:mark.m.miller@hp.com" target="_blank">mark.m.miller@hp.com</a>"</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">
}</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:.5in"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">}</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">After
completing my request I found that Keystone was
locked up and required a stop/start service command
to get it responding again. How do other people with
ldap backends handle this problem?</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span
style="color:#888888"><o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Mark</span><span
style="color:#888888"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a moz-do-not-send="true"
href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a moz-do-not-send="true"
href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev"
target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-Dolph <o:p></o:p></p>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
OpenStack-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a>
<a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a>
</pre>
</blockquote>
<br>
</body>
</html>