<div dir="ltr"><div>Thanks so much for helping me out here Erik. I'm making progress, but still running into a couple hang-ups. We're almost there! Also, I apologize for the some what erratic dump of info below. I'm feeling a tad overwhelmed, but trying to fight through it ;)<br><br></div>Here are the updates I've made thus far thanks to your assistance:<br><ol><li>Changed "/etc/keystone/domains/keystone.Default.conf" to "/etc/keystone/domains/keystone.mydomain.conf".</li><li>Added "driver = ldap" under "[identity]" in "/etc/keystone/domains/keystone.mydomain.conf".</li><li>Set "OPENSTACK_KEYSTONE_<wbr>MULTIDOMAIN_SUPPORT = True" in /etc/openstack_dashboard/<wbr>local_settings.</li><li>Replaced "/etc/keystone/policy.json" and "/etc/openstack_dashboard/<wbr>keystone-policy.json" with the v3cloudsample version (got it from OpenStack GitHub). Though I'm not exactly sure where I put that domain in this file?<br></li><li>Restarted httpd.</li></ol><p>OK, so now let's first test the local "admin" account. I navigate to the Horizon dashboard, enter in the domain "Default", the username "admin", and the password. This actually lets me log in! However, I can't get information from any other services. For example, selecting "Instances" results in a "Unable to retrieve instances". To troubleshoot this, I focused on a relatively "simpler" service like Glance.</p><p>When attempting to list images, it logs the below:<br></p><p>2017-07-26 09:14:23.009 4296 WARNING keystonemiddleware.auth_token [-] Identity response: {"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}<br>2017-07-26 09:14:23.010 4296 CRITICAL keystonemiddleware.auth_token [-] Unable to validate token: Identity server rejected authorization necessary to fetch token data</p><p>I also tried the below cURL:<br></p><p>curl -i \<br> -H "Content-Type: application/json" \<br> -d '<br>{ "auth": {<br> "identity": {<br> "methods": ["password"],<br> "password": {<br> "user": {<br> "name": "glance",<br> "domain": { "id": "default" },<br> "password": "MyPassword"<br> }<br> }<br> }<br> }<br>}' \<br> "<a href="http://localhost:5000/v3/auth/tokens">http://localhost:5000/v3/auth/tokens</a>" ; echo</p><p>{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}<br></p><p>However I do recall that when I originally set this up, the passwords were configured incorrectly for a bunch of service accounts (didn't set the passwords in the Packstack answer file). So I had to update the configuration file of said services to match what I setup in LDAP. I looked at the "CONFIG_GLANCE_KS_PW" in my original answer file and attempted to authenticate with that and bingo!</p><p>curl -i \<br> -H "Content-Type: application/json" \<br> -d '<br>{ "auth": {<br> "identity": {<br> "methods": ["password"],<br> "password": {<br> "user": {<br> "name": "glance",<br> "domain": { "id": "default" },<br> "password": "CONFIG_GLANCE_KS_PW"<br> }<br> }<br> }<br> }<br>}' \<br> "<a href="http://localhost:5000/v3/auth/tokens">http://localhost:5000/v3/auth/tokens</a>" ; echo</p><p>{"token": {"issued_at": "2017-07-26T14:51:07.000000Z", "audit_ids": ["rxLIb_6ARbeKlKV6X7S7mA"], "methods": ["password"], "expires_at": "2017-07-26T15:51:07.000000Z", "user": {"password_expires_at": null, "domain": {"id": "default", "name": "Default"}, "id": "9500dc78f7e24fe698e108537b6c9c71", "name": "glance"}}}</p><p>So I can either go through all of those configs and set the passwords back, or change the passwords for the accounts:</p><p>I try to update the password for the 'glance' user, however I get a "Duplicate entry" error:</p><p># openstack user set --domain default --password-prompt glance<br>User Password:<br>Repeat User Password:<br>Conflict occurred attempting to store user - Duplicate entry. (HTTP 409) (Request-ID: req-fe3965b4-0962-4d9b-9a85-8966b3508650)</p><p>As you see below, it appears I do have 2 of several users. Is it finding users in both LDAP and the local DB? I moved my domains/keystone_mydomain.conf file out and restarted httpd, however both users still show up. as you can see, it thinks they are all in the same "default" domain.<br></p><p># openstack user list --long<br>+----------------------------------+--------------+----------------------------------+---------+--------------------+------------------------------------+---------+<br>| ID | Name | Project | Domain | Description | Email | Enabled |<br>+----------------------------------+--------------+----------------------------------+---------+--------------------+------------------------------------+---------+<br>| 026a2c3fab15419f90c6a95ddb803ed8 | aodh | | default | | aodh@localhost | True |<br>| 0c19f40cb1364ebb876bfd90918dead2 | ceilometer | | default | | ceilometer@localhost | True |<br>| 11599c71ad114654b003b654e03a4e6b | neutron | | default | | neutron@localhost | True |<br>| 170716mgr | 170716mgr | | default | | | True |<br>| 39cc615e69824f108bc9366224f25a66 | nova | | default | | nova@localhost | True |<br>| 44c1b20c016d41ec9ea69f667540c227 | gnocchi | | default | | gnocchi@localhost | True |<br>| 498777mgr | 498777mgr | | default | | | True |<br>| 646b17dc9b7c4a4d8c96fe7fde9a2a7a | swift | | default | | swift@localhost | True |<br>| 7729a23a63824862825b2e7ed707b907 | heat-cfn | | default | | heat-cfn@localhost | True |<br>| 783720mgrod | 783720mgrod | | default | | | True |<br>| 7d8d021ac276468bb474a19b8fc1dfd2 | cinder | | default | | cinder@localhost | True |<br>| 86874cd1808a4c83ac998365a0ee4c4b | placement | | default | | placement@localhost | True |<br>| 9500dc78f7e24fe698e108537b6c9c71 | glance | | default | | glance@localhost | True |<br>| a24de5a8ef444a779a0fb460b663a9b1 | heat | | default | | heat@localhost | True |<br>| admin | admin | | default | | | None |<br>| c3c6aa2b0fc24e30a4615cbdd19bbbb0 | ironic | | default | | ironic@localhost | True |<br>| c76b9e28a88041c0b4805ea5932ddd2c | magnum | | default | | magnum@localhost | True |<br>| ceilometer | ceilometer | | default | | | True |<br>| cinder | cinder | | default | | | True |<br>| dde9fdcb268a4d44a29894f37981c448 | admin | f4b4256e014a47e983f2904a9a3dac8e | default | | root@localhost | True |<br>| glance | glance | | default | | | True |<br>| heat | heat | | default | | | True |<br>| magnum | magnum | | default | OpenStack | | True |<br>| magnum_admin | magnum_admin | | default | OpenStack | | True |<br>| neutron | neutron | | default | | | None |<br>| nova | nova | | default | | | True |<br>| openst_admin | openst_admin | | default | OpenStack | | True |<br>| osadmin | osadmin | f4b4256e014a47e983f2904a9a3dac8e | default | | | True |<br>| placement | placement | | default | | | True |<br>| swift | swift | | default | | | True |<br>| temgr | temgr | | default | | | True |<br>+----------------------------------+--------------+----------------------------------+---------+--------------------+------------------------------------+---------+<br></p><p>Would you happen to have any ideas on this duplicate users scenario?<br></p><p>On another note: now I go to try my domain account. This time I enter in the domain "mydomain", my username "temgr", and my password. I now get a somewhat familiar error:<br></p><p>"You are not authorized for any projects or domains."</p><p>If I try to find what projects my user is assigned to, I get the below error:<br></p><p># openstack project list --user temgr <br>You are not authorized to perform the requested action: identity:list_user_projects. (HTTP 403) (Request-ID: req-2ee22612-f023-4186-8cce-cf29005444a5)</p><p>If I switch back to the default *policy.json file, I can check (I created the "mydomain" domain and project to test).<br></p><p># openstack project list --user temgr --long<br>+----------------------------------+----------+----------------------------------+-----------------------------------+---------+<br>| ID | Name | Domain ID | Description | Enabled |<br>+----------------------------------+----------+----------------------------------+-----------------------------------+---------+<br>| 7872eb2b3630406b86b669aa978796bc | mydomain | 30d0cc8521be4074bb8289f6be12f3fe | | True |<br>| e33b4ba7b3a449f7b267694efe1dbd38 | services | default | Tenant for the openstack services | True |<br>| f4b4256e014a47e983f2904a9a3dac8e | admin | default | admin tenant | True |<br>+----------------------------------+----------+----------------------------------+-----------------------------------+---------+<b><br></b></p><p># openstack role list<br>+----------------------------------+------------------+<br>| ID | Name |<br>+----------------------------------+------------------+<br>| 2bf0aa5d94254e9a8a976000f925dcfd | ResellerAdmin |<br>| 614641898bf14b1bade2a9f331a52bac | heat_stack_owner |<br>| 6795eb6529be4eb395cc9f22f834ee72 | SwiftOperator |<br>| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ |<br>| c373a7e73ed84b00aac92698ef23af4a | admin |<br>| f9bd71427880479182c32b84a7c1611a | heat_stack_user |<br>+----------------------------------+------------------+</p><p># openstack domain list<br>+----------------------------------+----------+---------+--------------------+<br>| ID | Name | Enabled | Description |<br>+----------------------------------+----------+---------+--------------------+<br>| 30d0cc8521be4074bb8289f6be12f3fe | mydomain | True | |<br>| 90a99943256b4a22a5c51352d428a7e5 | heat | True | |<br>| default | Default | True | The default domain |<br>| f950f5d49d8f4acba4790113580a956f | magnum | True | |<br>+----------------------------------+----------+---------+--------------------+<br></p><p># openstack role assignment list --user temgr<br>+----------------------------------+-------+-------+----------------------------------+----------------------------------+-----------+<br>| Role | User | Group | Project | Domain | Inherited |<br>+----------------------------------+-------+-------+----------------------------------+----------------------------------+-----------+<br>| c373a7e73ed84b00aac92698ef23af4a | temgr | | 7872eb2b3630406b86b669aa978796bc | | False |<br>| c373a7e73ed84b00aac92698ef23af4a | temgr | | e33b4ba7b3a449f7b267694efe1dbd38 | | False |<br>| c373a7e73ed84b00aac92698ef23af4a | temgr | | f4b4256e014a47e983f2904a9a3dac8e | | False |<br>| c373a7e73ed84b00aac92698ef23af4a | temgr | | | 30d0cc8521be4074bb8289f6be12f3fe | False |<br>| c373a7e73ed84b00aac92698ef23af4a | temgr | | | default | False |<br>+----------------------------------+-------+-------+----------------------------------+----------------------------------+-----------+<br></p><p>Am I missing something with the above configuration?</p><p>Again.. Sorry this was so scattered. I can provide any additional information of course. Any ideas or direction would be much appreciated.<br></p><p><br></p></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 25, 2017 at 8:33 PM, Erik McCormick <span dir="ltr"><<a href="mailto:emccormick@cirrusseven.com" target="_blank">emccormick@cirrusseven.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Answers inline below...<br><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Tue, Jul 25, 2017 at 5:26 PM, Tristan Evans <span dir="ltr"><<a href="mailto:azurepancake@gmail.com" target="_blank">azurepancake@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><p>Hey Erik,</p>
<p>The funny thing is that LDAP authentication works fine as an identity backend until I move the "[ldap]" configuration into
<span>"/etc/keystone/domains</span>/<span>keyston<wbr>e.Default.conf</span>" and enable the below in "/etc/keystone/keystone.conf":</p><span class="m_5204893249544709177gmail-">
<p>[identity]<br>
domain_specific_drivers_enable<wbr>d = True<br>
domain_config_dir = /etc/keystone/domains</p>
</span><p>#driver = ldap</p></div></blockquote></span><div><span style="font-size:13.3333px">^^^ driver = ldap needs to be set in your domain file, but is missing from your sample below. </span></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><p><font size="2"><span style="font-size:10pt"><br></span></font></p><p><font size="2"><span style="font-size:10pt">Once that is setup and httpd is restarted, Keystone logs nothing, but 500 errors for all requests..<br></span></font></p><p><font size="2"><span style="font-size:10pt"></span></font>I did need
to create all the service users in the LDAP domain originally to get
this to work, as well as assign them to the correct projects and roles.
This makes me think it's not a LDAP configuration per
se, as it works with the same configuration, as long as it's in
"keystone.conf" and that the "domain_specific_drivers_enabl<wbr>ed" is set to
"false".</p>
<p>Regardless, I'm very new to this stuff so I could totally be wrong, so here is my LDAP config
<span>😊</span></p>
<div>[ldap]<br>url = ldap://<a href="http://domain.net" target="_blank">domain.net</a><br>user = CN=adquery,OU=LinuxAccts,OU=Us<wbr>erAccounts,OU=USACAN,OU=NorthA<wbr>merica,OU=Global,DC=DOMAIN,DC=<wbr>NET<br>password =password<br>suffix = DC=domain,DC=net<br>query_scope = sub<br>use_pool = true<br>use_auth_pool = true<br>user_tree_dn = OU=UserAccounts,OU=USACAN,OU=N<wbr>orthAmerica,OU=Global,DC=DOMAI<wbr>N,DC=NET<br>user_objectclass = person<br>user_filter = (memberOf=CN=OpenStackAdmins,O<wbr>U=OpenStackGroups,OU=UserAccou<wbr>nts,OU=USACAN,OU=NorthAmerica,<wbr>OU=Global,DC=DOMAIN,DC=NET)<br>user_id_attribute = sAMAccountName<br>user_name_attribute = sAMAccountName<br>user_mail_attribute = mail<br>user_pass_attribute = userPassword<br>user_enabled_attribute = userAccountControl<br>user_enabled_mask = 2<br>user_enabled_invert = false<br>user_enabled_default = 512<br>user_default_project_id_attrib<wbr>ute =<br>user_additional_attribute_mapp<wbr>ing =<br>group_tree_dn = OU=OpenStackGroups,OU=UserAcco<wbr>unts,OU=USACAN,OU=NorthAmerica<wbr>,OU=Global,DC=DOMAIN,DC=NET<br>group_objectclass = group<br>group_id_attribute = cn<br>group_name_attribute = name<br>group_member_attribute = member<br>user_allow_create = False<br>user_allow_update = False<br>user_allow_delete = False<br>group_allow_create = False<br>group_allow_update = False<br>group_allow_delete = False<br></div>
<p>I like your suggestion in regards to just using LDAP for my users.
However, this would be dependent on the above setting (enabling specific
drivers per domain) which seems to be giving me problems, right? In
that example, Keystone would need to use the SQL driver for the
"Default" domain for services and use the LDAP driver for the user
domain?</p></div></blockquote></div></div><div>That's right, but once you set it up with default in mysql and your domain in LDAP you can troubleshoot the second domain unencumbered. I will say there is some tradeoff in that you must now specify a domain for everything you do, including logging in to Horizon. </div><div><br></div><div>You'll need to enable "OPENSTACK_KEYSTONE_<wbr>MULTIDOMAIN_SUPPORT = True", in /etc/openstack_dashboard/<wbr>local_settings, replace /etc/keystone/policy.json with the v3cloudsample one, entering in the domain of your global admin user, and also replace /etc/openstack_dashboard/<wbr>keystone-policy.json with the same file.</div><span class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><p>I just realized I'm not even sure how Keystone
figures out which domain to try and authenticate against for a
particular account. Sine kind of prioritization? I think I need to do some more reading...</p></div></blockquote></span><div>In the CLI you need to specify --domain for most things. For Horizon, see above :) <br></div><div><div class="h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><p>Thanks again.<br></p></div><div class="m_5204893249544709177gmail-HOEnZb"><div class="m_5204893249544709177gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jul 25, 2017 at 4:30 PM, Erik McCormick <span dir="ltr"><<a href="mailto:emccormick@cirrusseven.com" target="_blank">emccormick@cirrusseven.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I can't tell you too much about why Heat and Magnum need their own<br>
domains (something to do with Trusts and necessary admin rights).<br>
<br>
In terms of the LDAP connection being broken, it would help to see<br>
your domain config file. Also did you create all of the users in the<br>
LDAP domain as they were in the mysql one?<br>
<br>
One other comments: It's really not a great idea to host the default<br>
domain in LDAP unless you've got a pretty robust HA LDAP setup. If<br>
anything goes haywire with your LDAP server, then none of your service<br>
accounts will be able to authenticate and everything will go boom. You<br>
would be better off leaving Default as your service domain and<br>
creating another one for your users in LDAP.<br>
<br>
Cheers,<br>
Erik<br>
<div><div class="m_5204893249544709177gmail-m_-4049288720949958768h5"><br>
On Tue, Jul 25, 2017 at 11:49 AM, Tristan Evans <<a href="mailto:azurepancake@gmail.com" target="_blank">azurepancake@gmail.com</a>> wrote:<br>
> Hi,<br>
><br>
> I am running OpenStack Ocata that as originally provisioned using RDO<br>
> Packstack.<br>
><br>
> I currently have Keystone configured to use a single identity backend which<br>
> is LDAP. Everything works great with this configuration except Heat and<br>
> Magnum. Through some troubleshooting, it appears the problem may be that<br>
> these services operate within their own domains ("heat" and "magnum"<br>
> respectively). This results in errors like the below (in keystone.log) when<br>
> trying to build a cluster with Magnum:<br>
><br>
> 2017-07-20 11:12:22.509 7494 ERROR<br>
> magnum.conductor.handlers.comm<wbr>on.trust_manager Failed to create trustee and<br>
> trust for Cluster<br>
> 2017-07-20 11:12:22.509 7494 ERROR<br>
> magnum.conductor.handlers.comm<wbr>on.trust_manager NotFound: Could not find<br>
> domain: f950f5d49d8f4acba4790113580a95<wbr>6f. (HTTP 404)<br>
><br>
> I also caught the below as well:<br>
><br>
> 2017-07-20 10:32:24.122 20553 WARNING keystone.identity.core Found multiple<br>
> domains being mapped to a driver that does not support that (e.g. LDAP)<br>
> 2017-07-20 10:32:24.122 20553 WARNING keystone.common.wsgi Could not find<br>
> domain: f950f5d49d8f4acba4790113580a95<wbr>6f.<br>
><br>
> The domain does indeed exist:<br>
><br>
> # openstack domain list<br>
> 90a99943256b4a22a5c51352d428a7<wbr>e5 | heat | True<br>
> default | Default | True | The default domain<br>
> f950f5d49d8f4acba4790113580a95<wbr>6f | magnum | True<br>
><br>
> So through some research, I found that I can configure the below settings in<br>
> keystone.conf to choose specific drivers for specific domains:<br>
><br>
> [identity]<br>
> domain_specific_drivers_enable<wbr>d = True<br>
> domain_config_dir = /etc/keystone/domains<br>
><br>
> And then migrate my entire "[ldap]" configuration as<br>
> /etc/keystone/domains/keystone<wbr>.Default.conf.<br>
><br>
> I then restart httpd and attempt to list domains:<br>
><br>
> # openstack domain list<br>
> An unexpected error prevented the server from fulfilling your request. (HTTP<br>
> 500) (Request-ID: req-9d64587c-8bda-401b-83df-a0<wbr>c166ea629b)<br>
><br>
> If I look up that request ID in the log:<br>
><br>
> 2017-07-20 14:36:46.828 2621 DEBUG keystone.middleware.auth<br>
> [req-9d64587c-8bda-401b-83df-a<wbr>0c166ea629b - - - - -] There is either no auth<br>
> token in the request or the certificate issuer is not trusted. No auth<br>
> context will be set. fill_context<br>
> /usr/lib/python2.7/site-packag<wbr>es/keystone/middleware/auth.py<wbr>:188<br>
> 2017-07-20 14:36:46.829 2621 INFO keystone.common.wsgi<br>
> [req-9d64587c-8bda-401b-83df-a<wbr>0c166ea629b - - - - -] POST<br>
> <a href="http://10.11.184.50:5000/v3/auth/tokens" rel="noreferrer" target="_blank">http://10.11.184.50:5000/v3/au<wbr>th/tokens</a><br>
> 2017-07-20 14:36:46.848 2621 WARNING keystone.common.wsgi<br>
> [req-9d64587c-8bda-401b-83df-a<wbr>0c166ea629b - - - - -] An unexpected error<br>
> prevented the server from fulfilling your request.<br>
><br>
> I can't seem to find any other interesting errors in keystone.log. The above<br>
> just repeats over and over for each service attempting to authenticate.<br>
><br>
> If I remove the "domain_specific_drivers_enabl<wbr>ed" and "domain_config_dir"<br>
> options from keystone.conf (with my "[ldap]" configurations removed as<br>
> well), I can then successfully authenticate using MySQL for identity.<br>
><br>
> I'm at a total loss on what may be wrong, and confused as to why Heat and<br>
> Magnum need their own domains. Would anyone be able to help point me in the<br>
> right direction?<br>
><br>
><br>
</div></div>> ______________________________<wbr>_________________<br>
> Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
> Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
> Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi<wbr>-bin/mailman/listinfo/openstac<wbr>k</a><br>
><br>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>