[Openstack] Keystone: domain_specific_drivers_enabled not working with LDAP

Tristan Evans azurepancake at gmail.com
Thu Jul 27 00:25:01 UTC 2017


Ok, just a quick update..

I resolved my duplicate user issue by simply deleting the duplicates..

example:

openstack user delete neutron

Now that I've done it, it seems incredibly obvious.. I think my brains were
a bit sizzled earlier ;)

The only problem that remains now is getting the LDAP accounts access. I
will keep playing with that, but I'm all ears to any additional advice you
may have to give :)

On Wed, Jul 26, 2017 at 1:03 PM, Tristan Evans <azurepancake at gmail.com>
wrote:

> Thanks so much for helping me out here Erik. I'm making progress, but
> still running into a couple hang-ups. We're almost there! Also, I apologize
> for the some what erratic dump of info below. I'm feeling a tad
> overwhelmed, but trying to fight through it ;)
>
> Here are the updates I've made thus far thanks to your assistance:
>
>    1. Changed "/etc/keystone/domains/keystone.Default.conf" to
>    "/etc/keystone/domains/keystone.mydomain.conf".
>    2. Added "driver = ldap" under "[identity]" in "/etc/keystone/domains/
>    keystone.mydomain.conf".
>    3. Set "OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True" in
>    /etc/openstack_dashboard/local_settings.
>    4. Replaced "/etc/keystone/policy.json" and
>    "/etc/openstack_dashboard/keystone-policy.json" with the v3cloudsample
>    version (got it from OpenStack GitHub). Though I'm not exactly sure where I
>    put that domain in this file?
>    5. Restarted httpd.
>
> OK, so now let's first test the local "admin" account. I navigate to the
> Horizon dashboard, enter in the domain "Default", the username "admin", and
> the password. This actually lets me log in! However, I can't get
> information from any other services. For example, selecting "Instances"
> results in a "Unable to retrieve instances". To troubleshoot this, I
> focused on a relatively "simpler" service like Glance.
>
> When attempting to list images, it logs the below:
>
> 2017-07-26 09:14:23.009 4296 WARNING keystonemiddleware.auth_token [-]
> Identity response: {"error": {"message": "The request you have made
> requires authentication.", "code": 401, "title": "Unauthorized"}}
> 2017-07-26 09:14:23.010 4296 CRITICAL keystonemiddleware.auth_token [-]
> Unable to validate token: Identity server rejected authorization necessary
> to fetch token data
>
> I also tried the below cURL:
>
> curl -i \
>   -H "Content-Type: application/json" \
>   -d '
> { "auth": {
>     "identity": {
>       "methods": ["password"],
>       "password": {
>         "user": {
>           "name": "glance",
>           "domain": { "id": "default" },
>           "password": "MyPassword"
>         }
>       }
>     }
>   }
> }' \
>   "http://localhost:5000/v3/auth/tokens" ; echo
>
> {"error": {"message": "The request you have made requires
> authentication.", "code": 401, "title": "Unauthorized"}}
>
> However I do recall that when I originally set this up, the passwords were
> configured incorrectly for a bunch of service accounts (didn't set the
> passwords in the Packstack answer file). So I had to update the
> configuration file of said services to match what I setup in LDAP. I looked
> at the "CONFIG_GLANCE_KS_PW" in my original answer file and attempted to
> authenticate with that and bingo!
>
> curl -i \
>   -H "Content-Type: application/json" \
>   -d '
> { "auth": {
>     "identity": {
>       "methods": ["password"],
>       "password": {
>         "user": {
>           "name": "glance",
>           "domain": { "id": "default" },
>           "password": "CONFIG_GLANCE_KS_PW"
>         }
>       }
>     }
>   }
> }' \
>   "http://localhost:5000/v3/auth/tokens" ; echo
>
> {"token": {"issued_at": "2017-07-26T14:51:07.000000Z", "audit_ids":
> ["rxLIb_6ARbeKlKV6X7S7mA"], "methods": ["password"], "expires_at":
> "2017-07-26T15:51:07.000000Z", "user": {"password_expires_at": null,
> "domain": {"id": "default", "name": "Default"}, "id": "
> 9500dc78f7e24fe698e108537b6c9c71", "name": "glance"}}}
>
> So I can either go through all of those configs and set the passwords
> back, or change the passwords for the accounts:
>
> I try to update the password for the 'glance' user, however I get a
> "Duplicate entry" error:
>
> # openstack user set --domain default --password-prompt glance
> User Password:
> Repeat User Password:
> Conflict occurred attempting to store user - Duplicate entry. (HTTP 409)
> (Request-ID: req-fe3965b4-0962-4d9b-9a85-8966b3508650)
>
> As you see below, it appears I do have 2 of several users. Is it finding
> users in both LDAP and the local DB? I moved my
> domains/keystone_mydomain.conf file out and restarted httpd, however both
> users still show up. as you can see, it thinks they are all in the same
> "default" domain.
>
> # openstack user list --long
> +----------------------------------+--------------+---------
> -------------------------+---------+--------------------+---
> ---------------------------------+---------+
> | ID                               | Name         |
> Project                          | Domain  | Description        |
> Email                              | Enabled |
> +----------------------------------+--------------+---------
> -------------------------+---------+--------------------+---
> ---------------------------------+---------+
> | 026a2c3fab15419f90c6a95ddb803ed8 | aodh
> |                                  | default |                    |
> aodh at localhost                     | True    |
> | 0c19f40cb1364ebb876bfd90918dead2 | ceilometer
> |                                  | default |                    |
> ceilometer at localhost               | True    |
> | 11599c71ad114654b003b654e03a4e6b | neutron
> |                                  | default |                    |
> neutron at localhost                  | True    |
> | 170716mgr                        | 170716mgr
> |                                  | default |  |       | True    |
> | 39cc615e69824f108bc9366224f25a66 | nova
> |                                  | default |                    |
> nova at localhost                     | True    |
> | 44c1b20c016d41ec9ea69f667540c227 | gnocchi
> |                                  | default |                    |
> gnocchi at localhost                  | True    |
> | 498777mgr                        | 498777mgr
> |                                  | default |  |         | True    |
> | 646b17dc9b7c4a4d8c96fe7fde9a2a7a | swift
> |                                  | default |                    |
> swift at localhost                    | True    |
> | 7729a23a63824862825b2e7ed707b907 | heat-cfn
> |                                  | default |                    |
> heat-cfn at localhost                 | True    |
> | 783720mgrod                      | 783720mgrod
> |                                  | default |                |  |
> True    |
> | 7d8d021ac276468bb474a19b8fc1dfd2 | cinder
> |                                  | default |                    |
> cinder at localhost                   | True    |
> | 86874cd1808a4c83ac998365a0ee4c4b | placement
> |                                  | default |                    |
> placement at localhost                | True    |
> | 9500dc78f7e24fe698e108537b6c9c71 | glance
> |                                  | default |                    |
> glance at localhost                   | True    |
> | a24de5a8ef444a779a0fb460b663a9b1 | heat
> |                                  | default |                    |
> heat at localhost                     | True    |
> | admin                            | admin
> |                                  | default |
> |                                    | None    |
> | c3c6aa2b0fc24e30a4615cbdd19bbbb0 | ironic
> |                                  | default |                    |
> ironic at localhost                   | True    |
> | c76b9e28a88041c0b4805ea5932ddd2c | magnum
> |                                  | default |                    |
> magnum at localhost                   | True    |
> | ceilometer                       | ceilometer
> |                                  | default |
> |                                    | True    |
> | cinder                           | cinder
> |                                  | default |
> |                                    | True    |
> | dde9fdcb268a4d44a29894f37981c448 | admin        |
> f4b4256e014a47e983f2904a9a3dac8e | default |                    |
> root at localhost                     | True    |
> | glance                           | glance
> |                                  | default |
> |                                    | True    |
> | heat                             | heat
> |                                  | default |
> |                                    | True    |
> | magnum                           | magnum
> |                                  | default | OpenStack
> |                                    | True    |
> | magnum_admin                     | magnum_admin
> |                                  | default | OpenStack
> |                                    | True    |
> | neutron                          | neutron
> |                                  | default |
> |                                    | None    |
> | nova                             | nova
> |                                  | default |
> |                                    | True    |
> | openst_admin                     | openst_admin
> |                                  | default | OpenStack
> |                                    | True    |
> | osadmin                          | osadmin      |
> f4b4256e014a47e983f2904a9a3dac8e | default |
> |                                    | True    |
> | placement                        | placement
> |                                  | default |
> |                                    | True    |
> | swift                            | swift
> |                                  | default |
> |                                    | True    |
> | temgr                            | temgr
> |                                  | default |  |       | True    |
> +----------------------------------+--------------+---------
> -------------------------+---------+--------------------+---
> ---------------------------------+---------+
>
> Would you happen to have any ideas on this duplicate users scenario?
>
> On another note: now I go to try my domain account. This time I enter in
> the domain "mydomain", my username "temgr", and my password. I now get a
> somewhat familiar error:
>
> "You are not authorized for any projects or domains."
>
> If I try to find what projects my user is assigned to, I get the below
> error:
>
> # openstack project list --user temgr
> You are not authorized to perform the requested action:
> identity:list_user_projects. (HTTP 403) (Request-ID:
> req-2ee22612-f023-4186-8cce-cf29005444a5)
>
> If I switch back to the default *policy.json file, I can check (I created
> the "mydomain" domain and project to test).
>
> # openstack project list --user temgr --long
> +----------------------------------+----------+-------------
> ---------------------+-----------------------------------+---------+
> | ID                               | Name     | Domain
> ID                        | Description                       | Enabled |
> +----------------------------------+----------+-------------
> ---------------------+-----------------------------------+---------+
> | 7872eb2b3630406b86b669aa978796bc | mydomain |
> 30d0cc8521be4074bb8289f6be12f3fe |                                   |
> True    |
> | e33b4ba7b3a449f7b267694efe1dbd38 | services |
> default                          | Tenant for the openstack services |
> True    |
> | f4b4256e014a47e983f2904a9a3dac8e | admin    |
> default                          | admin tenant                      |
> True    |
> +----------------------------------+----------+-------------
> ---------------------+-----------------------------------+---------+
>
> # openstack role list
> +----------------------------------+------------------+
> | ID                               | Name             |
> +----------------------------------+------------------+
> | 2bf0aa5d94254e9a8a976000f925dcfd | ResellerAdmin    |
> | 614641898bf14b1bade2a9f331a52bac | heat_stack_owner |
> | 6795eb6529be4eb395cc9f22f834ee72 | SwiftOperator    |
> | 9fe2ff9ee4384b1894a90878d3e92bab | _member_         |
> | c373a7e73ed84b00aac92698ef23af4a | admin            |
> | f9bd71427880479182c32b84a7c1611a | heat_stack_user  |
> +----------------------------------+------------------+
>
> # openstack domain list
> +----------------------------------+----------+---------+---
> -----------------+
> | ID                               | Name     | Enabled |
> Description        |
> +----------------------------------+----------+---------+---
> -----------------+
> | 30d0cc8521be4074bb8289f6be12f3fe | mydomain | True
> |                    |
> | 90a99943256b4a22a5c51352d428a7e5 | heat     | True
> |                    |
> | default                          | Default  | True    | The default
> domain |
> | f950f5d49d8f4acba4790113580a956f | magnum   | True
> |                    |
> +----------------------------------+----------+---------+---
> -----------------+
>
> # openstack role assignment list --user temgr
> +----------------------------------+-------+-------+--------
> --------------------------+----------------------------------+-----------+
> | Role                             | User  | Group |
> Project                          | Domain                           |
> Inherited |
> +----------------------------------+-------+-------+--------
> --------------------------+----------------------------------+-----------+
> | c373a7e73ed84b00aac92698ef23af4a | temgr |       |
> 7872eb2b3630406b86b669aa978796bc |                                  |
> False     |
> | c373a7e73ed84b00aac92698ef23af4a | temgr |       |
> e33b4ba7b3a449f7b267694efe1dbd38 |                                  |
> False     |
> | c373a7e73ed84b00aac92698ef23af4a | temgr |       |
> f4b4256e014a47e983f2904a9a3dac8e |                                  |
> False     |
> | c373a7e73ed84b00aac92698ef23af4a | temgr |
> |                                  | 30d0cc8521be4074bb8289f6be12f3fe |
> False     |
> | c373a7e73ed84b00aac92698ef23af4a | temgr |
> |                                  | default                          |
> False     |
> +----------------------------------+-------+-------+--------
> --------------------------+----------------------------------+-----------+
>
> Am I missing something with the above configuration?
>
> Again.. Sorry this was so scattered. I can provide any additional
> information of course. Any ideas or direction would be much appreciated.
>
>
>
> On Tue, Jul 25, 2017 at 8:33 PM, Erik McCormick <
> emccormick at cirrusseven.com> wrote:
>
>> Answers inline below...
>>
>> On Tue, Jul 25, 2017 at 5:26 PM, Tristan Evans <azurepancake at gmail.com>
>> wrote:
>>
>>> Hey Erik,
>>>
>>> The funny thing is that LDAP authentication works fine as an identity
>>> backend until I move the "[ldap]" configuration into
>>> "/etc/keystone/domains/keystone.Default.conf" and enable the below in
>>> "/etc/keystone/keystone.conf":
>>>
>>> [identity]
>>> domain_specific_drivers_enabled = True
>>> domain_config_dir = /etc/keystone/domains
>>>
>>> #driver = ldap
>>>
>> ^^^ driver = ldap needs to be set in your domain file, but is missing
>> from your sample below.
>>
>>>
>>> Once that is setup and httpd is restarted, Keystone logs nothing, but
>>> 500 errors for all requests..
>>>
>>> I did need to create all the service users in the LDAP domain originally
>>> to get this to work, as well as assign them to the correct projects and
>>> roles. This makes me think it's not a LDAP configuration per se, as it
>>> works with the same configuration, as long as it's in "keystone.conf" and
>>> that the "domain_specific_drivers_enabled" is set to "false".
>>>
>>> Regardless, I'm very new to this stuff so I could totally be wrong, so
>>> here is my LDAP config 😊
>>> [ldap]
>>> url = ldap://domain.net
>>> user = CN=adquery,OU=LinuxAccts,OU=UserAccounts,OU=USACAN,OU=NorthA
>>> merica,OU=Global,DC=DOMAIN,DC=NET
>>> password =password
>>> suffix = DC=domain,DC=net
>>> query_scope = sub
>>> use_pool = true
>>> use_auth_pool = true
>>> user_tree_dn = OU=UserAccounts,OU=USACAN,OU=N
>>> orthAmerica,OU=Global,DC=DOMAIN,DC=NET
>>> user_objectclass = person
>>> user_filter = (memberOf=CN=OpenStackAdmins,O
>>> U=OpenStackGroups,OU=UserAccounts,OU=USACAN,OU=NorthAmerica,
>>> OU=Global,DC=DOMAIN,DC=NET)
>>> user_id_attribute = sAMAccountName
>>> user_name_attribute = sAMAccountName
>>> user_mail_attribute = mail
>>> user_pass_attribute = userPassword
>>> user_enabled_attribute = userAccountControl
>>> user_enabled_mask = 2
>>> user_enabled_invert = false
>>> user_enabled_default = 512
>>> user_default_project_id_attribute =
>>> user_additional_attribute_mapping =
>>> group_tree_dn = OU=OpenStackGroups,OU=UserAcco
>>> unts,OU=USACAN,OU=NorthAmerica,OU=Global,DC=DOMAIN,DC=NET
>>> group_objectclass = group
>>> group_id_attribute = cn
>>> group_name_attribute = name
>>> group_member_attribute = member
>>> user_allow_create = False
>>> user_allow_update = False
>>> user_allow_delete = False
>>> group_allow_create = False
>>> group_allow_update = False
>>> group_allow_delete = False
>>>
>>> I like your suggestion in regards to just using LDAP for my users.
>>> However, this would be dependent on the above setting (enabling specific
>>> drivers per domain) which seems to be giving me problems, right? In that
>>> example, Keystone would need to use the SQL driver for the "Default" domain
>>> for services and use the LDAP driver for the user domain?
>>>
>> That's right, but once you set it up with default in mysql and your
>> domain in LDAP you can troubleshoot the second domain unencumbered. I will
>> say there is some tradeoff in that you must now specify a domain for
>> everything you do, including logging in to Horizon.
>>
>> You'll need to enable "OPENSTACK_KEYSTONE_MULTIDOMAIN_SUPPORT = True",
>> in /etc/openstack_dashboard/local_settings, replace
>> /etc/keystone/policy.json with the v3cloudsample one, entering in the
>> domain of your global admin user, and also replace
>> /etc/openstack_dashboard/keystone-policy.json with the same file.
>>
>>> I just realized I'm not even sure how Keystone figures out which domain
>>> to try and authenticate against for a particular account. Sine kind of
>>> prioritization? I think I need to do some more reading...
>>>
>> In the CLI you need to specify --domain for most things. For Horizon, see
>> above :)
>>
>>> Thanks again.
>>>
>>> On Tue, Jul 25, 2017 at 4:30 PM, Erik McCormick <
>>> emccormick at cirrusseven.com> wrote:
>>>
>>>> I can't tell you too much about why Heat and Magnum need their own
>>>> domains (something to do with Trusts and necessary admin rights).
>>>>
>>>> In terms of the LDAP connection being broken, it would help to see
>>>> your domain config file. Also did you create all of the users in the
>>>> LDAP domain as they were in the mysql one?
>>>>
>>>> One other comments: It's really not a great idea to host the default
>>>> domain in LDAP unless you've got a pretty robust HA LDAP setup. If
>>>> anything goes haywire with your LDAP server, then none of your service
>>>> accounts will be able to authenticate and everything will go boom. You
>>>> would be better off leaving Default as your service domain and
>>>> creating another one for your users in LDAP.
>>>>
>>>> Cheers,
>>>> Erik
>>>>
>>>> On Tue, Jul 25, 2017 at 11:49 AM, Tristan Evans <azurepancake at gmail.com>
>>>> wrote:
>>>> > Hi,
>>>> >
>>>> > I am running OpenStack Ocata that as originally provisioned using RDO
>>>> > Packstack.
>>>> >
>>>> > I currently have Keystone configured to use a single identity backend
>>>> which
>>>> > is LDAP. Everything works great with this configuration except Heat
>>>> and
>>>> > Magnum. Through some troubleshooting, it appears the problem may be
>>>> that
>>>> > these services operate within their own domains ("heat" and "magnum"
>>>> > respectively). This results in errors like the below (in
>>>> keystone.log) when
>>>> > trying to build a cluster with Magnum:
>>>> >
>>>> > 2017-07-20 11:12:22.509 7494 ERROR
>>>> > magnum.conductor.handlers.common.trust_manager Failed to create
>>>> trustee and
>>>> > trust for Cluster
>>>> > 2017-07-20 11:12:22.509 7494 ERROR
>>>> > magnum.conductor.handlers.common.trust_manager NotFound: Could not
>>>> find
>>>> > domain: f950f5d49d8f4acba4790113580a956f. (HTTP 404)
>>>> >
>>>> > I also caught the below as well:
>>>> >
>>>> > 2017-07-20 10:32:24.122 20553 WARNING keystone.identity.core Found
>>>> multiple
>>>> > domains being mapped to a driver that does not support that (e.g.
>>>> LDAP)
>>>> > 2017-07-20 10:32:24.122 20553 WARNING keystone.common.wsgi Could not
>>>> find
>>>> > domain: f950f5d49d8f4acba4790113580a956f.
>>>> >
>>>> > The domain does indeed exist:
>>>> >
>>>> > # openstack domain list
>>>> > 90a99943256b4a22a5c51352d428a7e5 | heat    | True
>>>> > default                          | Default | True    | The default
>>>> domain
>>>> > f950f5d49d8f4acba4790113580a956f | magnum  | True
>>>> >
>>>> > So through some research, I found that I can configure the below
>>>> settings in
>>>> > keystone.conf to choose specific drivers for specific domains:
>>>> >
>>>> > [identity]
>>>> > domain_specific_drivers_enabled = True
>>>> > domain_config_dir = /etc/keystone/domains
>>>> >
>>>> > And then migrate my entire "[ldap]" configuration as
>>>> > /etc/keystone/domains/keystone.Default.conf.
>>>> >
>>>> > I then restart httpd and attempt to list domains:
>>>> >
>>>> > # openstack domain list
>>>> > An unexpected error prevented the server from fulfilling your
>>>> request. (HTTP
>>>> > 500) (Request-ID: req-9d64587c-8bda-401b-83df-a0c166ea629b)
>>>> >
>>>> > If I look up that request ID in the log:
>>>> >
>>>> > 2017-07-20 14:36:46.828 2621 DEBUG keystone.middleware.auth
>>>> > [req-9d64587c-8bda-401b-83df-a0c166ea629b - - - - -] There is either
>>>> no auth
>>>> > token in the request or the certificate issuer is not trusted. No auth
>>>> > context will be set. fill_context
>>>> > /usr/lib/python2.7/site-packages/keystone/middleware/auth.py:188
>>>> > 2017-07-20 14:36:46.829 2621 INFO keystone.common.wsgi
>>>> > [req-9d64587c-8bda-401b-83df-a0c166ea629b - - - - -] POST
>>>> > http://10.11.184.50:5000/v3/auth/tokens
>>>> > 2017-07-20 14:36:46.848 2621 WARNING keystone.common.wsgi
>>>> > [req-9d64587c-8bda-401b-83df-a0c166ea629b - - - - -] An unexpected
>>>> error
>>>> > prevented the server from fulfilling your request.
>>>> >
>>>> > I can't seem to find any other interesting errors in keystone.log.
>>>> The above
>>>> > just repeats over and over for each service attempting to
>>>> authenticate.
>>>> >
>>>> > If I remove the "domain_specific_drivers_enabled" and
>>>> "domain_config_dir"
>>>> > options from keystone.conf (with my "[ldap]" configurations removed as
>>>> > well), I can then successfully authenticate using MySQL for identity.
>>>> >
>>>> > I'm at a total loss on what may be wrong, and confused as to why Heat
>>>> and
>>>> > Magnum need their own domains. Would anyone be able to help point me
>>>> in the
>>>> > right direction?
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > Mailing list: http://lists.openstack.org/cgi
>>>> -bin/mailman/listinfo/openstack
>>>> > Post to     : openstack at lists.openstack.org
>>>> > Unsubscribe : http://lists.openstack.org/cgi
>>>> -bin/mailman/listinfo/openstack
>>>> >
>>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20170726/c30a66cf/attachment.html>


More information about the Openstack mailing list