[openstack-dev] [all][keystone] Increase of USER_ID length maximum from 64 to 255
Jay Pipes
jaypipes at gmail.com
Fri Feb 28 01:20:43 UTC 2014
On Thu, 2014-02-27 at 17:02 -0500, Adam Young wrote:
> On 02/27/2014 12:52 PM, Jay Pipes wrote:
> > On Thu, 2014-02-27 at 16:13 +0000, Henry Nash wrote:
> >> So a couple of things about this:
> >>
> >>
> >> 1) Today (and also true for Grizzly and Havana), the user can chose
> >> what LDAP attribute should be returned as the user or group ID. So it
> >> is NOT a safe assumption today (ignoring any support for
> >> domain-specific LDAP support) that the format of a user or group ID is
> >> a 32 char UUID. Quite often, I would think, that email address would
> >> be chosen by a cloud provider as the LDAP id field, by default we use
> >> the CN. Since we really don't want to ever change the user or group
> >> ID we have given out from keystone for a particular entity, this means
> >> we need to update nova (or anything else) that has made a 32 char
> >> assumption.
> > I don't believe this is correct. Keystone is the service that deals with
> > authentication. As such, Keystone should be the one and only one service
> > that should have any need whatsoever to need to understand a non-UUID
> > value for a user ID. The only value that should ever be communicated
> > *from* Keystone should be the UUID value of the user.
>
> We never said it was a UUID, its only been that in the "allocated by
> Keystone" cases thus far.
Every user record in Keystone should have an "allocated by Keystone"
user ID that is a UUID value. Why? Because every single other OpenStack
service uses UUID values for identifiers (not names), and Keystone
should be no different.
> userids are blobs....well, character blocks, and are not expected to be
> interprested by the end services, but also not confined to be 32
> characters long.
I would argue that they absolutely *should* be confined to 32 characters
(or technically, 16 bytes, which is the size of a UUID).
> Most LDAP use cases do not allow us to write data back the Directory.
> Federation has the same use cases.
Frankly, I don't care :) Let's say Super-Duper Identity Driver has 100
character IDs natively. Does that mean that these IDs need be sent all
over the OpenStack ecosystem? No. Have a translation table in Keystone
and spread the UUID user ID throughout OpenStack.
> That means that we are going to end
> up with a slew of ephemeral data in Keystone for no real value.
Not sure why you think this is ephemeral data... What is ephemeral about
a non-UUID identifier from another auth system (federated or not) that
is mapped to a Keystone user UUID? If you look at virtually every
flexible authentication mechanism in virtually all web frameworks out
there, you will see something very similar, where there is a table that
maps external IDs to the IDs that are native to some platform.
Best,
-jay
> >
> > If the Keystone service uses LDAP or federation for alternative
> > authentication schemes, then Keystone should have a mapping table that
> > translates those elongated and non-UUID identifiers values (email
> > addresses, LDAP CNs, etc) into the UUID value that is then communicated
> > to all other OpenStack services.
> The short of it is that we need to define a max userid length and stick
> to it. Thus far, we've had 64 characters in Keystone, and I think we
> can stick with that. Nova needs to expand its view to 64 chars.
>
> The rest is Keystone internals to deal with.
>
> >
> > Best,
> > -jay
> >
> >> 2) In oder to support the ability for service providers to be able to
> >> have the identity part of keystone be satisfied by a customer LDAP
> >> (i.e. for a given domain, have a specific LDAP), then, as has been
> >> stated, we need to subsequently, when handed an API call with just a
> >> user or group ID, be able to "route" this call to the correct LDAP.
> >> Trying to keep true to the openstack design principles, we had
> >> planned to encode a domain identifier into the user or group ID - i.e.
> >> distribute the data to where it is needed, in ortherwords, the user
> >> and group ID provide all the info we need to route the call to the
> >> right place. Two implementations come to mind:
> >> 2a) Simply concatenate the user/group ID with the domain_id, plus some
> >> separator and make a composite public facing ID. e.g.
> >> "user_entity_id@@UUID_of_domain". This would have a technical maximum
> >> size of 64+2+64 (i.e. 130), although in reality since we control
> >> domain_id and we know it is always 32 char UUID - in fact the max size
> >> would be 98. This has the problem of increasing the size of the
> >> public facing field beyond the existing 64. This is what we had
> >> planned for IceHouse - and is currently in review.
> >> 2b) Use a similar concatenation idea as 2a), but limit the total size
> >> to the existing 64. Since we control domain_id, we could (internally
> >> and not visibly to the outside world), create a domain_index, that was
> >> used in place of domain_id in the publicly visible field, to minimize
> >> the number of chars it requires. So the public facing composite ID
> >> might be something like <up to 54 chars of entity_id>@@<8 chars of
> >> domain_index>. There is a chance, of course, that the 54 char
> >> restriction might be problematic for LDAP users....but I doubt it. We
> >> would make that a restriction and if it really became a problem, we
> >> could consider a field size increase at a later release
> >> 3) The alternative to 2a and 2b is to have, as had been suggested, an
> >> internal mapping table that maps external facing entity_ids to a
> >> domain plus local entity ID. The problem with this idea is that:
> >> - This could become a very big table (you will essentially have an
> >> entry for every user in every corporate LDAP that has accessed a given
> >> openstack)
> >> - Since most LDAPs are RO, we will never see deletes...so we won't
> >> know when (without some kind of garbage collection) to cull entries
> >> - It obviously does not solve 1) - since existing LDAP support can
> >> break the 32 char limit - and so it isn't true that this mapping table
> >> causes all public facing entity IDs to be simple 32 char UUIDs
> >>
> >>
> >> From a delivery into IceHouse point of view any of the above are
> >> possible, since the actual mapping used is relatively small part of
> >> the patch. I personally favor 2b), since it is simple, has "less
> >> moving parts" and does not change any external facing requirements for
> >> storage of user and group IDs (above and beyond what is true today).
> >>
> >>
> >> Henry
> >> On 27 Feb 2014, at 03:46, Adam Young <ayoung at redhat.com> wrote:
> >>
> >>> On 02/26/2014 08:25 AM, Dolph Mathews wrote:
> >>>
> >>>> On Tue, Feb 25, 2014 at 2:38 PM, Jay Pipes <jaypipes at gmail.com>
> >>>> wrote:
> >>>> On Tue, 2014-02-25 at 11:47 -0800, Morgan Fainberg wrote:
> >>>> > For purposes of supporting multiple backends for
> >>>> Identity (multiple
> >>>> > LDAP, mix of LDAP and SQL, federation, etc) Keystone is
> >>>> planning to
> >>>> > increase the maximum size of the USER_ID field from an
> >>>> upper limit of
> >>>> > 64 to an upper limit of 255. This change would not
> >>>> impact any
> >>>> > currently assigned USER_IDs (they would remain in the
> >>>> old simple UUID
> >>>> > format), however, new USER_IDs would be increased to
> >>>> include the IDP
> >>>> > identifier (e.g. USER_ID@@IDP_IDENTIFIER).
> >>>>
> >>>>
> >>>> -1
> >>>>
> >>>> I think a better solution would be to have a simple
> >>>> translation table
> >>>> only in Keystone that would store this longer identifier
> >>>> (for folks
> >>>> using federation and/or LDAP) along with the Keystone user
> >>>> UUID that is
> >>>> used in foreign key relations and other mapping tables
> >>>> through Keystone
> >>>> and other projects.
> >>>>
> >>>>
> >>>> Morgan and I talked this suggestion through last night and agreed
> >>>> it's probably the best approach, and has the benefit of zero
> >>>> impact on other services, which is something we're obviously
> >>>> trying to avoid. I imagine it could be as simple as a user_id to
> >>>> domain_id lookup table. All we really care about is "given a
> >>>> globally unique user ID, which identity backend is the user from?"
> >>>>
> >>>>
> >>>> On the downside, it would likely become bloated with unused
> >>>> ephemeral user IDs, so we'll need enough metadata about the
> >>>> mapping to implement a purging behavior down the line.
> >>> UUIDs are 32 chars long. Its really just uuid@@uuid that pushes us
> >>> over the 64 character limit.
> >>> If we can shorten up the IDP_ID we can fit everything in 64 chars
> >>> (which means only Nova needs to expand its column size)
> >>>
> >>> What if we enumerated IDPs by index, from 10000000 to 99999999 or
> >>> something comparable, and then use the new domain_index (or prot
> >>> domain id to not be a uuid). Then the above scheme would work and
> >>> no migration would be required.
> >>>
> >>>
> >>>>
> >>>>
> >>>> The only identifiers that would ever be communicated to
> >>>> any non-Keystone
> >>>> OpenStack endpoint would be the UUID user and tenant IDs.
> >>>>
> >>>> > There is the obvious concern that projects are utilizing
> >>>> (and storing)
> >>>> > the user_id in a field that cannot accommodate the
> >>>> increased upper
> >>>> > limit. Before this change is merged in, it is important
> >>>> for the
> >>>> > Keystone team to understand if there are any places that
> >>>> would be
> >>>> > overflowed by the increased size.
> >>>>
> >>>>
> >>>> I would go so far as to say the user_id and tenant_id
> >>>> fields should be
> >>>> *reduced* in size to a fixed 16-char BINARY or 32-char
> >>>> CHAR field for
> >>>> performance reasons. Lengthening commonly-used and
> >>>> frequently-joined
> >>>> identifier fields is not a good option, IMO.
> >>>>
> >>>> Best,
> >>>> -jay
> >>>>
> >>>> > The review that would implement this change in size
> >>>> > is https://review.openstack.org/#/c/74214 and is
> >>>> actively being worked
> >>>> > on/reviewed.
> >>>> >
> >>>> >
> >>>> > I have already spoken with the Nova team, and a single
> >>>> instance has
> >>>> > been identified that would require a migration (that
> >>>> will have a fix
> >>>> > proposed for the I3 timeline).
> >>>> >
> >>>> >
> >>>> > If there are any other known locations that would have
> >>>> issues with an
> >>>> > increased USER_ID size, or any concerns with this change
> >>>> to USER_ID
> >>>> > format, please respond so that the issues/concerns can
> >>>> be addressed.
> >>>> > Again, the plan is not to change current USER_IDs but
> >>>> that new ones
> >>>> > could be up to 255 characters in length.
> >>>> >
> >>>> >
> >>>> > Cheers,
> >>>> > Morgan Fainberg
> >>>> > —
> >>>> > Morgan Fainberg
> >>>> > Principal Software Engineer
> >>>> > Core Developer, Keystone
> >>>> > m at metacloud.com
> >>>> >
> >>>> >
> >>>>
> >>>> > _______________________________________________
> >>>> > OpenStack-dev mailing list
> >>>> > OpenStack-dev at lists.openstack.org
> >>>> >
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> OpenStack-dev mailing list
> >>>> OpenStack-dev at lists.openstack.org
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> OpenStack-dev mailing list
> >>>> OpenStack-dev at lists.openstack.org
> >>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>>
> >>> _______________________________________________
> >>> OpenStack-dev mailing list
> >>> OpenStack-dev at lists.openstack.org
> >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >> _______________________________________________
> >> OpenStack-dev mailing list
> >> OpenStack-dev at lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
More information about the OpenStack-dev
mailing list