Open Stack

On Thu, 2014-02-27 at 16:13 +0000, Henry Nash wrote:
> So a couple of things about this:
> 
> 
> 1) Today (and also true for Grizzly and Havana), the user can chose
> what LDAP attribute should be returned as the user or group ID.  So it
> is NOT a safe assumption today (ignoring any support for
> domain-specific LDAP support) that the format of a user or group ID is
> a 32 char UUID.  Quite often, I would think, that email address would
> be chosen by a cloud provider as the LDAP id field, by default we use
> the CN.  Since we really don't want to ever change the user or group
> ID we have given out from keystone for a particular entity, this means
> we need to update nova (or anything else) that has made a 32 char
> assumption.

I don't believe this is correct. Keystone is the service that deals with
authentication. As such, Keystone should be the one and only one service
that should have any need whatsoever to need to understand a non-UUID
value for a user ID. The only value that should ever be communicated
*from* Keystone should be the UUID value of the user.

If the Keystone service uses LDAP or federation for alternative
authentication schemes, then Keystone should have a mapping table that
translates those elongated and non-UUID identifiers values (email
addresses, LDAP CNs, etc) into the UUID value that is then communicated
to all other OpenStack services.

Best,
-jay

> 2) In oder to support the ability for service providers to be able to
> have the identity part of keystone be satisfied by a customer LDAP
> (i.e. for a given domain, have a specific LDAP), then, as has been
> stated, we need to subsequently, when handed an API call with just a
> user or group ID, be able to "route" this call to the correct LDAP.
>  Trying to keep true to the openstack design principles, we had
> planned to encode a domain identifier into the user or group ID - i.e.
> distribute the data to where it is needed, in ortherwords, the user
> and group ID provide all the info we need to route the call to the
> right place. Two implementations come to mind:
> 2a) Simply concatenate the user/group ID with the domain_id, plus some
> separator and make a composite public facing ID.  e.g.
> "user_entity_id@@UUID_of_domain".  This would have a technical maximum
> size of 64+2+64 (i.e. 130), although in reality since we control
> domain_id and we know it is always 32 char UUID - in fact the max size
> would be 98.  This has the problem of increasing the size of the
> public facing field beyond the existing 64.  This is what we had
> planned for IceHouse - and is currently in review.
> 2b) Use a similar concatenation idea as 2a), but limit the total size
> to the existing 64. Since we control domain_id, we could (internally
> and not visibly to the outside world), create a domain_index, that was
> used in place of domain_id in the publicly visible field, to minimize
> the number of chars it requires.  So the public facing composite ID
> might be something like <up to 54 chars of entity_id>@@<8 chars of
> domain_index>.  There is a chance, of course, that  the 54 char
> restriction might be problematic for LDAP users....but I doubt it.  We
> would make that a restriction and if it really became a problem, we
> could consider a field size increase at a later release
> 3) The alternative to 2a and 2b is to have, as had been suggested, an
> internal mapping table that maps external facing entity_ids to a
> domain plus local entity ID.  The problem with this idea is that:
> - This could become a very big table (you will essentially have an
> entry for every user in every corporate LDAP that has accessed a given
> openstack)
> - Since most LDAPs are RO, we will never see deletes...so we won't
> know when (without some kind of garbage collection) to cull entries
> - It obviously does not solve 1) - since existing LDAP support can
> break the 32 char limit - and so it isn't true that this mapping table
> causes all public facing entity IDs to be simple 32 char UUIDs
> 
> 
> From a delivery into IceHouse point of view any of the above are
> possible, since the actual mapping used is relatively small part of
> the patch.  I personally favor 2b), since it is simple, has "less
> moving parts" and does not change any external facing requirements for
> storage of user and group IDs (above and beyond what is true today).
> 
> 
> Henry
> On 27 Feb 2014, at 03:46, Adam Young <ayoung at redhat.com> wrote:
> 
> > On 02/26/2014 08:25 AM, Dolph Mathews wrote:
> > 
> > > 
> > > On Tue, Feb 25, 2014 at 2:38 PM, Jay Pipes <jaypipes at gmail.com>
> > > wrote:
> > >         On Tue, 2014-02-25 at 11:47 -0800, Morgan Fainberg wrote:
> > >         > For purposes of supporting multiple backends for
> > >         Identity (multiple
> > >         > LDAP, mix of LDAP and SQL, federation, etc) Keystone is
> > >         planning to
> > >         > increase the maximum size of the USER_ID field from an
> > >         upper limit of
> > >         > 64 to an upper limit of 255. This change would not
> > >         impact any
> > >         > currently assigned USER_IDs (they would remain in the
> > >         old simple UUID
> > >         > format), however, new USER_IDs would be increased to
> > >         include the IDP
> > >         > identifier (e.g. USER_ID@@IDP_IDENTIFIER).
> > >         
> > >         
> > >         -1
> > >         
> > >         I think a better solution would be to have a simple
> > >         translation table
> > >         only in Keystone that would store this longer identifier
> > >         (for folks
> > >         using federation and/or LDAP) along with the Keystone user
> > >         UUID that is
> > >         used in foreign key relations and other mapping tables
> > >         through Keystone
> > >         and other projects.
> > > 
> > > 
> > > Morgan and I talked this suggestion through last night and agreed
> > > it's probably the best approach, and has the benefit of zero
> > > impact on other services, which is something we're obviously
> > > trying to avoid. I imagine it could be as simple as a user_id to
> > > domain_id lookup table. All we really care about is "given a
> > > globally unique user ID, which identity backend is the user from?"
> > > 
> > > 
> > > On the downside, it would likely become bloated with unused
> > > ephemeral user IDs, so we'll need enough metadata about the
> > > mapping to implement a purging behavior down the line.
> > UUIDs are 32 chars long.  Its really just uuid@@uuid that pushes us
> > over the 64 character limit.  
> > If we can shorten up the IDP_ID we can fit everything in 64 chars
> > (which means only Nova needs to expand its column size)
> > 
> > What if we enumerated IDPs by index, from 10000000 to 99999999 or
> > something comparable, and then use the new domain_index (or prot
> > domain id to not be a uuid).  Then the above scheme would work and
> > no migration would be required.
> > 
> > 
> > >  
> > >         
> > >         The only identifiers that would ever be communicated to
> > >         any non-Keystone
> > >         OpenStack endpoint would be the UUID user and tenant IDs.
> > >         
> > >         > There is the obvious concern that projects are utilizing
> > >         (and storing)
> > >         > the user_id in a field that cannot accommodate the
> > >         increased upper
> > >         > limit. Before this change is merged in, it is important
> > >         for the
> > >         > Keystone team to understand if there are any places that
> > >         would be
> > >         > overflowed by the increased size.
> > >         
> > >         
> > >         I would go so far as to say the user_id and tenant_id
> > >         fields should be
> > >         *reduced* in size to a fixed 16-char BINARY or 32-char
> > >         CHAR field for
> > >         performance reasons. Lengthening commonly-used and
> > >         frequently-joined
> > >         identifier fields is not a good option, IMO.
> > >         
> > >         Best,
> > >         -jay
> > >         
> > >         > The review that would implement this change in size
> > >         > is https://review.openstack.org/#/c/74214 and is
> > >         actively being worked
> > >         > on/reviewed.
> > >         >
> > >         >
> > >         > I have already spoken with the Nova team, and a single
> > >         instance has
> > >         > been identified that would require a migration (that
> > >         will have a fix
> > >         > proposed for the I3 timeline).
> > >         >
> > >         >
> > >         > If there are any other known locations that would have
> > >         issues with an
> > >         > increased USER_ID size, or any concerns with this change
> > >         to USER_ID
> > >         > format, please respond so that the issues/concerns can
> > >         be addressed.
> > >         >  Again, the plan is not to change current USER_IDs but
> > >         that new ones
> > >         > could be up to 255 characters in length.
> > >         >
> > >         >
> > >         > Cheers,
> > >         > Morgan Fainberg
> > >         > —
> > >         > Morgan Fainberg
> > >         > Principal Software Engineer
> > >         > Core Developer, Keystone
> > >         > m at metacloud.com
> > >         >
> > >         >
> > >         
> > >         > _______________________________________________
> > >         > OpenStack-dev mailing list
> > >         > OpenStack-dev at lists.openstack.org
> > >         >
> > >         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > >         
> > >         
> > >         
> > >         _______________________________________________
> > >         OpenStack-dev mailing list
> > >         OpenStack-dev at lists.openstack.org
> > >         http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > > 
> > > 
> > > 
> > > 
> > > _______________________________________________
> > > OpenStack-dev mailing list
> > > OpenStack-dev at lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> > 
> > 
> > _______________________________________________
> > OpenStack-dev mailing list
> > OpenStack-dev at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev