Open Stack

On Sat, 13 Sep 2014 06:48:19 -0400
Sean Dague <sean at dague.net> wrote:

> On 09/13/2014 02:28 AM, Kenichi Oomichi wrote:
> > 
> > Hi Chris,
> > 
> > Thanks for bring it up here,
> > 
> >> -----Original Message-----
> >> From: Chris St. Pierre [mailto:stpierre at metacloud.com]
> >> Sent: Saturday, September 13, 2014 2:53 AM
> >> To: openstack-dev at lists.openstack.org
> >> Subject: [openstack-dev] [nova] Expand resource name allowed
> >> characters
> >>
> >> We have proposed that the allowed characters for all resource
> >> names in Nova (flavors, aggregates, etc.) be expanded to all
> >> printable unicode characters and horizontal spaces:
> >> https://review.openstack.org/#/c/119741
> >>
> >> Currently, the only allowed characters in most resource names are
> >> alphanumeric, space, and [.-_].
> >>
> >>
> >> We have proposed this change for two principal reasons:
> >>
> >> 1. We have customers who have migrated data forward since Essex,
> >> when no restrictions were in place, and thus have characters in
> >> resource names that are disallowed in the current version of
> >> OpenStack. This is only likely to be useful to people migrating
> >> from Essex or earlier, since the current restrictions were added
> >> in Folsom.
> >>
> >> 2. It's pretty much always a bad idea to add unnecessary
> >> restrictions without a good reason. While we don't have an
> >> immediate need to use, for example, the ever-useful
> >> http://codepoints.net/U+1F4A9 in a flavor name, it's hard to come
> >> up with a reason people *shouldn't* be allowed to use it.
> >>
> >> That said, apparently people have had a need to not be allowed to
> >> use some characters, but it's not clear why:
> >> https://bugs.launchpad.net/nova/+bug/977187 So I guess if anyone
> >> knows any reason why these printable characters should not be
> >> joined in holy resource naming, speak now or forever hold your
> >> peace.
> > 
> > I also could not find the reason of current restriction on the bug
> > report, and I'd like to know it as the history.
> > On v2 API(not v2.1), each resource name contains the following
> > restriction for its name:
> > 
> >   Resource  | Length  | Pattern
> >  -----------+---------+----------------------------------
> >   aggregate | 1-255   | nothing
> >   backup    | nothing | nothing
> >   flavor    | 1-255   | '^[a-zA-Z0-9. _-]*[a-zA-Z0-9_-]+
> >             |         |   [a-zA-Z0-9. _-]*$'
> >   keypair   | 1-255   | '^[a-zA-Z0-9 _-]+$'
> >   server    | 1-255   | nothing
> >   cell      | 1-255   | don't contain "." and "!"
> > 
> > On v2.1 API, we have applied the same restriction rule[1] for whole
> > resource names for API consistency, so maybe we need to consider
> > this topic for whole names.
> > 
> > [1]:
> > https://github.com/openstack/nova/blob/master/nova/api/validation/parameter_types.py#L44
> 
> Honestly, I bet this had to do with how the database used to be set
> up.
> 

So it turns out that utf8 support in MySQL does not support UTF-8 4 byte
multibyte characters (only 1-3 bytes). For example if you do a create
image call with an image name to glance with a 4 byte multibyte
character in the name it will 500. I'd guess we do something
similar in places with the Nova API where we have inadequate input
validation. If you want 4 byte support you need to use utf8mb4 instead.

I don't know if postgresql has any restrictions (I don't think it
does) or if db2 does too. But I don't think we can/should make it a
complete free for all. It should at most be what most databases support.

I think its a big enough change that this late in the cycle we should
push it off to Kilo. It's always much easier to loosen input validation
than tighten it (or have to have an "oops" revert on an officially
released Nova). Perhaps some tests to verify that everything we allow
past the input validation checks we can actually store.


> That being said, i'm pro letting names be 'utf8'. The id fields that
> are strings (like flavor_id) I think we should keep constrained, as
> we do actually do joins on them. (And as jay said, the current utf8
> schema is actually highly inefficient and bloaty.)

Chris