[openstack-dev] [nova] Do we still want to lowercase metadata keys?
Jay Pipes
jaypipes at gmail.com
Mon Aug 13 13:04:43 UTC 2018
On 08/13/2018 06:06 AM, Matthew Booth wrote:
> Thanks mriedem for answering my previous question, and also pointing
> out the related previous spec around just forcing all metadata to be
> lowercase:
>
> (Spec: approved in Newton) https://review.openstack.org/#/c/311529/
> (Online migration: not merged, abandoned)
> https://review.openstack.org/#/c/329737/
>
> There are other code patches, but the above is representative. What I
> had read was the original bug:
>
> https://bugs.launchpad.net/nova/+bug/1538011
>
> The tl;dr is that the default collation used by MySQL results in a bug
> when creating 2 metadata keys which differ only in case. The proposal
> was obviously to simply make all metadata keys lower case. However, as
> melwitt pointed out in the bug at the time that's a potentially user
> hostile change. After some lost IRC discussion it seems that folks
> believed at the time that to fix this properly would seriously
> compromise the performance of these queries. The agreed way forward
> was to allow existing keys to keep their case, but force new keys to
> be lower case (so I wonder how the above online migration came
> about?).
>
> Anyway, as Rajesh's patch shows, it's actually very easy just to fix
> the MySQL misconfiguration:
>
> https://review.openstack.org/#/c/504885/
>
> So my question is, given that the previous series remains potentially
> user hostile, the fix isn't as complex as previously believed, and it
> doesn't involve a performance penalty, are there any other reasons why
> we might want to resurrect it rather than just go with Rajesh's patch?
> Or should we ask Rajesh to expand his patch into a series covering
> other metadata?
Keep in mind this patch is only related to *aggregate* metadata, AFAICT.
Any patch series that tries to "fix" this issue needs to include all of
the following:
* input automatically lower-cased [1]
* inline (note: not online, inline) data migration inside the
InstanceMeta object's _from_db_object() method for existing
non-lowercased keys
* change the collation of the aggregate_metadata.key column (note: this
will require an entire rebuild of the table, since this column is part
of a unique constraint [3]
* online data migration for migrating non-lowercased keys to their
lowercased counterpars (essentially doing `UPDATE key = LOWER(key) WHERE
LOWER(key) != key` once the collation has been changed)
None of the above touches the API layer. I suppose some might argue that
the REST API should be microversion-bumped since the expected behaviour
of the API will change (data will be transparently changed in one
version of the API and not another). I don't personally think that's
something I would require a microversion for, but who knows what others
may say.
Best,
-jay
[1]
https://github.com/openstack/nova/blob/16f89fd093217d22530570e8277b561ea79f46ff/nova/objects/aggregate.py#L295
and
https://github.com/openstack/nova/blob/16f89fd093217d22530570e8277b561ea79f46ff/nova/objects/aggregate.py#L331
and
https://github.com/openstack/nova/blob/16f89fd093217d22530570e8277b561ea79f46ff/nova/objects/aggregate.py#L356
[2]
https://github.com/openstack/nova/blob/16f89fd093217d22530570e8277b561ea79f46ff/nova/objects/aggregate.py#L248
[3]
https://github.com/openstack/nova/blob/16f89fd093217d22530570e8277b561ea79f46ff/nova/db/sqlalchemy/api_models.py#L64
More information about the OpenStack-dev
mailing list