On 17/06/2025 16:14, Rodolfo Alonso Hernandez wrote:
Hello all:
I initially found this issue in Neutron [1] but it seems to be more of an OpenStack wide problem. In Neutron, when you try to create a tag with different case (upper, lower), the API fails: $ openstack port create --network private port_tags $ openstack port set --tag tag1 port_tags $ openstack port set --tag TAG1 port_tags # this command will fail with the following message: ConflictException: 409: Client Error for url: http://192.168.10.100/networking/v2.0/ports/21788aea-98b7-4606-bcf4-5f963ce6..., Failed to create a duplicate Tag: for attribute(s) ['tags.PRIMARY'] with value(s) 47-TAG1
That happens because in Neutron we don't sanitize the input "tags" and we consider that "tag1" is different to "Tag1", as recommended by [2]: "Tags are case sensitive".
ya... so the https://specs.openstack.org/openstack/api-wg/guidelines/tags.html#tags-restr... is not an authoritative source. the actual nova spec that defines the behavior is https://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/ta... it also said it should be case sensitive. however when it was implated while the python code assumes it will be case sensitive the coalition type of the db was never enforced. the actual API contract is encoded in our api ref https://docs.openstack.org/api-ref/compute/#server-tags-servers-tags and that does not specify if its cases sensitive or not. you are entirely correct that it should be case sensitive but that was never enforce at the db layer.
But checking the table definitions in OpenStack (for example [3][4][5]), all projects seem to have the same problem: we define the tuple (resource, tag) as the primary key. But if we use a case insensitive table collation (for example "utf8_general_ci" or "utf8mb3_general_ci"), the comparison between "tag1" and "TAG1" will match.
In the Neutron API, because we don't consider this scenario, the API raises the mentioned exception. In Nova, for example, the command succeeds but the new tag is not added.
So the question here is: * Do we change the API definition to make it case insensitive? That will need, most probably, to revisit all stored tags and convert them to lower case. It will also require sanitizing the API input and making all tags lower case.
we have a related bug for metadata keys on host aggregates https://bugs.launchpad.net/nova/+bug/1538011 i strongly push back on making that change if we were too we would likely need a new api micro-version on the nova side. i would much prefer to enforce utf8mb4_bin as the coalition type on all tables in nova where we have string keys. that also has upgrade impacts both on the db side and for those how incorrect depending on case insensitive behavior. the python code does a exact match internally, but in some deployment you would not have been able to create additional keys where they key only differs by case. so the summary of the behviaor is as follows, if the coalation type was case insentive the unique constratint did not allow you to have keys that only differ by case, if the coalation type was case sensitive you could. in both instances the python code did case sensitive comparison. so if we were to enfroce utf8mb4_bin you would gain the ablity to have keys that differ by case, that would allow addtaionl keys to be added with a differing case that was only sometimes supproted in the past.
* Do we enforce a charset and collation case sensitive for these specific tables?
no unfortuanly nova almost never does that and its problematic. where we have spcifed anyuthing it just "CHARACTER SET utf8" with no colation type https://github.com/openstack/nova/blob/64ca204c9cf497b0dcfff2d3a24b0dd795a57... https://github.com/openstack/nova/blob/64ca204c9cf497b0dcfff2d3a24b0dd795a57... although we have on rare ocation set utf8_bin https://github.com/openstack/nova/blob/64ca204c9cf497b0dcfff2d3a24b0dd795a57... that will either map to utf8mb3_bin or utf8mb4_bin depending on the db version. the server tag are just specifed as utf8 as the charater set and like all the other tabels colaation is undefiend meaning it will use the db servers config. https://github.com/openstack/nova/blob/64ca204c9cf497b0dcfff2d3a24b0dd795a57... we also have some other unfortunate choices like using a signed 32bit int as a primary key in someplace where we really should be using a uint64, we have had at least one ci/development cloud hit that limit needing manual db surgery to reindex all the records. we talked about this when it happend in a ptg session a year or two ago but nova may need to revisit that dicussion. i think we should make some db schema changes to normalize all dbs to use specific coalition types and consier the data types we use for specific fiels sint32->uint64. the unfortunate relativity is for the most part we rely on the defautl db colation types which depends on the locale used when installing the server.
Regards.
[1]https://bugs.launchpad.net/neutron/+bug/2114819 [2]https://specs.openstack.org/openstack/api-wg/guidelines/tags.html#tags-restr... [3]https://github.com/openstack/neutron/blob/39b183645e071f9c2926f0bd870723fb6e... [4]https://github.com/openstack/nova/blob/64ca204c9cf497b0dcfff2d3a24b0dd795a57... [5]https://github.com/openstack/octavia/blob/2fb6ba9343e39f60f058a781afd14b906d...