[cinder][nova][glance][ironic] restricting image metadata values to 255 chars
Hello stackers,
It has come to our attention recently that nova, cinder, and glance treat image properties differently. This came up in a discussion on a patch to extend the allowable length of image metadata values in cinder [0].
Briefly:
- glance allows image property values to be unrestricted in the schema, but effectively limited to 65535 bytes by the database.
- cinder restricts the length of an image property value to 255 chars (by request schema) if you try to update the volume_image_properties on a volume, but if you create a volume from an image, cinder happily copies over all the image properties from glance, giving them an effective limit of 65535 bytes.
- nova mostly truncates image properties values to 255 chars (it's a bit more complicated, but we can consider that to be nova's behavior for the purposes of this discussion).
(We're not sure what ironic does, but they're in the subject line as a consumer of images and volumes.)
Note that we're talking about metadata *values* here; the metadata keys are already restricted by charset to basically US-ASCII and 255 length. Values are allowed to be unicode.
The point of this email is that it would be good for the services to be consistent about image metadata values. Thus, we propose that all services should restrict image metadata values to 255 unicode characters. (Which is sort of happening already, but not consistently.)
The further point of this email is to find out what the impact of this restriction would be on current users. So please respond if you have a use case that would be affected by this change, and could not be addressed by, for example, instead of having one enormous value for a single key, the metadata was broken into a number of key/value pairs that would satisfy the 255 char restriction.
Thanks for your attention to this!
On Thu, 30 Nov 2023 at 20:55, Jorge Merlino jorge.merlino@canonical.com wrote:
Hello stackers,
It has come to our attention recently that nova, cinder, and glance treat image properties differently. This came up in a discussion on a patch to extend the allowable length of image metadata values in cinder [0].
Briefly:
- glance allows image property values to be unrestricted in the
schema, but effectively limited to 65535 bytes by the database.
- cinder restricts the length of an image property value to 255 chars
(by request schema) if you try to update the volume_image_properties on a volume, but if you create a volume from an image, cinder happily copies over all the image properties from glance, giving them an effective limit of 65535 bytes.
- nova mostly truncates image properties values to 255 chars (it's a
bit more complicated, but we can consider that to be nova's behavior for the purposes of this discussion).
(We're not sure what ironic does, but they're in the subject line as a consumer of images and volumes.)
Note that we're talking about metadata *values* here; the metadata keys are already restricted by charset to basically US-ASCII and 255 length. Values are allowed to be unicode.
The point of this email is that it would be good for the services to be consistent about image metadata values. Thus, we propose that all services should restrict image metadata values to 255 unicode characters. (Which is sort of happening already, but not consistently.)
The further point of this email is to find out what the impact of this restriction would be on current users. So please respond if you have a use case that would be affected by this change, and could not be addressed by, for example, instead of having one enormous value for a single key, the metadata was broken into a number of key/value pairs that would satisfy the 255 char restriction.
Thanks for your attention to this!
Hi Jorge & all,
To be very frank here, this sounds like a nova problem and your approach to fix the issue in Cinder seemed to be sensible and generally accepted there.
The metadefs basic concepts are explained in [0], Compute defines subsection of that, all predefined metadefs can be found from [1]
The very arrogant and false claim from your patch reviews that Nova would define all image properties and that definition would limit it to 255 characters is, well, hilarious.
Even the field definitions in the code do not specify such max size for most of those properties Nova _expects_ it might see and knows how to use, like Sean pointed out, it just happens to be limited how much of that Nova is able to store in their own database due to the field restriction at their DB layer. Lots of them properties are things like booleans or enumerations of values for Nova and thus have practical limitations. There are also fields like ListOfStrings, no max length defined.
Restricting the image properties from all the other consumers because Nova can't handle storing the copies of them, and thus breaking the Images API v2 and any use cases that uses image properties outside of Nova's fairly narrow scope would just be unacceptable. Either Nova should overcome their internal limitations and figure out how to deal with the resources or at very least understand to ignore them properties they have no capacity or interest handling.
Good luck and thanks for your patience to fight the windmills trying to get a simple fix in, Erno "jokke" Kuvaja
[0] https://docs.openstack.org/glance/latest/user/metadefs-concepts.html [1] https://github.com/openstack/glance/tree/master/etc/metadefs
On 30/11/23 19:34, Erno Kuvaja wrote:
On Thu, 30 Nov 2023 at 20:55, Jorge Merlino <jorge.merlino@canonical.com mailto:jorge.merlino@canonical.com> wrote:
Hello stackers, It has come to our attention recently that nova, cinder, and glance treat image properties differently. This came up in a discussion on a patch to extend the allowable length of image metadata values in cinder [0]. Briefly: - glance allows image property values to be unrestricted in the schema, but effectively limited to 65535 bytes by the database. - cinder restricts the length of an image property value to 255 chars (by request schema) if you try to update the volume_image_properties on a volume, but if you create a volume from an image, cinder happily copies over all the image properties from glance, giving them an effective limit of 65535 bytes. - nova mostly truncates image properties values to 255 chars (it's a bit more complicated, but we can consider that to be nova's behavior for the purposes of this discussion). (We're not sure what ironic does, but they're in the subject line as a consumer of images and volumes.) Note that we're talking about metadata *values* here; the metadata keys are already restricted by charset to basically US-ASCII and 255 length. Values are allowed to be unicode. The point of this email is that it would be good for the services to be consistent about image metadata values. Thus, we propose that all services should restrict image metadata values to 255 unicode characters. (Which is sort of happening already, but not consistently.) The further point of this email is to find out what the impact of this restriction would be on current users. So please respond if you have a use case that would be affected by this change, and could not be addressed by, for example, instead of having one enormous value for a single key, the metadata was broken into a number of key/value pairs that would satisfy the 255 char restriction. Thanks for your attention to this! [0] https://review.opendev.org/c/openstack/cinder/+/868485 <https://review.opendev.org/c/openstack/cinder/+/868485>Hi Jorge & all,
To be very frank here, this sounds like a nova problem and your approach to fix the issue in Cinder seemed to be sensible and generally accepted there.
The metadefs basic concepts are explained in [0], Compute defines subsection of that, all predefined metadefs can be found from [1]
The very arrogant and false claim from your patch reviews that Nova would define all image properties and that definition would limit it to 255 characters is, well, hilarious.
Even the field definitions in the code do not specify such max size for most of those properties Nova _expects_ it might see and knows how to use, like Sean pointed out, it just happens to be limited how much of that Nova is able to store in their own database due to the field restriction at their DB layer. Lots of them properties are things like booleans or enumerations of values for Nova and thus have practical limitations. There are also fields like ListOfStrings, no max length defined.
Restricting the image properties from all the other consumers because Nova can't handle storing the copies of them, and thus breaking the Images API v2 and any use cases that uses image properties outside of Nova's fairly narrow scope would just be unacceptable. Either Nova should overcome their internal limitations and figure out how to deal with the resources or at very least understand to ignore them properties they have no capacity or interest handling.
Good luck and thanks for your patience to fight the windmills trying to get a simple fix in, Erno "jokke" Kuvaja
[0] https://docs.openstack.org/glance/latest/user/metadefs-concepts.html https://docs.openstack.org/glance/latest/user/metadefs-concepts.html [1] https://github.com/openstack/glance/tree/master/etc/metadefs https://github.com/openstack/glance/tree/master/etc/metadefs
I'll tend to agree with you, but I see some consistency problems from the user's point of view. For example, if they create a server from an image with long metadata values and then create an image (snapshot) from that server, the new image gets the same metadata parameters as the original image but truncated to 255 chars.
Thinking from the perspective of a Openstack user (that does not care what particular service implements what) it would be nice if there was some kind of global policy regarding this. As the shortest length supported right now is nova's, a simple (but maybe not the best?) solution would be to adapt to that length everywhere.
Regards Jorge
participants (2)
-
Erno Kuvaja
-
Jorge Merlino