[nova][glance][entropy][database] update glance metadata for nova instance
Hi Openstack Discuss,
I have an issue with nova not synchronizing changes between a glance image and it's local image meta information in nova.
I have updated a glance image with the property "hw_rng_model=virtio", and that successfully passes that to new instances created using the updated image. However existing instances do not receive this new property.
I have located the image metadata within the nova database, in the **instance_system_metadata** table, and can see it's not updated for the existing instances, and only adding the relevant rows for instances that are created when that property is present. The key being "image_hw_rng_model" and "virtio" being the value.
Is there a way to tell nova to update the table for existing instances, and synchronizing the two databases? Or is this the kind of thing that would need to be done *shudder* manually...? If so, are there any experts out there who can point me to some documentation on doing this correctly before I go butcher a couple of dummy nova database?
Regards, Jordan
On Mon, 2019-08-26 at 18:18 +1200, Jordan Ansell wrote:
Hi Openstack Discuss,
I have an issue with nova not synchronizing changes between a glance image and it's local image meta information in nova.
I have updated a glance image with the property "hw_rng_model=virtio", and that successfully passes that to new instances created using the updated image. However existing instances do not receive this new property.
I have located the image metadata within the nova database, in the **instance_system_metadata** table, and can see it's not updated for the existing instances, and only adding the relevant rows for instances that are created when that property is present. The key being "image_hw_rng_model" and "virtio" being the value.
Is there a way to tell nova to update the table for existing instances, and synchronizing the two databases? Or is this the kind of thing that would need to be done *shudder* manually...?
this is idealy not something you would do at all. nova create a local copy of the image metadata the instace was booted with intionally to not pick up chagne you make to the image metadata after you boot the instance. in some case those change could invalidate the host the image is on so it in general in not considerd safe to just sync them
for the random number generator it should be ok but if you were to add a trait requirement of alter the numa topology then it could invalidate the host as a candiate for that instance. so if you want to do this then you need to update it manually as nova is working as intended by not syncing the data.
If so, are there any experts out there who can point me to some documentation on doing this correctly before I go butcher a couple of dummy nova database?
there is no docs for doing this as it is not a supported feature. you are circumventing a safty feature we have in nova to prevent change to running instances after they are first booted by change to the flavor extra spec or image metadata.
Regards, Jordan
On 8/26/19 4:24 AM, Sean Mooney wrote:
On Mon, 2019-08-26 at 18:18 +1200, Jordan Ansell wrote:
Hi Openstack Discuss,
I have an issue with nova not synchronizing changes between a glance image and it's local image meta information in nova.
I have updated a glance image with the property "hw_rng_model=virtio", and that successfully passes that to new instances created using the updated image. However existing instances do not receive this new property.
I have located the image metadata within the nova database, in the **instance_system_metadata** table, and can see it's not updated for the existing instances, and only adding the relevant rows for instances that are created when that property is present. The key being "image_hw_rng_model" and "virtio" being the value.
Is there a way to tell nova to update the table for existing instances, and synchronizing the two databases? Or is this the kind of thing that would need to be done *shudder* manually...?
this is idealy not something you would do at all. nova create a local copy of the image metadata the instace was booted with intionally to not pick up chagne you make to the image metadata after you boot the instance. in some case those change could invalidate the host the image is on so it in general in not considerd safe to just sync them
for the random number generator it should be ok but if you were to add a trait requirement of alter the numa topology then it could invalidate the host as a candiate for that instance. so if you want to do this then you need to update it manually as nova is working as intended by not syncing the data.
If so, are there any experts out there who can point me to some documentation on doing this correctly before I go butcher a couple of dummy nova database?
there is no docs for doing this as it is not a supported feature. you are circumventing a safty feature we have in nova to prevent change to running instances after they are first booted by change to the flavor extra spec or image metadata.
Regards, Jordan
I agree with everything Sean says here. I just want to remind you that if you use the nova image-create action on an instance, the image properties put on the new image are pulled from the nova database. So if you do decide to update the DB manually (not that I am recommending that!), don't forget that any already existing snapshot images will have the "wrong" value for the property. (You can update them via the Images API.)
On 27/08/19 1:47 AM, Brian Rosmaita wrote:
On 8/26/19 4:24 AM, Sean Mooney wrote:
On Mon, 2019-08-26 at 18:18 +1200, Jordan Ansell wrote:
Hi Openstack Discuss,
I have an issue with nova not synchronizing changes between a glance image and it's local image meta information in nova.
I have updated a glance image with the property "hw_rng_model=virtio", and that successfully passes that to new instances created using the updated image. However existing instances do not receive this new property.
I have located the image metadata within the nova database, in the **instance_system_metadata** table, and can see it's not updated for the existing instances, and only adding the relevant rows for instances that are created when that property is present. The key being "image_hw_rng_model" and "virtio" being the value.
Is there a way to tell nova to update the table for existing instances, and synchronizing the two databases? Or is this the kind of thing that would need to be done *shudder* manually...?
this is idealy not something you would do at all. nova create a local copy of the image metadata the instace was booted with intionally to not pick up chagne you make to the image metadata after you boot the instance. in some case those change could invalidate the host the image is on so it in general in not considerd safe to just sync them
for the random number generator it should be ok but if you were to add a trait requirement of alter the numa topology then it could invalidate the host as a candiate for that instance. so if you want to do this then you need to update it manually as nova is working as intended by not syncing the data.
If so, are there any experts out there who can point me to some documentation on doing this correctly before I go butcher a couple of dummy nova database?
there is no docs for doing this as it is not a supported feature. you are circumventing a safty feature we have in nova to prevent change to running instances after they are first booted by change to the flavor extra spec or image metadata.
Regards, Jordan
I agree with everything Sean says here. I just want to remind you that if you use the nova image-create action on an instance, the image properties put on the new image are pulled from the nova database. So if you do decide to update the DB manually (not that I am recommending that!), don't forget that any already existing snapshot images will have the "wrong" value for the property. (You can update them via the Images API.)
Thanks Sean and Brian..!
I hadn't considered the snapshots.. that's a really good point! And thank you for the warnings, I can see why this isn't something that's synchronized automatically :S
Regards, Jordan
On 27/08/19 11:00 AM, Jordan Ansell wrote:
On 27/08/19 1:47 AM, Brian Rosmaita wrote:
On 8/26/19 4:24 AM, Sean Mooney wrote:
On Mon, 2019-08-26 at 18:18 +1200, Jordan Ansell wrote:
Hi Openstack Discuss,
I have an issue with nova not synchronizing changes between a glance image and it's local image meta information in nova.
I have updated a glance image with the property "hw_rng_model=virtio", and that successfully passes that to new instances created using the updated image. However existing instances do not receive this new property.
I have located the image metadata within the nova database, in the **instance_system_metadata** table, and can see it's not updated for the existing instances, and only adding the relevant rows for instances that are created when that property is present. The key being "image_hw_rng_model" and "virtio" being the value.
Is there a way to tell nova to update the table for existing instances, and synchronizing the two databases? Or is this the kind of thing that would need to be done *shudder* manually...?
this is idealy not something you would do at all. nova create a local copy of the image metadata the instace was booted with intionally to not pick up chagne you make to the image metadata after you boot the instance. in some case those change could invalidate the host the image is on so it in general in not considerd safe to just sync them
for the random number generator it should be ok but if you were to add a trait requirement of alter the numa topology then it could invalidate the host as a candiate for that instance. so if you want to do this then you need to update it manually as nova is working as intended by not syncing the data.
If so, are there any experts out there who can point me to some documentation on doing this correctly before I go butcher a couple of dummy nova database?
there is no docs for doing this as it is not a supported feature. you are circumventing a safty feature we have in nova to prevent change to running instances after they are first booted by change to the flavor extra spec or image metadata.
Regards, Jordan
I agree with everything Sean says here. I just want to remind you that if you use the nova image-create action on an instance, the image properties put on the new image are pulled from the nova database. So if you do decide to update the DB manually (not that I am recommending that!), don't forget that any already existing snapshot images will have the "wrong" value for the property. (You can update them via the Images API.)
Thanks Sean and Brian..!
I hadn't considered the snapshots.. that's a really good point! And thank you for the warnings, I can see why this isn't something that's synchronized automatically :S
Regards, Jordan
Hi all,
I wanted to share a follow-up to this with two points:
* We've found another way to "give" and existing instance entropy using the API following an update to flavor and image metadata. * The documentation on entropy rates **everywhere** seems to be incorrect and could do with some updating..
Instead of updating the nova database and re-scheduling an instance, one can create a snapshot, add the "hw_rng_model=virtio" property to the snapshot, then launch the instance from that image using a flavor with the entropy properties. And boom! We have a copy of an existing instance with the addition of entropy :). Not perfect, but potentially better than an unsupported and risky operation.
With regard to the flavor documentation, it's written in the libvirt documentation [1] that the unit of the period attribute is *milliseconds* not seconds. However all documentation I came across for the "hw_rng:rate_period" of a flavor says this is in *seconds*. I've submitted bugs on the docs.openstack.org site, however if you are in charge of some other documentation please update your info :) There's a big difference between 100 bytes every millisecond and 100 bytes every 1000 milliseconds..!
Regards, Jordan
[1] https://libvirt.org/formatdomain.html#elementsRng [2] https://bugs.launchpad.net/nova/+bug/1843541 [3] https://bugs.launchpad.net/nova/+bug/1843542
participants (3)
-
Brian Rosmaita
-
Jordan Ansell
-
Sean Mooney