[Nova] - OS_TYPE - Setting after the fact.
Hello List, I have the following situation and would be thankful for any help or suggestions. We provide images to our customers, upon which we set the os_type metadata. We use this to help schedule instances to host aggregates, so we can control where certain OS's land. This works great for new instances, that are created from our images. We have the following problems though. 1. Instances created prior to the introduction of the os_type metadata existing on the image, this metadata is not flowed down, and as such these instances have NULL 2. Instances created from a snapshot do not seem to get the os_type flowed down 3. Instances imported using our migration tool do not end up with the os_type being set either. Currently the only way I can see to set os_type on an instance is to manually (yuk) update the database for each and every instance, which is missing it. Does anyone have any ideas how this can be updated without manually modifying the database. My second thought was to maybe make use of the instance metadata that you can set via the CLI or API, but then I run in to the issue that there is not a nova filter that is able to use instance metadata, and in my brief play with the code, it looks like (user) metadata is not passed in as part of the instance spec dict that is used to schedule instances. I was thinking of writing a custom filter, but I'm not sure how compliant it would be to have a function making a call to try and collect the instance metadata. In summary, I need to solve two things. 1. How do I set os_type on instances in a way that doesn't involve editing the database. 2. Or How can I use another piece of metadata that I can set via the api, which also is exposed to the filter scheduler. Rgds Steve. The future has already arrived. It's just not evenly distributed yet - William Gibson
Hello List,
I have the following situation and would be thankful for any help or suggestions.
We provide images to our customers, upon which we set the os_type metadata. We use this to help schedule instances to host aggregates, so we can control where certain OS's land. This works great for new instances, that are created from our images.
We have the following problems though.
1. Instances created prior to the introduction of the os_type metadata existing on the image, this metadata is not flowed down, and as such these instances have NULL This is the expected behaivor. we snapshot/copy the image metadata at the time the instnace was created into the instance_system_metadata table to ensure the change to the image after the fact do not affect existing vms. 2. Instances created from a snapshot do not seem to get the os_type flowed down 3. Instances imported using our migration tool do not end up with the os_type being set either.
On Mon, 2021-11-15 at 10:21 +0000, Steven Relf wrote: likely because the meataddata was not set on the image or volume before the vm was created.
Currently the only way I can see to set os_type on an instance is to manually (yuk) update the database for each and every instance, which is missing it.
os_type on the instance is not used anymore and should not be set at all https://github.com/openstack/nova/blob/master/nova/objects/instance.py#L170 the os_type filed in the instance tabel was replaced with the os_type filed in the image_metatdata https://github.com/openstack/nova/blob/master/nova/objects/image_meta.py#L55... which is stored in teh instance_system_metadata table in the db.
Does anyone have any ideas how this can be updated without manually modifying the database.
My second thought was to maybe make use of the instance metadata that you can set via the CLI or API, but then I run in to the issue that there is not a nova filter that is able to use instance metadata, and in my brief play with the code, it looks like (user) metadata is not passed in as part of the instance spec dict that is used to schedule instances.
we do have nova manage command that allow operators to set some fo the image metadata that coudl be extended to allow any image metadta to be set however the operator would then be resposible for ensureing that the chagne to the metadata do not invalide the placment of the current vm or rectifying it if it would. The only way i see to do this via the api would be a resize to same flavor which would update the flavor and image metadta and find a new host to move the vm to that is vaild for the new requiremetns. we had discussed added a new recreate api for this usecase previously but the feedback was to just extend resize.
I was thinking of writing a custom filter, but I'm not sure how compliant it would be to have a function making a call to try and collect the instance metadata.
In summary, I need to solve two things.
1. How do I set os_type on instances in a way that doesn't involve editing the database.
2. Or How can I use another piece of metadata that I can set via the api, which also is exposed to the filter scheduler. you wont be able to use any metadata in the flavor or image since both are cahced at instance create. you might be able to use server metadata or a server tag. there is no intree filetr however that would work. in tree filters are not allwo to make api calls or RPC calls and should avoid making db queires in the filter.
the only way to do this today would be a rebuild, a nova manage command would be inline with what we have done for machine_type. allowing the image metadata to be change via an api is likely not approriate unless its a new instance action like recreate which would use the updated image metadta and move the instance(optionally to the same host). https://github.com/openstack/nova/commit/c70cde057d20bb2c05a133e52b9ec560bd7... for now i think that is the best approch to take. ensure that all image have the correct os_type set perhaps using a glance import plugin to also update update user submitted images then via an sql script or a new nova manage command update all existing images. the request spec does not currently contaien any instnace tag or server metadata https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L... as such there is no efficnet way to implement this as a custom filter.
Rgds Steve.
The future has already arrived. It's just not evenly distributed yet - William Gibson
Hey Sean, Thanks for responding. It sounds like a nova-manage command is the way forward, to avoid editing the DB directly, the operator would then need to ensure no breaches of aggregate requirements. Ill have a look at the nova-mange command you have referenced. We do now have the os_type set on all our provided images, but this doesn’t stop a customer uploading an image without it set. I guess that’s where a glance image plugin would come in, can you point me at some documentation for me to have a read around. I don’t like the idea of having to resize, as these aren’t our instances, as we are operating a public cloud. On an aside, out of curiosity, why was the decision to not cascade these types of changes made, is it simply to provide immutability to instances? Rgds Steve.
On Mon, 2021-11-15 at 15:35 +0000, Steven Relf wrote:
Hey Sean,
Thanks for responding.
It sounds like a nova-manage command is the way forward, to avoid editing the DB directly, the operator would then need to ensure no breaches of aggregate requirements. Ill have a look at the nova-mange command you have referenced.
We do now have the os_type set on all our provided images, but this doesn’t stop a customer uploading an image without it set. I guess that’s where a glance image plugin would come in, can you point me at some documentation for me to have a read around.
I don’t like the idea of having to resize, as these aren’t our instances, as we are operating a public cloud. hi yes so there is an image property inject plugin but that is mainly to inject static metadata https://docs.openstack.org/glance/latest/admin/interoperable-image-import.ht... you would porably need to modify that to use libguestfs to inspect the image and then add the os_type based on what it finds.
On an aside, out of curiosity, why was the decision to not cascade these types of changes made, is it simply to provide immutability to instances?
immutablity is one reason and maintaining the validity fo its current placment. adding cpu pinnign extra spec for example would invalidate the placment of any vm that did not have it set also via the flavor. you could add traits requests as another exampel that would make some vm invlaide for there current host. so in generall modifying the image extra specs on a runing instnace can invalidate the current placment of the vm due to chagne in the behavior of fliters or alter the guest abi which might break worklaods. that is why we said this would have to be an operation that you oppted into (proppagation of update extra specs) rahter then an operation that happened by default. it become espically problematic for image with shared/ public or comunity visiblity as a minor chagne to adress a supprot request from one customer might impact other customers. so to be safe we making it immuntable for the lifetime of the the instance unless you change the image via a rebuild in the case of image properties or resize in the case of flavor extra specs.
Rgds Steve.
participants (2)
-
Sean Mooney
-
Steven Relf