On Mon, 2024-07-22 at 13:46 +0200, Dmitriy Rabotyagov wrote:
Oh, yes, for sure it won't be applicable as a CVE fix, but could be some day2 discussion for the path forward to eliminate all possible future vectors, which could possibly simplify logic.
Why I was thinking about linux namespaces specifically, is because all of these deployment toolings (maybe except sunbeam?) are used to (and still do) deal with OVS Neutron driver, which rely on network namespaces a lot. So, theoretically, they could be used to isolate qemu-img from both networking and generic filesystem access for purposes of image conversion.
so redhat is deploying cinder volume on openshift and it does not support user namespaces and user namespacs are experimental in k8s we may or many not have premeisoin to create network namesapces or chroots when hosted on k8s/openshift for cinder volume but it might be possible on the compute node side at least for out new insttall nova is deployed a podman container and we may be able to create nested namesapces btu i dont think openstack code in generall should do that. neutron is an exception ranther than a template to follow in my view but im not ruling out some addtiona form of lockdown via namespace or similar. i would prefer to take the approce fo havign a pure python implementation of the image inspectorts with support for all the formats we support and reject all that are not recognised. that would be simpler for us to lockdown then sandboxing qemu-img. we wont be reimpelmting convert and resize logic but we can remove the use of qemu-img for basic image inspection which will significantly reduce the attach surface. that is the current direction we are moving in.
But yeah. It really can be non-trivial, slow and not worth it after all. But mainly I was just wondering about thoughts on that topic overall and if this was ever raised at all.
пн, 22 июл. 2024 г. в 13:20, <smooney@redhat.com>:
On Sat, 2024-07-20 at 19:21 +0200, Dmitriy Rabotyagov wrote:
Aha, ok, thanks so much for clarifying, that makes way more sense now!
I was actually wondering if running such conversions inside some kind of namespaces (sandboxing) was raised as an option to isolate conversions from the rest of data and why it was rejected (if it was raised)?
it wasn't but it would not be backporatable if we were to do that. we cannot introduced any new depencices on podman or systemd ectra.
as such adding sandboxing would be a new feature which could only be done on master. it is something we can consider but we would have to ensure that any solution we did use would not break any deployemnt tool that deploy openstack inside our outside of a container so its less simple then you might think to do that.
kolla deploys opentack in docker. redhat's new installer deploy opesntack on a mixture of openshfit and podman on rhel
tripleo used podman
canonical sunbeam uses a mix of containers (lxd?) and snaps?
the point im trying to make is its very hard to create a sand box that will work in all deployment tools.
we cant assume we can use podman, docker, systemd nspwan or flatpaks https://github.com/containers/bubblewrap as using any of the above may break one of all of the deployment tools.
it may also cause issues with thing like apparmor and selinux.
so adding sandboxing would be a very non trivial effort.
that does not mean we cant explore that in the future but it would not be somethign we could do as part of the cve fix.
On Sat, Jul 20, 2024, 13:50 <smooney@redhat.com> wrote:
On Fri, 2024-07-19 at 21:44 +0000, Jeremy Stanley wrote:
On 2024-07-19 23:23:38 +0200 (+0200), Dmitriy Rabotyagov wrote:
Wait, I just started realizing the bugfix ..
So basically all previously uploaded images not in raw format are useless after upgrading so the safe version of cinder/nova? [...]
I'm no expert in Glance matters, but my understanding is that with OSSA-2024-001 fixes in place you can't lie about the kind of image you're uploading. If you upload QCOW2 you must upload it as QCOW2 type and not try to pretend that it's a raw image, or upload an ISO as ISO type and not try to claim that it's raw. If you want to upload a different image type than what the cloud wants you to upload, then Glance should still be able to perform image conversion for you as well.
Previously, you could pretend a QCOW2 was raw, and then rely on QEmu's image auto-detection to figure out what it really was, but that's dangerous because QEmu is not intended to be used with untrusted images and some specially-crafted images can be used by an attacker to get copies of arbitrary files off the server where its tools (even the qemu-img executable) are run. Glance now has an image inspection feature which doesn't rely on QEmu itself to determine whether an image is safe enough to hand off to QEmu tools or might be used to root your cloud.
yes the way we mitigated this CVE was by adding logic that allow use to validate the source fromat matches what it was specified as in glance.
so image conversion is still a thing that you can do nova for example still allow you to configre the comptue hosts to use raw or qcow images and you can then boot a vm form any format qemu can convert form provided you uploaded it to glance and declard the format correctly
we have ci converge ensuring you can boot form a amazon ami image the cirros uec images with nova configured to use qcow for the guest vm root disk.
all of that still works but if you upload a qcow as a raw image to glance then nova and cinder will say "no you said this was raw this is unsafe" and refues to proceed.
we are currently missing that for iso format but its on our todo list to build out more tempest coverage for image formats that are reasonable to use.
its still possible to have the old conversion behavior provided you dont lie about the format when uploading it to glance. we also do other check to introspect the format for unsafe features. for qcow backing files are an example of a generally unsafe feature.
qcow image used with openstack should be single file flat images with no refernce to backing files or other external files.