> So what might a safer approach look like? My answer is to try and offload the processing of untrusted data to the virtual machine
> implementation in silicon -- it has strong separation guarantees and it can be orchestrated so that there is simply no access to
> anything not required for the strict processing of the image. So for example no access to a file system, a network, and so forth.
> Specifically, what if processing this untrusted and potentially malicious data was done as a KVM guest with a custom virtual
> machine manager (VMM)?
Isn't this basically libguestfs? I mean, perhaps we can't use libguestfs to actually do all the things we want, but it's basically the same model I think. Last I knew, we tried to avoid doing that as much as possible. Partially for overhead, partially for complexity, etc.
I did ponder libguestfs along the way. The big difference, at least to my understanding, is that libguestfs runs a linux kernel in its KVM guest with an associated small operating system, and then uses standard command line tools to service requests. Instar is more like a unikernel than libguestfs in the sense that instar _does_ not run a kernel as we think of it, and has no userspace. Instar is more like having the qemu-img operations performed on a service processor that just happens to be an x86 virtual core if that makes sense. I think that makes the security posture (and complexity) of the two quite different.
> The primary use case I'm targeting is validating untrusted images at upload time (e.g., in Glance), though the same approach could
> apply anywhere qemu-img is invoked on user-supplied data.
Glance already does this with format inspector, and in a way that doesn't store (or duplicate) all the data - it inspects it mid-stream, and for both upload and import in the same way. It also has tighter restrictions on things we want to disallow than even qemu-img does. It also does it for basically all the formats we support (like vhdx and vmdk) and even supports strict checking of things qemu-img doesn't support (like MBR/GPT). Nova also uses format inspector to double-check what it got from glance, and in some cases what is on-disk before it does stuff with it (like upload). Cinder does as well and I think ironic too?
If you discount moving pages around in memory, which I think all inspection tools would have to do, instar does not copy the input data prior to inspection.
> The current goal of Instar is to be command line compatible with qemu-img (especially `qemu-img info` and `qemu-img convert`),
> but I also wonder about opportunities to get out of the game of parsing command output. Whilst Instar can certainly emit JSON
> formatted output, there are other options as well like running it as a persistent daemon which services requests it receives
> over a unix domain socket. I've played that game with other code recently and find protobufs to be a reasonable protocol for such
> things. I don't know how OpenStack feels about protobufs however.
I haven't quite grokked if you're really planning a full replacement of qemu-img or a wrapping of it. However, qcow2 has evolved multiple times over the years in ways that we'd need to support for things like format conversions and so a full re-implementation would need to handle all of that, and with the same concerned-about-integrity level of QA that feels like would make that quite a job. The VMDK format is actually a group of formats all loosely related but quite different. Every time we've accidentally or intentionally broken some seemingly obscure VMDK thing, we hear about it from operators so it definitely has to work. And of course, we need LUKS support for the existing cinder and upcoming glance and nova support for that format.
Instar is a replacement, not a wrapper, noting that its also a prototype and therefore incomplete. That said, for the subcommands that are "done", I am unaware of an image or set of command line options that produces output that is different from qemu-img, and an example of such would be treated as a bug. You're right that qemu-img has changed its behaviour over time. To address this at build time, instar builds all qemu-img versions going back to 6.0 and builds a mapping of the various historical output formats. Then at run time instar attempts to detect the version of qemu-img installed on your machine and gives you that output format.
> At the moment I'd describe Instar as a "vigorous prototype". It implements `qemu-img info` and most of `qemu-img convert`, but
> does not yet know how to resize, rebase, snapshot, and so forth. I'd be interested in whether other people think this is an
> interesting idea or not, although honestly it was just kind of fun to build over the summer holidays. I'd particularly welcome
> feedback on whether this approach would be useful for your deployment, examples of images that produce unexpected results, and thoughts
> on the daemon / protobuf interface idea.
If your goal is to be fully command-line compatible and all we need to do is allow telling nova and glance what the binary is then that seems fine to me. Personally, I think we're in a much better spot than we once were in terms of inspecting malicious images. The recent nova redux was a place where we had missed applying the same inspection to one spot, but glance totally caught it before any data could be exfiltrated. If we want to further tighten down our actual use of qemu-img I think we have a lot of room to be able to just quarantine its execution away from all the dangerous bits.
That is my goal for now. Perhaps I'll encounter some way that's impossible. Again, I don't really mind if you guys use it or not -- I just think its interesting.
Michael