[openstack-dev] [cinder] No middle-man - when does/will Nova directly connect iSCSI volumes?
john.griffith8 at gmail.com
Fri Jun 24 14:05:38 UTC 2016
On Fri, Jun 24, 2016 at 2:19 AM, Daniel P. Berrange <berrange at redhat.com>
> On Thu, Jun 23, 2016 at 09:09:44AM -0700, Walter A. Boring IV wrote:
> > volumes connected to QEMU instances eventually become directly connected?
> > > Our long term goal is that 100% of all network storage will be
Oh, didn't know this at all. Is this something Nova has been working on
for a while? I'd love to hear more about the reasoning, the plan etc. It
would also be really neat to have an opportunity to participate.
> > > to directly by QEMU. We already have the ability to partially do this
> > > iSCSI, but it is lacking support for multipath. As & when that gap is
> > > addressed though, we'll stop using the host OS for any iSCSI stuff.
Any chance anybody has any insight on how to make this work? I tried
configuring this last week and it appears to be broken in a few places.
> > > So if you're requiring access to host iSCSI volumes, it'll work in the
> > > short-medium term, but in the medium-long term we're not going to use
> > > that so plan accordingly.
> > What is the benefit of this largely monolithic approach? It seems that
> > moving everything into QEMU is diametrically opposed to the unix model
> > itself and
> > is just a re-implementation of what already exists in the linux world
> > outside of QEMU.
> There are many benefits to having it inside QEMU. First it gives us
> improved isolation between VMs, because we can control the network
> I/O directly against the VM using cgroup resource controls.
> us improved security, particularly in combination with LUKS encryption
> since the unencrypted block device is not directly visible / accessible
> to any other process. It gives us improved reliability / managability
> since we avoid having to spawn the iscsi client tools which have poor
> error reporting and have been frequent sources of instability in our
True, the iscsi tools aren't the greatest.
> infrastructure (eg see how we have to blindly re-run the same command
> many times over because it randomly times out). It will give us improved
> I/O performance because of a shorter I/O path to get requests from QEMU
> out to the network.
I'd love to hear more on the design and how it all comes together.
Particularly the performance info. Like I said, I tried to set it up
against master but seems I'm either missing something in the config or it's
> NB, this is not just about iSCSI, the same is all true for RBD where
> we've also stopped using in-kernel RBD client and do it all in QEMU.
> > Does QEMU support hardware initiators? iSER?
> No, this is only for case where you're doing pure software based
> iSCSI client connections. If we're relying on local hardware that's
> a different story.
I'm confused, so what's the iser driver referenced in the patch commit
So there's a different story for that?
> > We regularly fix issues with iSCSI attaches in the release cycles of
> > OpenStack,
> > because it's all done in python using existing linux packages. How often
> This is a great example of the benefit that in-QEMU client gives us. The
> Linux iSCSI client tools have proved very unreliable in use by OpenStack.
> This is a reflection of the very architectural approach. We have individual
> resources needed by distinct VMs, but we're having to manage them as a host
> wide resource and that's creating us unneccessary complexity and having a
> poor effect on our reliability overall.
> > are QEMU
> > releases done and upgraded on customer deployments vs. python packages
> > (os-brick)?
> We're removing the entire layer of instability by removing the need to
> deal with any command line tools, and thus greatly simplifying our
> setup on compute nodes. No matter what we might do in os-brick it'll
> never give us a simple or reliable system - we're just papering over
> the flaws by doing stuff like blindly re-trying iscsi commands upon
This all sounds like it could be a good direction to go in. I'd love to
see more info on the plan, how it works, and how to test it out a bit.
Didn't find a spec, any links, reviews or config info available?
Wish I would've caught this on ML or IRC or wherever, would've loved to
have participated a bit.
> |: http://berrange.com -o- http://www.flickr.com/photos/dberrange/
> |: http://libvirt.org -o- http://virt-manager.org
> |: http://autobuild.org -o- http://search.cpan.org/~danberr/
> |: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the OpenStack-dev