Open Stack

Hi everyone,

We're currently working on an improved mechanism for plumbing file access 
into a VM or container and in most cases some interaction and 
configuration on the hypervisor/host is required to make it happen.  The 
goal is to create something that generalizes beyond the current NFS and 
CIFS-only assumptions that are baked into the current crop of Manila 
drivers.

The current target is to use the new VSOCK zero-configuration sockets, 
which I will be talking about at the summit on Wednesday:

 - mount distributed fs on host, export via knfsd to guest over VSOCK 
 - run Ganesha with distributed fs FSAL on host, export to guest over VSOCK

(In short, VSOCK requires no configuration on the guest--so we can 
continue to treat it as a black box--and only a simple network id 
assignment on the host.  This reduces driver complexity, reduces the 
potential for user error, and improves security.)

Or, the same can be done using a more configuration-intensive private 
network:

 - mount distributed fs on host, export via knfsd to guest over private 
   host/guest network
 - run ganesha with distributed fs FSAL on host, export to guest over 
   private host/guest network

We are also interested in container use-cases (e.g., for Nova-managed 
containers using lxc or nova-docker):

 - mount distributed fs on host, bind mount a volume/directory into the 
guest/container namespace

>From a plumbing perspective, these approaches appear to be the most 
attractive as far as efficiency and security.  When it comes to making 
openstack actually orchestrate it, however, things get complicated (and 
as a developer in a peripheral project I don't have the clearest view of 
how things should work).

The problem is that in any of these scenarios, the default/naive strategy 
of having Manila go poking around on the host machine is very fragile:

 - What if the host is down when the Manila configuration is made?
 - What if the VM is migrated to another host?
 - What if Manila doesn't understand what (the) Nova (driver) is doing?

For block volumes, there is a simple separation of roles: Cinder manages 
the volumes themselves, and Nova attaches them to guests.  This means 
there is some volume-related code in Nova, but it also means that that 
code is running in a context where it can Do The Right Thing.  I believe 
the same is true of Manila file volumes as well... as soon as 
you look beyond the current assumption that all volumes must be 
reached via a network file protocol (and usually a Neutron 
network).

That is, we would love to see a new set of Nova APIs to attach and detach 
a manila fs to the guest, along with a "get share info" call that returns 
any details required for the user/tenant to do the final mount within the 
guest.

This enables our new topologies:

 - Attach API might mount CephFS on the host and exportfs it over VSOCK to 
the guest VM.  Get-info API would direct the user to mount -t nfs vsock:1/ 
with -o nfsvers=4.1,proto=vsock,clientaddr=N ..
 - Attach API might configure/start a Ganesha instance doing the same
 - Attach API might mount CephFS on the host and then mount --bind the 
share to $guestroot/dev/manila/$shareid.  Get-info API would direct the 
user to mount --bind /dev/manila/$shareid.

I suspect it would also clean up some existing Manila topologies (though 
I'm not sure).  For example, the current the NFS-based Manila drivers will 
help you configure a Neutron net that can access the storage server.  The 
attach API might attach the guest to that same network.  (Maybe.. I'm 
super fuzzy on how this stuff works!)

What do people (especially the Nova folks) think about this?  I would love 
sit down with anyone to talk about this this week in Tokyo on either/both 
the Nova and Manila side of the picture.

Thanks!
sage