Open Stack

Hi Andrea,

> Andrea Y Ma <ayma at ...> writes: 

> I'm trying to use devstack with manila and nfs-ganesha with glusterfs as 
> the backend 
> 
> Environment: I have glusterfs installed on two bare metal systems and 
> devstack using kilo installed on another bare metal system. 
>  
> I'm a little confused about how I should be using ganesha nfs with 
> glusterfs as the backend. 
> 
> Should nfs-ganesha be installed in: 
>     a) my glusterFS servers and update the manila.conf setting 
>          glusterfs_nfs_server_type= Ganesha
>     b) my devstack 
>     c) a service VM that I manually spawn? 

Very sorry about the late reply. We've been held up by our deliverables for the
Liberty release. Thanks to your questions it's been easy for us to figure out
the gaps in our documentation. We definitely plan to cover the misses for
Liberty. For now, I've answered your questions below.

The NFS-Ganesha server can be installed in a), b) or c). To use
Ganesha-GlusterFS with Manila, Manila's share service needs root access to the
Ganesha server and the GlusterFS server machines to perform privileged actions.
This can be done by enabling password-less ssh root login for the manila
service user to the Ganesha server and the GlusterFS server machines when both
these servers run in machines remote to the manila service.  And then, Manila
config options need to be set to identify the Ganesha and GlusterFS servers.

> 
> I see the following link: 
>         http://docs.openstack.org/developer/manila/devref/ganesha.html
>    which points to there being ganesha support built into manila. 
>    The only requirement is that I have nfs-ganesha 2.1 or later. 
>    This would seem to imply that I should have nfs-ganesha installed 
>    on my devstack instance where manila is running. However from the 
>    values in the wiki I'm not sure how to point the nanila.conf file
>    to use those parameters. Should I just add them to the glusterfs
>    section I have or do I make a new ganesha section and update a 
>    manila config setting?

You can add the ganesha config options to the glusterfs section. Here's an
example, 
##
## glusterfs-ganesha section
[glusterfs_ganesha1]
share_driver = manila.share.drivers.glusterfs.GlusterfsShareDriver
driver_handles_share_servers = False
## config option specifying the GlusterFS volume and GlusterFS server node
glusterfs_target = root at 172.16.53.14:/manila-nfs-volume
glusterfs_nfs_server_type = Ganesha
## Config option indicating the IP of the Ganesha server node
## Here Ganesha server and GlusterFS server are in the same machine.
glusterfs_ganesha_server_ip = 172.16.53.14
ganesha_service_name = nfs-ganesha
share_backend_name = GLUSTERFS_GANESHA
## End of Config file

It's not necessary for NFS-Ganesha to run in the GlusterFS server machine. It
can run on a machine that can connect to the GlusterFS server, Manila host,
and the Manila client's network. The manila-share service figures out Ganesha
server's location using the "glusterfs_ganesha_server_ip" option. The
glusterfs driver doc should've covered this option, but it does not. This needs
to be sorted out.

The ganesha library in Manila makes use of NFS-Ganesha's D-Bus service and
Systemd/Upstart service file. Check whether they're available in the Ganesha
server machine. A key yet easy-to-miss step to configure Ganesha for Manila
is to create a ganesha export directory with an empty file 'INDEX.conf'
within it.
This file needs to be included by the Ganesha's config file, done by adding the
following line to ganesha.conf,
%include <ganesha_export_dir>/INDEX.conf

The above step is covered in the ganesha doc.
  
> In the glusterFS driver I saw a ganesha reference in this link:
> http://docs.openstack.org/developer/manila/devref/glusterfs_driver.html
>     So this made me wonder if nfs-ganesha should be installed on the 
>     baremetals running glusterfs. I don't see a client/server nfs-ganesha
>     rpm so was thinking I need to install nfs-ganesha on both the 
>     devstack and the glusterfs servers. Please correct me if I am wrong. 
>     The instructions I found for nfs-ganesha was the following:
>     http://blog.gluster.org/2014/09/glusterfs-and-nfs-ganesha-integration/

The following two NFS-Ganesha rpms need to be installed, nfs-ganesha and
nfs-ganesha-fsal-gluster(for nfs-ganesha v2.1)/nfs-ganesha-gluster(for
nfs-ganesha v2.2)
I'm afraid that the gluster blog you mention here is dated. I've requested
the gluster community to update this doc.

> 
> 
> I was also looking at this video:
>     https://www.youtube.com/watch?v=1zUSFzbhThM 
>     #vBrownBag 2014 OpenStack Paris - Bill Owen, Csaba Henk – 
> Supporting NFS Ganesha in Manila)
>     In the video it shows a slide titled "Improved Ganesha-Based 
> Drivers with Service VM". I'm wondering how the serviceVM got generated. 
> When I tried just with the glusterFsShareDriver I did not notice a service
> VM get spawned. Perhaps I had an error in my setup for using the 
> glusterFSShareDriver so wanted to make sure.  Or perhaps I need to do some 
> 
> combination of the generic and glusterFsShareDrivers because I did see 
> with 
> the generic driver a service VM getting spawned.  Or am I supposed to 
> manually spawn a service VM and install nfs-ganesha on that newly 
> spawned service VM.  If so how do I tell manila about it?

The GlusterfsShaserDriver with Ganesha does not yet suppport handling of share
servers. The behaviour that you observe is expected. And the topic of the slide
in the Paris talk still remains a Blueprint that's yet to be implemented. 

> 
> Once I have it running then do I need to do something different for the 
> share protocol or is there a document on what the extra-specs should be? 

By default, Manila's DevStack plugin sets up manila.conf to allow NFS and CIFS
to be used as share protocols. It also creates a default share type with the
spec "driver_handles_share_servers" as True. "glusterfs" driver with Ganehsa
does not handle share servers so you'd need to create a share type with the
spec set as False. Here's how you can do it as an OpenStack admin,
$ manila type-create gluster_ganesha false

You can also add the following extra spec type,
$ manila type-key gluster_ganesha set share_backend_name=GLUSTERFS_GANESHA

And in the current master(Liberty in progress), not in Kilo, you'd need to add
the following step,
$ manila type-key gluster_ganesha set snapshot_support=False

You can now create a share using glusterfs driver,
$ manila create --share-type gluster_ganesha NFS 1


Once you've created a share, you'd like to know about the bug,
https://bugs.launchpad.net/manila/+bug/1493080
that'd be fixed in Liberty. Not sure if the fix would be backported to Kilo.

Thanks,

Ramana