Open Stack

Hello,

I have created an OpenStack Havana environment and configured Nova to use
libgfapi. I'm running into an odd issue, though:

The cloud consists of five compute nodes. Four of them are also running
Gluster and host a Distributed Replicated volume called "volumes". All
Cinder services are running on the Cloud Controller. cinder-volume can
successfully mount the Gluster volume and create volumes with it.

nova-compute is able to boot from a volume and is successfully using
libgfapi (grep gluster /etc/libvirt/qemu/*.xml).

Except on the fifth compute node -- the one that is not running Gluster.

I have narrowed the issue down to the "libvirt-qemu" user on c05 not being
able to use qemu to connect to the Gluster service on any of the other four
compute nodes. "root" on c05 can. When libvirt-qemu tries, the command just
hangs.

On any of the other four compute nodes, as long as they connect to their
local Gluster service, everything works.

I have previously posted this to the gluster-users mailing list, but
haven't found a solution. There's a lot of supplemental details, so here's
the link to the thread:

http://supercolony.gluster.org/pipermail/gluster-users/2013-December/038302.html


In order to get libgfapi working on Ubuntu, I had to jump through a few
hoops. First, I used, what I believe is, an unofficial Gluster repo:

https://launchpad.net/~semiosis/+archive/ubuntu-glusterfs-3.4

Next, I had to recompile the qemu package that is in the Ubuntu Havana repo
to support libgfapi:

https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1224517

I haven't had time to try to replicate this issue on a RedHat-based
distribution. If anyone can confirm this issue does *not* exist on RedHat,
at least I know the issue is localized to Ubuntu and can focus attention
there.

Or if anyone knows what the exact cause and solution is, that'd be great,
too  :)


To get around this issue for now, I have dropped c05 from the cloud. On the
Cloud Controller, I have made an entry in /etc/hosts called "gluster" that
points to c01. On the four other compute nodes, the "gluster" entry in
/etc/hosts points to 127.0.0.1. This allows me to specify

gluster:/volumes

in /etc/cinder/shares.conf and everyone will connect to the right Gluster
service. I feel this is a dirty hack, though.

Thanks,
Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131218/cf9206c4/attachment.html>