Open Stack

Hi all,

I've a centos 6 rdo setup with
- controller node w/ everything (compute running but no instances),
- compute node (active),
- nfs servers for cinder backend
-- basically it started as all in one, then I added the big raid nfs
server and lotsa cores/ram compute server.

The guests are all running off bootable cinder volumes.

It's been running OK for a few weeks.

Now
- after reboot (kernel upgrade) the compute node failed to resume the
guests with "error: cannot open file '/var/lib/nova/mnt/ID': No such
file or directory",
- after restart of all nova services I was able to start one of the
guests but it came up with read-only / and "mount: cannot remount block
device /dev/vda1 read-write, is write-protected",
- one of the VMs wouldn't boot at all b/c grub is unable to mount root.

Looking at the /var/lib/nova/mnt/ on the compute node, one of the mounts
is owned by root:root and volumes in it are owned by qemu:qemu, mode
666. The other mount and volumes in it are owned by nobody:nobody, which
I'm guessing is the source of at least the "read-only fs" problem. The
difference is that mount1 is physically on the controller node while
mount2 is on a dedicated storage server.

So my questions are

1. Did anyone actually ever run havana/cinder with nfs backend and if so
is there any documentation on what the ownership, permissions, and nfs
export options should be for it to work?

What settings, if any, does nova.conf on the compute node need to get
mount things properly?

2. The guest that won't boot because grub is unable to mount root: does
that sound like cinder volume corruption? Or should I look someplace else?

TIA
-- 
Dimitri Maziuk
Programmer/sysadmin
BioMagResBank, UW-Madison -- http://www.bmrb.wisc.edu

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 255 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140213/b2d23d70/attachment.sig>