Open Stack

Hello,

I am setting up an OpenStack lab, and I am hoping someone can help me 
with a concept I'm having trouble grasping. I have a simple setup with a 
controller running keystone, glance, volume, dashboard etc. A single 
compute node, running compute and network and I have a nexenta SAN 
server for iSCSI volumes. I am using libvirt_type=kvm.

My setup is working fine, I can start instances, create iSCSI volumes 
and attach them to instances. What I don't understand is the logic 
behind the compute instances_path. When I start a new instance a root 
and ephemeral disk is created, but on the local disk of the compute 
node, set to instances_path=/var/lib/nova/instances. This seems to be by 
design, but I don't understand why this would be if I could utilize my 
san server for the root disk. If I am correct in understanding this, the 
root disk is persistent and the ephemeral is not, so any changes made to 
ephemeral disk are expected to be lost at some point. However in the 
event that my physical compute node were to crash and the instance 
started on another compute node any changes to the root disk are lost 
until I can recover the crashed node.

I can see that I have an option to boot from a volume, but the volume 
must be first manually created and then, from what I understand, I have 
to populate the volume myself using dd or something similar. Can this 
not be done using the glance image store like it does when it uses the 
local instance path?

In my ideal setup the compute nodes would have very minimal local block 
devices (maybe diskless) and get almost all of their storage via iSCSI 
LUNs. Is the only way around this to manually setup an iSCSI LUN for 
each compute node and mount it at, instances_path=/path/to/iscsi/mount? 
When I did some googleing about this I fond a post suggesting that one 
could use an NFS share for the instances path, I would prefer not to use 
NFS if I can avoid it.

How do other people manage similar setups, do most people just use 
local disks for the instance path? I feel like I must be missing 
something in my thinking. Does any one do as I suggested above or am I 
thinking about this all wrong?


Many thanks for your time and any help!

Kind Regards
Jake