Open Stack

> From: Warren Wang <warren at wangspeed.com>
> Sent: Wednesday, October 12, 2016 10:02 PM
> To: Adam Kijak
> Cc: Abel Lopez; openstack-operators
> Subject: Re: [Openstack-operators] [openstack-operators][ceph][nova] How do you handle Nova on Ceph?
> 
> If fault domain is a concern, you can always split the cloud up into 3 regions, each having a dedicate Ceph cluster. It isn't necessarily going to mean more hardware, just logical splits. This is kind of assuming that the network doesn't share the same fault domain though.

This is not an option because having Region1-1, Region1-2, ..., Region1-10 would not be very convenient for users.

> Alternatively, you can split the hardware for the Ceph boxes into multiple clusters, and use multi backend Cinder to talk to the same set of hypervisors to use multiple Ceph clusters. We're doing that to migrate from one Ceph cluster to another. You can even mount a volume from each cluster into a single instance.

Multiple Ceph clusters on Cinder is not a problem, I agree.
Unfortunately we use Ceph for Nova (disks of instances are on Ceph directly).

> Keep in mind that you don't really want to shrink a Ceph cluster too much. What's "too big"? You should keep growing so that the fault domains aren't too small (3 physical rack min), or you guarantee that the entire cluster stops if you lose network.
> Just my 2 cents,

Thanks!