Open Stack

> ________________________________________
> From: Clint Byrum <clint at fewbar.com>
> Sent: Wednesday, October 12, 2016 10:46 PM
> To: openstack-operators
> Subject: Re: [Openstack-operators] [openstack-operators][ceph][nova] How do     you handle Nova on Ceph?
> 
> Excerpts from Adam Kijak's message of 2016-10-12 12:23:41 +0000:
> > > ________________________________________
> > > From: Xav Paice <xavpaice at gmail.com>
> > > Sent: Monday, October 10, 2016 8:41 PM
> > > To: openstack-operators at lists.openstack.org
> > > Subject: Re: [Openstack-operators] [openstack-operators][ceph][nova] How do you handle Nova on Ceph?
> > >
> > > I'm really keen to hear more about those limitations.
> >
> > Basically it's all related to the failure domain ("blast radius") and risk management.
> > Bigger Ceph cluster means more users.
> 
> Are these risks well documented? Since Ceph is specifically designed
> _not_ to have the kind of large blast radius that one might see with
> say, a centralized SAN, I'm curious to hear what events trigger
> cluster-wide blasts.

In theory yes, Ceph is desgined to be fault tolerant,
but from our experience it's not always like that.
I think it's not well documented, but I know this case:
https://www.mail-archive.com/ceph-users@lists.ceph.com/msg32804.html

> > Growing the Ceph cluster temporary slows it down, so many users will be affected.
> One might say that a Ceph cluster that can't be grown without the users
> noticing is an over-subscribed Ceph cluster. My understanding is that
> one is always advised to provision a certain amount of cluster capacity
> for growing and replicating to replaced drives.

I agree that provisioning a fixed size Cluster would solve some problems but planning the capacity is not always easy.
Predicting the size and making it cost effective (empty big Ceph cluster costs a lot on the beginning) is quite difficult.
Also adding a new Ceph cluster will be always more transparent to users than manipulating existing one especially when growing pool PGs)