Open Stack

On Wed, 16 Mar 2016 13:23:31 +1300
Mark Kirkwood <mark.kirkwood at catalyst.net.nz> wrote:

> So integrating swift-recon into regular monitoring/alerting 
> (collectd/nagios or whatever) is one approach (mind you most folk 
> already monitor disk usage data... and there is nothing overly special 
> about ensuring you don't run of space)!

So the overall conclusion is that the operator must monitor the cluster's
state and not let it run out of space. If you do in fact run out, second
order trouble starts happening, in particular pending processing will
not run right.

In case of one or a few nodes run out of space due to a bug or some
unrelated problem, Swift may maintain the desired durability by using
so-called "handoff" devices. If you restore the primaries, replication
will relocate affected partitions from handoffs. That will keep the
cluster functional while the recovery is being implemented.

But overall there's no magic. The general idea is, you make your customers
pay and if the business is profitable, they pay you enough to buy new
storage just fast enough to keep ahead of them filling it. For operators
of private clouds, we have quotas.

-- Pete