[Openstack] [SWIFT] What happens when the shard count on available disk is less than the total shard count?

Stephen Wood smwood4 at gmail.com
Sat Mar 1 07:22:51 UTC 2014


Thanks, Pete. I do not plan to wipe any data. I am phasing out the machines
that are running the container service now in favor of a few hosts that
have much faster disks.

Just out of curiosity, what would happen if the total partitions was
something like 32768 but the total aggregate weight of all the disks in the
cluster was less than this value? Does that cause a negative affect on the
cluster or performance?


On Fri, Feb 28, 2014 at 9:19 PM, Pete Zaitcev <zaitcev at redhat.com> wrote:

> On Fri, 28 Feb 2014 16:01:26 -0800
> Stephen Wood <smwood4 at gmail.com> wrote:
>
> > When originally running swift-ring-builder container.builder create, I
> set
> > the partitions at 15 to give a total of 32768 partitions to split across
> 23
> > hosts with 12 disks each. Now I am replacing the container service on
> these
> > 23 hosts with 4 hosts that have 1 disk.
>
> Are you going to re-run create, wiping all the customer data in
> the cluster?
>
> If you want to live-migrate, then you cannot change the partition
> power that you selected when you stood up the cluster, and we should
> not be having this conversation at all.
>
> If you want to build everything anew, then you can use the recommended
> formula from the Joe Arnold's book or a calculator like this:
>  http://rackerlabs.github.io/swift-ppc/
>   (currently not working in my Firefox)
>
> For your SSD example that would be (100 * 1 * 4), rounded up is 512
> or power 9. However, I would really not want to use 9, but use 15
> instead. The reason is, the overhead is absolutely negligible for
> powers that small, but you can't change that number over the lifetime
> of the cluster. So, today you have 4 SSDs, tomorrow you have 20,
> then what?
>
> > Since I have 4 hosts with 1 disk each, do I calculate the weight of each
> of
> > these disks as 2^15 / 4 so that the same overall partition numbers are
> > available even though we're only using a handful of the disks?
>
> Well, that sounds sane... Yes, if your original weights were about
> 118, then adding a big SSD with a weight of 8100 will result in
> expected effect. I would still try and phase it in gradually still,
> just in case if nothing else, per this:
>  https://swiftstack.com/blog/2012/04/09/swift-capacity-management/
>
> -- Pete
>



-- 
Stephen Wood
www.heystephenwood.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140228/0cee255f/attachment.html>


More information about the Openstack mailing list