Open Stack

That formula doesn't make sense to me?  4097 seems to be a magic number in
there?  what does 2 ^ 12 have to do with anything?

You can't *really* say how many objects will be in a partition without
thinking about how many objects will be in the cluster.  But number of
partitions per disk (2^part_power * replica_count / number_of_drives) can
be informative.

For example, if you've only got 50 disks and you're running a big 'ol part
power like 18, then as soon as you get a few 100K objects in your swift
cluster you'll have near full coverage in the partition space and can
expect ~15K partitions on each disk.  That's 15,000 top-level directories
in /srv/node/sdX/objects - and under each ahead of the .data file you have
a part/suffix/hash directories to boot!

With a part power of 17 that top-level directory count on those 50 devices
(which is like what - 3 or 4 servers?) drops down to a much more manageable
7864.  Which could easily scale up almost TWO orders of magnitude, 80x or
so at least, to 3,000-4,000 drives before you'll be worried too much about
lumpy drive utilization that might pop up as the partition count per drives
falls off depending on the distribution of your object size.

If you just want to estimate the number of objects on a single disk,
assuming your disks are all the same size (*and* you have *either* roughly
even sized zones & regions *or* your number of zones and/or number of
regions are >= number of replicas) you can just say:

number of objects in cluster / number of devices in cluster

^ nothing magic, swift's placement strategy tries pretty hard to be well
distributed even as you add and remove capacity things will try to even out.


On Mon, Feb 24, 2014 at 8:25 PM, Remo Mattei <remo at italy1.com> wrote:

> Hi again,
> I have read about this formula can someone enhance with some additional
> details about how many objects can be in a swift cluster?
>
>
> §To find out how many items a partition will hold use the following
> formula
> §For example, if the
> §partition num is 2^18,
> §replica_num is 3,
> §disk_num is 50,
> §item_num ~= 2^18 * 3 * 4097 / 50 ~= 64440238.
>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140225/a14fdc78/attachment.html>