Open Stack

I strongly prefer to configure rsyncd with a module per disk:

https://github.com/openstack/swift/blob/c0640f87107d84d262c20bdc1250b805ae8f9482/etc/rsyncd.conf-sample#L25

and then tune the per-disk connection limit to 2-4

There's not really hard and fast rule, in some sense it's related to
replicator concurrency (the producer of the rsync's that *want* those
connections) - targeting roughly:

   replicator-concurrency * #nodes ~== (max_conenction * nodes or
max_connection_per_module_per_disk * #disks)

But max connections is about enforcing a sane limit, if you hit that limit
*a lot* it may be limiting throughput (and can lead to other
inefficiencies).  But having a few max connections fire now and again is
probably a good thing, since there's no guarantee that every replicator
coro in the cluster might not be thinking it should be talking to a
partition that just so happen to all have a replica on this one disk at
this one time.  You generally don't want too many connections hammering a
single spindle because of the await it introduces for client requests that
might need to hit that disk (but there's auditor configuration management
as well, and the new ionice options for tuning might be an option as well).

-Clay

On Tue, Dec 13, 2016 at 3:11 PM, Mark Kirkwood <
mark.kirkwood at catalyst.net.nz> wrote:

> Any thoughts on this one guys?
>
>
>
> On 06/12/16 09:56, Mark Kirkwood wrote:
>
>> Hi,
>>
>> Is there a way to calculate rsync max connection (i.e based on number of
>> hosts, devices, object server workers etc)?
>>
>> Some context: we have recently gone live with Swift 2.7.0 cluster (2
>> regions, 6 hosts overall, each with 16 cores/32 threads and 4 drives). We
>> have rsync max connections to 25 (which I think is being set by
>> puppet-swift.., is the default value). We are seeing errors in the rsync
>> log: 'max connections (25) reached'. I'd like to be able to say to the
>> operations guys: set 'max connections to x, because <some rules for
>> calculating it>'.
>>
>>
>> Cheers
>>
>>
>> Mark
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>> Post to     : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi
>> -bin/mailman/listinfo/openstack
>>
>
>
> _______________________________________________
> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
> k
> Post to     : openstack at lists.openstack.org
> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
> k
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20161213/afacf50d/attachment.html>