[Openstack] Swift generating thousands of rsyncs per second on a small cluster
Diogo Vieira
dfv at eurotux.com
Tue Jul 1 09:50:15 UTC 2014
> The object servers should only be talking to each other during replication. They should not talk to the proxy, and probably not the load balancer. Can you provide the output of "swift-ring-builder /etc/swift/object.builder" and more details on the network configuration of this system.
First please allow me to correct my first email. I found out that most of the traffic is REPLICATE requests and not rsyncs between storage nodes.
So here is the output of the ring builder:
# swift-ring-builder /etc/swift/object.builder
/etc/swift/object.builder, build version 5
262144 partitions, 3.000000 replicas, 1 regions, 5 zones, 5 devices, 0.00 balance
The minimum number of hours before a partition can be reassigned is 1
Devices: id region zone ip address port replication ip replication port name weight partitions balance meta
0 1 5 10.10.6.16 6000 10.10.6.16 6000 device4 10.00 157287 0.00
1 1 3 10.10.6.14 6000 10.10.6.14 6000 device2 10.00 157286 -0.00
2 1 1 10.10.6.17 6000 10.10.6.17 6000 device5 10.00 157286 -0.00
3 1 2 10.10.4.21 6000 10.10.4.21 6000 device1 10.00 157287 0.00
4 1 4 10.10.6.15 6000 10.10.6.15 6000 device3 10.00 157286 -0.00
Well, as far as network configuration goes, I'm running Openstack on two blades. The second one has only a compute node and Swift installed (1 Proxy Node and 1 Storage Node). Both machines are on the same vlan. The others Storage Nodes are on a different vlan as guest virtual machines of the first two. Between the two vlans there's a firewall where I logged the traffic.
It came to my attention that this should be the object-replicator doing its work, but the fact is that I get about a thousand or 2 requests per second between vlans - (the real machine running a Proxy and a Storage Node) and the virtual machines (all Storage Nodes only) - and it consumes a lot of resources (mainly CPU) seems to be a bit of a waste because the cluster is not being used for more than 2 weeks - nothing added or deleted from it.
Can you tell me if this is normal behaviour? If so, how will this scale when I add more objects? Will it keep getting more and more CPU usage?
> Generally the proxies are configured with multiple interfaces as the object servers and other backend storage nodes should not be accessible to clients directly.
Yes, I believe this is the case.
Thank you for your answer
Diogo Vieira <dfv at eurotux.com>
Programador
Eurotux Informática, S.A. | www.eurotux.com
(t) +351 253 680 300
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140701/b94997dc/attachment.html>
More information about the Openstack
mailing list