<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style id="owaParaStyle" type="text/css">P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">> So it seems a big number of inodes considering that we are using just a 15% of the total capacity. A different thing here is that we are using 512K of inode size and we have a
big amount of memory .
<div style="font-family: Times New Roman; color: #000000; font-size: 16px">
<div>
<div dir="ltr">
<div>
<div>> Also we always have one of our disks close to 100% of util, and this is caused by the object-auditor that scans all our disks continuously. </div>
<div><br>
</div>
<div>>So we was also thinking in the possibility to change the kind of disks that we are using, to use smaller and faster disks.</div>
<div>>Will be really util to know what kind of disks are you using in your old and new storage nodes, and compare that with our case.<br>
</div>
</div>
<div><font face="arial, sans-serif"><br>
Max,<br>
<br>
</font></div>
Old config:<br>
* 6 X 3.5 inch 2TB (I think 5400 RPM) <br>
* 10GB flashcache per disk, on 1 SSD<br>
* 48GB Mem<br>
<br>
moving to:<br>
* 10 X 1 TB 2.5 inch disk (7200RPM) <br>
* 10GB flashcache per disk, split over 2 x SSD <br>
* 256 GB Mem<br>
<br>
We are considering changing to 128GB memory for the new nodes:<br>
Strangely the 48GB is fully used but when we look at the 256GB machines there is just about 20GB used for Inodes and 55GB used in total.<br>
The vm.vfs_cache_pressure is set to 1 but you can't really set it to 0 because it will run out of memory (if not immediately it will do it after a re-balance in Swift).<br>
Maybe the machines are just not under enough load yet to make it cache more Inodes but currently it does not look like 256GB is needed.<br>
<br>
Concerning the object-auditor: <br>
We run a cronjob on our machines that will do a ionice on the swift processes: <br>
Preference is given to the object-server and replicator/auditor have a lower prio.<br>
Those processes will still max-out a disk but hopefully it should not hurt regular performance to much.<br>
<br>
Let me know if you run a SWIFT/COSbench against your setup.<br>
I'm curious what number of puts you are able to process currently.<br>
<br>
Cheers,<br>
Robert<br>
</div>
</div>
</div>
</div>
</body>
</html>