<html dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style id="owaParaStyle" type="text/css">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
P {margin-top:0;margin-bottom:0;}</style>
</head>
<body ocsi="0" fpstyle="1">
<div style="direction: ltr;font-family: Tahoma;color: #000000;font-size: 10pt;">
<div style="direction:ltr; font-family:Tahoma; color:#000000; font-size:10pt">On Mon, Jan 14, 2013 at 11:02 AM, Leander Bessa Beernaert
<span dir="ltr"><<a href="mailto:leanderbb@gmail.com" target="_blank">leanderbb@gmail.com</a>></span> wrote:<br>
<div style="font-family:Times New Roman; color:#000000; font-size:16px">
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex; border-left:1px #ccc solid; padding-left:1ex">
<div dir="ltr">Hello all,
<div><br>
</div>
<div><br>
</div>
<div>I'm trying to upload 200GB of 200KB files to Swift. I'm using 4 clients (each hosted on a different machine) with 10 threads each uploading files using the official python-swiftclient. Each thread is uploading to a separate container.</div>
<div><br>
</div>
<div>I have 5 storage nodes and 1 proxy node. The nodes are all running with a replication factor of 3. Each node has a quad-core i3 processor, 4GB of RAM and a gigabit network interface.</div>
<div><br>
</div>
<div>Is there any way I can speed up this process? At the moment it takes about 20 seconds per file or more.</div>
<br>
</div>
</blockquote>
</div>
<br>
It is very likely the system is starved on IO's.<br>
As a temporary workaround you can stop the object-replicator and object-auditor during the import to have less daemons competing for IO's.<br>
<br>
Some general troubleshooting tips:<br>
Use iotop to look for the processes consuming io's<br>
<br>
Assuming you use XFS:<br>
Make sure the filesystem is created with the appropriate inode size as described in the docs.<br>
(e.g. mkfs.xfs -i size=1024)<br>
<br>
Also with lots of files you need quite a bit of memory to cache the inodes into memory.
<br>
Use the xfs runtime stats to get some indication about the cache:<br>
<a href="http://xfs.org/index.php/Runtime_Stats" target="_blank">http://xfs.org/index.php/Runtime_Stats</a><br>
xs_dir_lookup and xs_ig_missed will give some indication how much IO's are spend on the inode lookups<br>
<br>
You can look at slabtop to see how much memory is used by the inode cache.<br>
<br>
Cheers,<br>
Robert<br>
<br>
</div>
</div>
</div>
</div>
</div>
</body>
</html>