<div dir="ltr">Send without compression is more slower because it send a large data.<div>In my environment, send 10GB disk to swift except compression spent 6min 40sec but with compression 4min 13sec.<div><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-04-02 23:16 GMT+09:00 Murali Balcha <span dir="ltr"><<a href="mailto:Murali.Balcha@triliodata.com" target="_blank">Murali.Balcha@triliodata.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Just curious. What is the overhead of compression and other backup processes? How much time does it take to upload a simple 50GB file to swift compare to backup
of 50 GB to swift?<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Duncan Thomas [mailto:<a href="mailto:duncan.thomas@gmail.com" target="_blank">duncan.thomas@gmail.com</a>]
<br>
<b>Sent:</b> Wednesday, April 01, 2015 6:13 AM<br>
<b>To:</b> OpenStack Development Mailing List (not for usage questions)<br>
<b>Subject:</b> Re: [openstack-dev] [cinder] Issue for backup speed<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">This is something we're working on (I work with the author of the patch you referenced) but the refactoring of the backup code in this cycle has made progress challenging. If you have a patch that works, please submit it, even if it needs
some cleaning up, we'd be happy to work with you on and testing, cleaning up and improvements.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">The basic problem is that backup is CPU bound (compression, ssl) so the existing parallelisation techniques used in cinder don't help. Running many cinder-backup processes can give you good aggregate throughput if you're running many backups
at once, but this appears not to be a common case, even in a large public cloud.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On 1 April 2015 at 11:41, Jae Sang Lee <<a href="mailto:hyangii@gmail.com" target="_blank">hyangii@gmail.com</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Hi,</span><u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">I tested Swift backup driver in Cinder-backup and that performance isn't high.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">In our test environment, The average time for backup 50G volume is 20min.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">I found a patch for this that add multi thread for swift backup driver(<a href="https://review.openstack.org/#/c/111314" target="_blank">https://review.openstack.org/#/c/111314</a>) but It's
also too slow. It looks like that patch doesn't implement thread properly.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Is there any improvement way about this? I'd appreciate other's thoughts on these issues.</span><u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif">Thanks.</span><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">
OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"><br>
<br clear="all">
<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<p class="MsoNormal">-- <u></u><u></u></p>
<div>
<p class="MsoNormal">Duncan Thomas<u></u><u></u></p>
</div>
</div>
</div></div></div>
</div>
<br>__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div>