[openstack-dev] [nova][cinder] Limits on volume read throughput?

Preston L. Bannister preston at bannister.us
Thu Mar 10 06:55:41 UTC 2016


In my case Centos 7 is using QEMU 1.5.3 ... which is *ancient*. This is on
a node with a packstack install of OpenStack. If you have a different
result, I would like to know why...

Got a bit further in my reading and testing. Also got my raw volume read
performance in an instance from ~300MB/s (with some tweaking) up to the
current ~800MB/s. Given the host raw volume read rate is ~1.2GB/s, and
there are substantial improvements in the software stack (Linux, iSCSI,
QEMU) in later versions ... this is a good result.

Found the main bottleneck was the iSCSI target in the physical Linux host.
(Not in my current test case, in QEMU.) From online presentations on
QEMU/iSCSI/Linux, there are known large improvements in more recent
versions. Punt. Need to re-test on top of Ubuntu Trusty LTS (what most
customers seem headed toward).  Will re-base my testing, at some point.

My testing (for simplicity) is on an all-in-one node.

Curious what other folk are getting with very-fast iSCSI targets. What is
the upper range?








On Mon, Mar 7, 2016 at 7:59 AM, Chris Friesen <chris.friesen at windriver.com>
wrote:

> Just a heads-up that the 3.10 kernel in CentOS/RHEL is *not* a stock 3.10
> kernel.  It has had many things backported from later kernels, though they
> may not have backported the specific improvements you're looking for.
>
> I think CentOS is using qemu 2.3, which is pretty new.  Not sure how new
> their libiscsi is though.
>
> Chris
>
> On 03/07/2016 12:25 AM, Preston L. Bannister wrote:
>
>> Should add that the physical host of the moment is Centos 7 with a
>> packstack
>> install of OpenStack. The instance is Ubuntu Trusty. Centos 7 has a
>> relatively
>> old 3.10 Linux kernel.
>>
>>  From the last week (or so) of digging, I found there were substantial
>> claimed
>> improvements in /both/ flash support in Linux and the block I/O path in
>> QEMU -
>> in more recent versions. How much that impacts the current measures, I do
>> not
>> (yet) know.
>>
>> Which suggests a bit of tension. Redhat folk are behind much of these
>> improvements, but RHEL (and Centos) are rather far behind. Existing RHEL
>> customers want and need careful, conservative changes. Folk deploying
>> OpenStack
>> need more aggressive release content, for which Ubuntu is currently the
>> best base.
>>
>> Will we see a "Redhat Cloud Base" as an offering with RHEL support
>> levels, and
>> more aggressive QEMU and Linux kernel inclusion?
>>
>> At least for now, building OpenStack clouds on Ubuntu might be a much
>> better bet.
>>
>>
>> Are those claimed improvements in QEMU and the Linux kernel going to make
>> a
>> difference in my measured result? I do not know. Still reading, building
>> tests,
>> and collecting measures...
>>
>>
>>
>>
>> On Thu, Mar 3, 2016 at 11:28 AM, Chris Friesen <
>> chris.friesen at windriver.com
>> <mailto:chris.friesen at windriver.com>> wrote:
>>
>>     On 03/03/2016 01:13 PM, Preston L. Bannister wrote:
>>
>>              > Scanning the same volume from within the instance still
>> gets the same
>>              > ~450MB/s that I saw before.
>>
>>              Hmmm, with iSCSI inbetween that could be the TCP memcpy
>> limitation.
>>
>>
>>         Measuring iSCSI in isolation is next on my list. Both on the
>> physical
>>         host, and
>>         in the instance. (Now to find that link to the iSCSI test,
>> again...)
>>
>>
>>     Based on earlier comments it appears that you're using the qemu
>> built-in
>>     iSCSI initiator.
>>
>>     Assuming that's the case, maybe it would make sense to do a test run
>> with
>>     the in-kernel iSCSI code and take qemu out of the picture?
>>
>>     Chris
>>
>>
>>
>> __________________________________________________________________________
>>     OpenStack Development Mailing List (not for usage questions)
>>     Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>>     <http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe
>> >
>>     http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160309/14a91b9f/attachment.html>


More information about the OpenStack-dev mailing list