Open Stack

When running ssbench  I've consistently gotten inconsistent results - 67% difference between the fastest and slowest CREATE times. Nothing (scenario, parameters, cluster, etc...) was varied between the runs.  I'm using tempAuth and not Keystone authentication.

Ssbench was run on a node  using the internal (10GbE) network that isn't currently part of the Swift cluster.
Is this commonly seen with Swift?
CREATE       Count:   406 (    0 error;     0 retries:  0.00%)  Average requests per second:  79.6
CREATE       Count:   389 (    0 error;     0 retries:  0.00%)  Average requests per second:  99.8
CREATE       Count:   412 (    0 error;     0 retries:  0.00%)  Average requests per second:  65.2
CREATE       Count:   416 (    0 error;     0 retries:  0.00%)  Average requests per second:  58.6
CREATE       Count:   406 (    0 error;     0 retries:  0.00%)  Average requests per second:  41.6
CREATE       Count:   409 (    0 error;     0 retries:  0.00%)  Average requests per second:  71.3
CREATE       Count:   395 (    0 error;     0 retries:  0.00%)  Average requests per second:  32.3

/var/log/swift entries for a PUT with a large latency doesn't seem to reveal much:
root at swift21:/home/swift/bin# grep txeaf22b65bbc442d091373399a931340e /var/log/swift/*
/var/log/swift/container.log:Aug  9 06:47:40 swift21 container-server 192.168.10.208 - - [09/Aug/2013:13:47:40 +0000] "PUT /sdc/8738/AUTH_test/ssbench_000074/100k.1_000180" 201 - "txeaf22b65bbc442d091373399a931340e" "-" "-" 0.0005
/var/log/swift/object.log:Aug  9 06:47:40 swift21 object-server 192.168.10.209 - - [09/Aug/2013:13:47:40 +0000] "PUT /sdd/12336/AUTH_test/ssbench_000074/100k.1_000180" 201 - "-" "txeaf22b65bbc442d091373399a931340e" "-" 5.3650

I'd like to investigate the possible source of the large result variation and requests with large latencies. Suggestions on where / what to look at/for are appreciated.

Details in: http://paste.openstack.org/show/43703

Thanks,
Tim

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130809/0e735c18/attachment.html>