<div dir="ltr"><div class="markdown-here-wrapper" style=""><p style="margin:0px 0px 1.2em!important">Hi Heiko, </p>
<p style="margin:0px 0px 1.2em!important">This command contains worker counts already. No need to start workers manually by using ssbench-worker. </p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">ssbench-master run-scenario -f large.scenario -u 200 -o 4000 --workers 4
</code></pre><ul style="margin:1.2em 0px;padding-left:2em">
<li style="margin:0.5em 0px">Ensure there’s no any workers “$ps aux | grep ssbench”</li>
<li style="margin:0.5em 0px">Try to kill workers “$ssbench-master kill-workers” </li>
<li style="margin:0.5em 0px">Try to start with lower “user count” (-u) and less operation counts(-o). </li>
</ul>
<p style="margin:0px 0px 1.2em!important">ssbench-master run-scenario -f large.scenario -u 10 -r 30 —workers 1</p>
<pre style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;font-size:1em;line-height:1.2em;margin:1.2em 0px"><code style="font-size:0.85em;font-family:Consolas,Inconsolata,Courier,monospace;margin:0px 0.15em;padding:0px 0.3em;white-space:pre-wrap;border:1px solid rgb(234,234,234);border-radius:3px;display:inline;background-color:rgb(248,248,248);white-space:pre;overflow:auto;border-radius:3px;border:1px solid rgb(204,204,204);padding:0.5em 0.7em;display:block!important">{
"name": "Small test scenario",
"sizes": [{
"name": "tiny",
"size_min": 100000,
"size_max": 160000
}, {
"name": "small",
"size_min": 400000,
"size_max": 4000000
}],
"initial_files": {
"tiny": 100
},
"operation_count": 1000,
"crud_profile": [1, 0, 0, 0],
"user_count": 5,
"container_base": "ssbench",
"container_count": 100,
"container_concurrency": 100
}
</code></pre><p style="margin:0px 0px 1.2em!important">Hugo</p>
<div title="MDH:SGkgSGVpa28swqA8ZGl2Pjxicj48L2Rpdj48ZGl2PlRoaXMgY29tbWFuZCBjb250YWlucyB3b3Jr
ZXIgY291bnRzIGFscmVhZHkuIE5vIG5lZWQgdG8gc3RhcnQgd29ya2VycyBtYW51YWxseSBieSB1
c2luZyBzc2JlbmNoLXdvcmtlci4mbmJzcDs8L2Rpdj48ZGl2Pjxicj48L2Rpdj48ZGl2PmBgYDwv
ZGl2PjxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPnNzYmVuY2gtbWFzdGVyIHJ1
bi1zY2VuYXJpbyAtZiBsYXJnZS5zY2VuYXJpbyAtdSAyMDAgLW8gNDAwMCAtLXdvcmtlcnMgNCZu
YnNwOzwvc3Bhbj48L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij5gYGA8
L3NwYW4+PC9kaXY+PGRpdj48c3BhbiBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+PGJyPjwvc3Bh
bj48L2Rpdj48ZGl2PiogRW5zdXJlIHRoZXJlJ3Mgbm8gYW55IHdvcmtlcnMgIiRwcyBhdXggfCBn
cmVwIHNzYmVuY2giPC9kaXY+PGRpdj4qIFRyeSB0byBraWxsIHdvcmtlcnMgIiRzc2JlbmNoLW1h
c3RlciBraWxsLXdvcmtlcnMiJm5ic3A7PC9kaXY+PGRpdj4qIFRyeSB0byBzdGFydCB3aXRoIGxv
d2VyICJ1c2VyIGNvdW50IiAoLXUpICZuYnNwO2FuZCBsZXNzIG9wZXJhdGlvbiBjb3VudHMoLW8p
LiZuYnNwOzwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZTogMTRweDsiPnNzYmVuY2gtbWFzdGVyIHJ1bi1zY2VuYXJpbyAtZiBsYXJn
ZS5zY2VuYXJpbyAtdSAxMCAtciAzMCAtLXdvcmtlcnMgMTwvc3Bhbj48YnI+PC9kaXY+PGRpdj48
YnI+PC9kaXY+PGRpdj5gYGA8L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7
Ij57PC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+PHNwYW4gc3R5bGU9ImZvbnQt
c2l6ZTogMTRweDsiPiZuYnNwOyAibmFtZSI6ICJTbWFsbCB0ZXN0IHNjZW5hcmlvIiw8L3NwYW4+
PGJyIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOiAxNHB4
OyI+Jm5ic3A7ICJzaXplcyI6IFt7PC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPiZuYnNwOyAmbmJzcDsgIm5hbWUiOiAidGlu
eSIsPC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+PHNwYW4gc3R5bGU9ImZvbnQt
c2l6ZTogMTRweDsiPiZuYnNwOyAmbmJzcDsgInNpemVfbWluIjogMTAwMDAwLDwvc3Bhbj48YnIg
c3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij4m
bmJzcDsgJm5ic3A7ICJzaXplX21heCI6IDE2MDAwMDwvc3Bhbj48YnIgc3R5bGU9ImZvbnQtc2l6
ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij4mbmJzcDsgfSwgezwvc3Bh
bj48YnIgc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0
cHg7Ij4mbmJzcDsgJm5ic3A7ICJuYW1lIjogInNtYWxsIiw8L3NwYW4+PGJyIHN0eWxlPSJmb250
LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+Jm5ic3A7ICZuYnNw
OyAic2l6ZV9taW4iOiA0MDAwMDAsPC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPiZuYnNwOyAmbmJzcDsgInNpemVfbWF4Ijog
NDAwMDAwMDwvc3Bhbj48YnIgc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6IDE0cHg7Ij4mbmJzcDsgfV0sPC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAx
NHB4OyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPiZuYnNwOyAiaW5pdGlhbF9maWxl
cyI6IHs8L3NwYW4+PGJyIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOiAxNHB4OyI+Jm5ic3A7ICZuYnNwOyAidGlueSI6IDEwMDwvc3Bhbj48YnIgc3R5bGU9
ImZvbnQtc2l6ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij4mbmJzcDsg
fSw8L3NwYW4+PGJyIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9udC1z
aXplOiAxNHB4OyI+Jm5ic3A7ICJvcGVyYXRpb25fY291bnQiOiAxMDAwLDwvc3Bhbj48YnIgc3R5
bGU9ImZvbnQtc2l6ZTogMTRweDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij4mbmJz
cDsgImNydWRfcHJvZmlsZSI6IFsxLCAwLCAwLCAwXSw8L3NwYW4+PGJyIHN0eWxlPSJmb250LXNp
emU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOiAxNHB4OyI+Jm5ic3A7ICJ1c2VyX2Nv
dW50IjogNSw8L3NwYW4+PGJyIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0i
Zm9udC1zaXplOiAxNHB4OyI+Jm5ic3A7ICJjb250YWluZXJfYmFzZSI6ICJzc2JlbmNoIiw8L3Nw
YW4+PGJyIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOiAx
NHB4OyI+Jm5ic3A7ICJjb250YWluZXJfY291bnQiOiAxMDAsPC9zcGFuPjxiciBzdHlsZT0iZm9u
dC1zaXplOiAxNHB4OyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPiZuYnNwOyAiY29u
dGFpbmVyX2NvbmN1cnJlbmN5IjogMTAwPC9zcGFuPjxiciBzdHlsZT0iZm9udC1zaXplOiAxNHB4
OyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPn08L3NwYW4+PGJyPjwvZGl2PjxkaXY+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPmBgYDwvc3Bhbj48L2Rpdj48ZGl2Pjxicj48
L2Rpdj48ZGl2PjxzcGFuIHN0eWxlPSJmb250LXNpemU6IDE0cHg7Ij48YnI+PC9zcGFuPjwvZGl2
PjxkaXY+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTogMTRweDsiPkh1Z288L3NwYW4+PC9kaXY+" style="height:0;width:0;max-height:0;max-width:0;overflow:hidden;font-size:0em;padding:0;margin:0"></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">2015-07-14 16:55 GMT+08:00 Heiko Krämer <span dir="ltr"><<a href="mailto:kraemer@avarteq.de" target="_blank">kraemer@avarteq.de</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
Hi guys,<br>
<br>
first of all, sorry for this cross post but it seems very critical:<br>
<a href="https://ask.openstack.org/en/question/78403/swift-broken-pipe/" rel="noreferrer" target="_blank">https://ask.openstack.org/en/question/78403/swift-broken-pipe/</a><br>
<br>
I'm running in a very strange issue if i'm using ssbench to test my cluster.<br>
<br>
* Swift 2.2.2<br>
* 2 Proxy Nodes (64Gig RAM, 10G interfaces, 16 cores)<br>
* 3 Storage nodes (12 SATA => Object, SSD => Container/Acccount, 10G<br>
interfaces, 8Cors)<br>
* L3 Keepalived LB<br>
* Ubuntu 14.04<br>
* kernel 3.19.x<br>
<br>
Memcache is installed on both Proxy nodes. disperation-check is working<br>
very well, ssbench isn't.<br>
<br>
ssbench is located on another server with 1G connectivity!<br>
<br>
|{<br>
"name": "Small test scenario",<br>
"sizes": [{<br>
"name": "tiny",<br>
"size_min": 100000,<br>
"size_max": 160000<br>
}, {<br>
"name": "small",<br>
"size_min": 400000,<br>
"size_max": 4000000<br>
}],<br>
"initial_files": {<br>
"tiny": 100,<br>
"small": 20<br>
},<br>
"operation_count": 1000,<br>
"crud_profile": [4, 3, 2, 2],<br>
"user_count": 5,<br>
"container_base": "ssbench",<br>
"container_count": 100,<br>
"container_concurrency": 100<br>
}<br>
<br>
sbench-master run-scenario -f large.scenario -u 200 -o 4000 --workers 4|<br>
<br>
Starting workers<br>
<br>
|/usr/local/bin/ssbench-worker -c 200 --zmq-host 10.0.0.4 -c 50<br>
--batch-size 8 4|<br>
<br>
Log output Proxy nodes:<br>
<br>
|Jul 14 08:35:27 proxy1 swift: ERROR with Object server<br>
<a href="http://192.168.100.7:6000/sdf" rel="noreferrer" target="_blank">192.168.100.7:6000/sdf</a> re: Trying to write to<br>
/v1/AUTH_a12e7b67dca043cba5eb395b6346b0a4/ssbench_000047/small_002615:<br>
#012Traceback (most recent call last):#012 File<br>
"/usr/lib/python2.7/dist-packages/swift/proxy/controllers/obj.py", line<br>
317, in _send_file#012 conn.send(chunk)#012 File<br>
"/usr/lib/python2.7/httplib.py", line 811, in send#012<br>
self.sock.sendall(data)#012 File<br>
"/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 376, in<br>
sendall#012 tail = self.send(data, flags)#012 File<br>
"/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 358, in<br>
send#012 total_sent += fd.send(data[total_sent:], flags)#012error:<br>
[Errno 32] Broken pipe|<br>
<br>
Object-Server count on each storage node: 28<br>
<br>
I tested this scenario without the loadbalancer to check if anything is<br>
wrong with the loadbalancer but without success.<br>
<br>
I'm searching since days to solve this problem but without success :(<br>
<br>
If you need more informations, please let me know.<br>
<br>
Cheers<br>
Heiko<br>
<br>
- --<br>
B. Sc. Informatik<br>
Heiko Krämer<br>
CIO/Administrator<br>
<br>
Twitter: @railshoster<br>
Avarteq GmbH<br>
Zweigstelle:<br>
Prinzessinnenstr. 20, 10969 Berlin<br>
<br>
- ----<br>
Geschäftsführer: Alexander Faißt, Dipl.-Inf.(FH) Julian Fischer<br>
Handelsregister: AG Saarbrücken HRB 17413, Ust-IdNr.: DE262633168<br>
Sitz:<br>
Science Park 2<br>
66123 Saarbrücken<br>
<br>
Tel: +49 (0)681 / 309 64 190<br>
Fax: +49 (0)681 / 309 64 191<br>
<br>
Visit:<br>
<a href="http://www.enterprise-rails.de/" rel="noreferrer" target="_blank">http://www.enterprise-rails.de/</a><br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v1<br>
<br>
iQEcBAEBAgAGBQJVpM5tAAoJELxFogM4ixOFXikIAKmtcdTRjFlPysXGUrCfhyl+<br>
e17s6dSlmxZWfwWriAusdbhhjMSnZtrcmE7g0HplNrR720xEVu9oQDLPnHblU2Cu<br>
+MCwI3F6A5p4ZliFQtwUikKxyGwQzc/4nJpCY7UY4Vg2UkWfcCX6TlfEykbih/yA<br>
7m9oZahSXktaZVwDe6oggs6zU8GBaL7ecoojgMZn3Mb4Pr2J2Mxfs60V9/yF2//x<br>
3yD4/9HKYcI+D4hE5ZEVETh2hnEvJZiM63txxwPN2zYvgjlJK2WDHQORoxlBrN0S<br>
IwxA2GFWDR2G0PxJJbYzl3Xr3hbgWTjp4Hey0nXClvagf84obV0mjbhbtPXz6iM=<br>
=Qr5Q<br>
-----END PGP SIGNATURE-----<br>
<br>
<br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>