Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi guys,

first of all, sorry for this cross post but it seems very critical:
https://ask.openstack.org/en/question/78403/swift-broken-pipe/

I'm running in a very strange issue if i'm using ssbench to test my cluster.

  * Swift 2.2.2
  * 2 Proxy Nodes (64Gig RAM, 10G interfaces, 16 cores)
  * 3 Storage nodes (12 SATA => Object, SSD => Container/Acccount, 10G
interfaces, 8Cors)
  * L3 Keepalived LB
  * Ubuntu 14.04
  * kernel 3.19.x

Memcache is installed on both Proxy nodes. disperation-check is working
very well, ssbench isn't.

ssbench is located on another server with 1G connectivity!

|{
  "name": "Small test scenario",
  "sizes": [{
    "name": "tiny",
    "size_min": 100000,
    "size_max": 160000
  }, {
    "name": "small",
    "size_min": 400000,
    "size_max": 4000000
  }],
  "initial_files": {
    "tiny": 100,
    "small": 20
  },
  "operation_count": 1000,
  "crud_profile": [4, 3, 2, 2],
  "user_count": 5,
  "container_base": "ssbench",
  "container_count": 100,
  "container_concurrency": 100
}

sbench-master run-scenario -f large.scenario -u 200 -o 4000 --workers 4|

Starting workers

|/usr/local/bin/ssbench-worker -c 200 --zmq-host 10.0.0.4 -c 50
--batch-size 8 4|

Log output Proxy nodes:

|Jul 14 08:35:27 proxy1 swift: ERROR with Object server
192.168.100.7:6000/sdf re: Trying to write to
/v1/AUTH_a12e7b67dca043cba5eb395b6346b0a4/ssbench_000047/small_002615:
#012Traceback (most recent call last):#012  File
"/usr/lib/python2.7/dist-packages/swift/proxy/controllers/obj.py", line
317, in _send_file#012    conn.send(chunk)#012  File
"/usr/lib/python2.7/httplib.py", line 811, in send#012   
self.sock.sendall(data)#012  File
"/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 376, in
sendall#012    tail = self.send(data, flags)#012  File
"/usr/lib/python2.7/dist-packages/eventlet/greenio.py", line 358, in
send#012    total_sent += fd.send(data[total_sent:], flags)#012error:
[Errno 32] Broken pipe|

Object-Server count on each storage node: 28

I tested this scenario without the loadbalancer to check if anything is
wrong with the loadbalancer but without success.

I'm searching since days to solve this problem but without success :(

If you need more informations, please let me know.

Cheers
Heiko

- -- 
B. Sc. Informatik
Heiko Krämer
CIO/Administrator

Twitter: @railshoster
Avarteq GmbH
Zweigstelle:
Prinzessinnenstr. 20, 10969 Berlin

- ----
Geschäftsführer: Alexander Faißt, Dipl.-Inf.(FH) Julian Fischer
Handelsregister: AG Saarbrücken HRB 17413, Ust-IdNr.: DE262633168
Sitz:
Science Park 2
66123 Saarbrücken

Tel: +49 (0)681 / 309 64 190
Fax: +49 (0)681 / 309 64 191

Visit:
http://www.enterprise-rails.de/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJVpM5tAAoJELxFogM4ixOFXikIAKmtcdTRjFlPysXGUrCfhyl+
e17s6dSlmxZWfwWriAusdbhhjMSnZtrcmE7g0HplNrR720xEVu9oQDLPnHblU2Cu
+MCwI3F6A5p4ZliFQtwUikKxyGwQzc/4nJpCY7UY4Vg2UkWfcCX6TlfEykbih/yA
7m9oZahSXktaZVwDe6oggs6zU8GBaL7ecoojgMZn3Mb4Pr2J2Mxfs60V9/yF2//x
3yD4/9HKYcI+D4hE5ZEVETh2hnEvJZiM63txxwPN2zYvgjlJK2WDHQORoxlBrN0S
IwxA2GFWDR2G0PxJJbYzl3Xr3hbgWTjp4Hey0nXClvagf84obV0mjbhbtPXz6iM=
=Qr5Q
-----END PGP SIGNATURE-----

-------------- next part --------------
A non-text attachment was scrubbed...
Name: kraemer.vcf
Type: text/x-vcard
Size: 345 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150714/d36fbd39/attachment.vcf>