*FYI more concrete logs * *Client Node* Request failed for objkey objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615 org.jclouds.http.HttpResponseException: command: DELETE http://www.example.com:80/v2/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615HTTP/1.1 failed with response: HTTP/1.1 409 Conflict; content: [<html><h1>Conflict</h1><p>There was a conflict when trying to complete your request.</p></html>] *Proxy Node * Apr 23 12:00:27 proxy1 swift-proxy allow user with role admin as account admin (txn: txe45b336b0ab44aee8a698-0053572ccb) (client_ip: 10.41.54.27) *Storage Node* Apr 23 12:00:26 object1* container-server* 10.41.54.9 - - [23/Apr/2014:03:00:26 +0000] "PUT /vdb1/955/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" 201 - "tx3ba78bc8924349d18f7af-0053572ccc" "PUT http://www.example.com/vdb1/861/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" "obj-server 17908" 0.0006 Apr 23 12:00:26 object1 *object-server* 10.41.54.21 - - [23/Apr/2014:03:00:26 +0000] "PUT /vdb1/861/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" 201 - "*PUT * http://www.example.com/v2/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" "tx3ba78bc8924349d18f7af-0053572ccc" "proxy-server 23927" 0.0113 Apr 23 12:00:26 object1 *object-server *10.41.54.19 - - [23/Apr/2014:03:00:26 +0000] "DELETE /vdb1/861/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" 409 - "*DELETE * http://www.example.com/v2/AUTH_215c458021024a8c87471157a7040644/zoom82/objkey19fbed5e-cb58-4a6d-b4fa-3b5768c7b615" "txe45b336b0ab44aee8a698-0053572ccb" "proxy-server 10071" 0.0003 On Wed, Apr 23, 2014 at 5:09 AM, Clay Gerrard <clay.gerrard at gmail.com>wrote: > 409 on DELETE (object?) is a pretty specific error. That should mean that > the timestamp assigned to the delete is earlier than the timestamp of the > data file. > > Most likely mean that you're getting some time-drift on your proxies (but > that assumes multi-node) or maybe that you're reusing names between threads > and your object server's see PUT(ts1) PUT(ts3) DELETE(ts2) - but that'd be > a pretty tight race... > > Should all be logged - try and find a DELETE that went 409 and trace the > transaction id. > > > On Mon, Apr 21, 2014 at 5:54 AM, taurus huang <huanggeng.8552 at gmail.com>wrote: > >> Please provide the log file： /var/log/swift/swift.log AND >> /var/log/keystone/keystone.log >> >> >> On Mon, Apr 21, 2014 at 11:55 AM, Sumit Gaur <sumitkgaur at gmail.com>wrote: >> >>> Hi >>> I using jclouds lib integrated with Openstack Swift+ keystone >>> combination. Things are working fine except stability test. After 20-30 >>> hours of test jclouds/SWIFT start degrading in TPS and keep going down over >>> the time. >>> >>> 1) I am running the (PUT-GET-DEL) cycle in 10 parallel threads. >>> 2) I am getting a lot of 409 and DEL failure for the response too from >>> SWIFT. >>> >>> >>> Can sombody help me what is going wrong here ? >>> >>> Thanks >>> sumit >>> >>> _______________________________________________ >>> OpenStack-dev mailing list >>> OpenStack-dev at lists.openstack.org >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> >> _______________________________________________ >> OpenStack-dev mailing list >> OpenStack-dev at lists.openstack.org >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> >> > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev at lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140423/3d2c03a3/attachment.html>