Large files multipart upload fails - OpenStack Object Storage (swift)

Komarov, Aleksandr aleksandr.komarov at itglobal.com
Wed Aug 3 16:30:41 UTC 2022


Hi to all!

Please suggest how to fix the situation.

When uploading files larger than 100GB, at the stage of multipart-manifest validation, we get 404 error for some segments that were previously uploaded successfully. As a result, the client receives a 400 response. Uploading a large file is not possible.

Logs:
Aug 1 13:18:30 swift01-object01 object-server: IP - - [01/Aug/2022:13:18:30 +0000] "PUT /mpathc/12130/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/000
00032" 201 - "PUT http://domain-name/v1/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/00000032" "tx30e82374b98845bab6883-0062e7d225" "proxy-server 8367" 128.7641 "-" 21577 0
Aug 1 13:18:30 swift01-object01 container-server: IP - - [01/Aug/2022:13:18:30 +0000] "PUT /mpathb/952/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/00
000032" 201 - "PUT http://domain-name/mpathe/12130/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/00000032" "tx30e82374b98845bab6883-0062
e7d225" "object-server 24710" 0.0005 "-" 13211 0
Aug 1 14:01:46 swift01-object01 object-server: IP - - [01/Aug/2022:14:01:46 +0000] "HEAD /mpathc/12130/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/00000032" 404 - "HEAD http://domain-name/v1/AUTH_5affc785b3454cddb2d1f69ce62b4441/bucket/.file-segments/150.file/00000032" "txf09676de38964ca299b77-0062e7dcc8" "proxy-server 8367" 0.0003 "-" 21579 0

At the same time, some segments successfully found. With files less than 50GB everything is fine.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.openstack.org/pipermail/openstack-discuss/attachments/20220803/dd87f905/attachment.htm>


More information about the openstack-discuss mailing list