[swift] clean up orphan segment parts
Hey there! We've been using swift and we came across this bug https://bugs.launchpad.net/swift/+bug/1813202. We decided to work it out and have an ongoing patch to solve this issue alongside ronaldo.maia@luizalabs.com, we would like to share and get some feedback before we submit it. To solve this, we took inspiration in obj/expirer.py: In S3 controllers we register task objects in an internal account, each task object points to an orphaned slo object, which can be created in two situations, when a multipart-upload is not completed or when a multipart object is overwritten. These tasks will be checked by the daemon (a new one we made called "s3-slo-expirer") after some time and deleted from the base accounts. Rafael
On Wed, 29 May 2024 11:01:55 -0300 Rafael Tavares Oliveira <rafael.oliveira@profusion.mobi> wrote:
... we would like to share and get some feedback before we submit it.
Well that's pretty hard, I sympathize. Nobody wants to look at imaginary code. It helps to put something out.
.. These tasks will be checked by the daemon (a new one we made called "s3-slo-expirer") after some time and deleted from the base accounts.
We already have so many mechanisms for this, starting with watchers. Alistair and the crew are working on the issue of lost S3 segments by adding a whole new xLO (ALO IIRC). He also started by adding yet another daemon, but last I heard he managed to fold into a replicator or something. There's a branch around, called "MPU", where all of that lives. Try to get on IRC during the weekly Swift meeting on Wednesday. The location and time is on OpenStack meeting wiki. -- Pete
Well that's pretty hard, I sympathize. Nobody wants to look at imaginary code. It helps to put something out.
So, we were trying to discuss before putting something out to try deliver something that would be more aligned to the project's expectations. Regardless, we have submitted our code in this topic: https://review.opendev.org/q/topic:%22remove-multipart-orphans%22 On Thu, May 30, 2024 at 11:19 AM Pete Zaitcev <zaitcev@redhat.com> wrote:
On Wed, 29 May 2024 11:01:55 -0300 Rafael Tavares Oliveira <rafael.oliveira@profusion.mobi> wrote:
... we would like to share and get some feedback before we submit it.
Well that's pretty hard, I sympathize. Nobody wants to look at imaginary code. It helps to put something out.
.. These tasks will be checked by the daemon (a new one we made called "s3-slo-expirer") after some time and deleted from the base accounts.
We already have so many mechanisms for this, starting with watchers.
Alistair and the crew are working on the issue of lost S3 segments by adding a whole new xLO (ALO IIRC). He also started by adding yet another daemon, but last I heard he managed to fold into a replicator or something. There's a branch around, called "MPU", where all of that lives.
Try to get on IRC during the weekly Swift meeting on Wednesday. The location and time is on OpenStack meeting wiki.
-- Pete
participants (2)
-
Pete Zaitcev
-
Rafael Tavares Oliveira