We have used 'mixed-policy MPU' with some buckets i.e. the main bucket is 3-replica but we manually create the +segments bucket as EC policy. (Note there is a bug fix needed https://bugs.launchpad.net/swift/+bug/2038459 fixed by commit 60c04f1, so you'll need to be on a recent release). This isn't quite as flexible as what you've described.
This would be a nice-to-have feature for the new native-MPUs https://review.opendev.org/q/project:openstack/swift+branch:feature/mpu+status:open feature where the segments/parts buckets are hidden, if we could dynamically create per-policy 'parts' buckets (embed the policy index in the hidden container name) and have the MPU manifest point to the correct parts bucket. One challenge will be figuring out how to represent the policy for the parts container with an S3 CreateMultipartUpload.
Alistair
On Mon, Jun 24, 2024 at 1:04 PM Rafael Tavares Oliveira < rafael.oliveira@profusion.mobi> wrote:
Hey there!
We would very much like to allow multiple objects in the same bucket to have multiple storage policies, different from their container. Since we noticed that this is not possible now, we'd like to take a shot at it. But first, any suggestions or ideas on implementing this?
The idea we are following here and trying to implement would be to use the same logic as the `+segments` bucket: each bucket would have a `+cold` counterpart, in the cold storage policy, which would store such objects, and we would keep a manifest/symlink in the original bucket, which we can use to retrieve the content from cold storage.
Just as a piece of context, we are basing this solution on yoga release and using S3 apis mainly.
Thanks for the help! Rafael