Paulo, I believe you'll find that we're thinking along the same lines. Please review my proposal at http://etherpad.openstack.org/P9MMYSWE6U One quick observation is that SHA-1 is totally inadequate for fingerprinting objects in a public object store. An attacker could easily predict the fingerprint of content likely to be posted, generate alternate content that had the same SHA-1 fingerprint and pre-empt the signature. For example: an ISO of an open source OS distribution. If I get my false content with the same fingerprint into the repository first then everyone who downloads that ISO will get my altered copy. SHA-256 is really needed to make this type of attack infeasible. I also think that distributed deduplication works very well with object versioning. Your comments on the proposal cited above would be great to hear. From: openstack-bounces+caitlin.bestler=nexenta.com at lists.launchpad.net [mailto:openstack-bounces+caitlin.bestler=nexenta.com at lists.launchpad.net] On Behalf Of Paulo Ricardo Motta Gomes Sent: Thursday, March 08, 2012 1:19 PM To: openstack at lists.launchpad.net Subject: [Openstack] Enabling data deduplication on Swift Hello everyone, I'm a student of the European Master in Distributed Computing (EMDC) currently working on my master thesis on distributed content-addressable storage/deduplication. I'm happy to announce I will be contributing the outcome of my thesis work to OpenStack by enabling both object-level and block-level deduplication functionality on Swift (https://answers.launchpad.net/swift/+question/156862). I have written a detailed blog post where I describe the initial architecture of my solution: http://paulormg.com/2012/03/05/enabling-deduplication-in-a-distributed-object-storage/ Feedback from the OpenStack/Swift community would be very appreciated. Cheers, Paulo -- European Master in Distributed Computing - www.kth.se/emdc<http://www.kth.se/emdc> Royal Institute of Technology - KTH Instituto Superior Técnico - IST http://paulormg.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.openstack.org/pipermail/openstack/attachments/20120309/18d0aecc/attachment.html>