[openstack-dev] Information Dispersal Algorithm implementation in OpenStack

Clay Gerrard clay.gerrard at gmail.com
Mon Nov 10 20:04:12 UTC 2014


I've never read that paper before and can not find a free copy online.
Based on the abstract it seems to be a parity based algorithm (erasure
codes) and would not be directly applicable to replication based
dispersion/data placement.

There is current work to enable an erasure code scheme for data encoding
[1] - but the current effort is focused on using the existing swift ring
implementation for partition desperation and placement.

If you research overlaps with the general idea of taking a object and
splitting into n chunks such that any k of the m chunks can construct the
file object you may want to align with the current development on-going in
this area [2].

Best regards,

1.
https://github.com/openstack/swift-specs/blob/master/specs/swift/erasure_coding.rst
2.
https://review.openstack.org/#/q/status:open+project:openstack/swift+branch:feature/ec,n,z

On Mon, Nov 10, 2014 at 2:23 PM, Aniket K <aniket.k at tcs.com> wrote:

> Dear Team,
>
> In OpenStack Swift, built-in replication technique is used to provide 3x+
> data redundancy that is enabling OpenStack to provide high availability. We
> are studying use of *Information Dispersal Algorithm(IDA)* as a technique
> instead of replication to provide high availability along with security.
>
> We would like to know,
> 1.        Was it considered in Swift before? If so, can you please share
> your opinion?
> 2.        Will this fit into existing Swift architecture that supports
> eventual consistency?
> 3.        Do you see advantages like storage efficiency, security etc. of
> using this technique over replication?
>
> Any pointer/suggestions/guidance in this regard will be appreciated.
>
> More information about IDA can be found at:
> *http://en.wikipedia.org/wiki/Secret_sharing*
> <http://en.wikipedia.org/wiki/Secret_sharing>
>
> Thanks in advance.
>
> Regards,
> Aniket Kulkarni
>
> =====-----=====-----=====
> Notice: The information contained in this e-mail
> message and/or attachments to it may contain
> confidential or privileged information. If you are
> not the intended recipient, any dissemination, use,
> review, distribution, printing or copying of the
> information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If
> you have received this communication in error,
> please notify us by reply e-mail or telephone and
> immediately and permanently delete the message
> and any attachments. Thank you
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141110/f07ecd37/attachment.html>


More information about the OpenStack-dev mailing list