[openstack-dev] [Swift] Erasure coding and geo replication

Mark Kirkwood mark.kirkwood at catalyst.net.nz
Tue Feb 16 04:38:59 UTC 2016


On 16/02/16 17:10, Mark Kirkwood wrote:
> On 15/02/16 23:29, Kota TSUYUZAKI wrote:
>> Hello Mark,
>>
>> AFAIK, a few reasons for that we still are in working progress for
>> erasure code + geo replication.
>>
>>>> and expect to survive a region outage...
>>>>
>>>> With that I mind I did some experiments (Liberty swift) and it looks
>>>> to me like if you have:
>>>>
>>>> - num_data_frags < num_nodes in (smallest) region
>>>>
>>>> and:
>>>>
>>>> - num_parity_frags = num_data_frags
>>>>
>>>>
>>>> then having a region fail does not result in service outage.
>>
>> Good point but note that the PyECLib v1.0.7 (pinned to Kilo/Liberty
>> stable) still have a problem which cannot decode the original data
>> when all feed fragments are parity frags[1]. (i.e. if set
>> num_parity_frags = num_data frags and then, num_parity_frags comes
>> into proxy for GET request, it will fail at the decoding) The problem
>> was already resolved in the PyECLib/liberasurecode at master
>> branch and current swift master has the PyECLib>=1.0.7 dependencies so
>> if you thought to use the newest Swift, it might be not
>> a matter.
>
>
> Ah right, in my testing I always took down my "1st" region...which will
> have had data fragments therein. For interest I'll try to provoke a
> situation where I have all parity ones to assemble (and see what happens).
>


So tried this out - still works fine. Checking version of pyeclib I see 
Ubuntu 15.10 is giving me:

- Swift 2.5.0
- pyeclib 1.0.8

Hmmm - Canonical deliberately upping the version of pyeclib (shock)? 
Interesting...anyway explains why I cannot get it to fail. However, all 
your other points are noted, and again thanks!

Regards

Mark




More information about the OpenStack-dev mailing list