Open Stack

On 4 September 2014 08:13, Robert Collins <robertc at robertcollins.net> wrote:
> On 3 September 2014 23:50, Nejc Saje <nsaje at redhat.com> wrote:
>
> Forgive my slowness :).
>
>> Disclaimer: in Ironic terms, node = conductor, key = host
>
> Sadly not inside the hash_ring code :/. host == conductor, key == data.
>
>> The test I linked fails with Ironic hash ring code (specifically the part
>> that tests consistency). With 1000 keys being mapped to 10 nodes, when you
>> add a node:
>> - current ceilometer code remaps around 7% of the keys (< 1/#nodes)
>> - Ironic code remaps > 90% of the keys
>
> Ok thats pretty definitive and definitely not intended. However
> remapping 7% when adding 10% capacity is also undesirable - we'd like
> to remap 1/11 -> +-9%.
>
>> The problem lies in the way you build your hash ring[1]. You initialize a
>> statically-sized array and divide its fields among nodes. When you do a
>> lookup, you check which field in the array the key hashes to and then return
>> the node that that field belongs to. This is the wrong approach because when
>> you add a new node, you will resize the array and assign the fields
>> differently, but the same key will still hash to the same field and will
>> therefore hash to a different node.
>
> You're referring to part2host where we round-robin using mod to map a
> partition (hash value of key) to a host(conductor). Then when we add a
> conductor this entire map scales out - yup I see the issue.
>
> Have you filed a bug for this?

https://review.openstack.org/118932 has an equivalent test, which
failed before the fixes were applied to the Ironic hash ring
implementation.

Cheers,
Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud