Open Stack

Hello,

We were working on expanding one of our clusters (Ussuri on Ubuntu
18.04) and are wondering about the rebalance behavior of
swift-ring-builder. When we run it in debug mode on a 15/4 EC ring, we
see this message about "Unable to finish rebalance plan after 2
attempts" and are seeing 100% partitions reassigned.

DEBUG: Placed 10899/2 onto dev r1z3-10.40.48.72/d10
DEBUG: Placed 2183/3 onto dev r1z5-10.40.48.76/d11
DEBUG: Placed 1607/1 onto dev r1z3-10.40.48.70/d28
DEBUG: Assigned 32768 parts
DEBUG: Gather start is 10278 (Last start was 25464)
DEBUG: Unable to finish rebalance plan after 2 attempts
Reassigned 32768 (100.00%) partitions. Balance is now 63.21.
Dispersion is now 0.00
-------------------------------------------------------------------------------
NOTE: Balance of 63.21 indicates you should push this
      ring, wait at least 1 hours, and rebalance/repush.
-------------------------------------------------------------------------------

Moving 100% seems scary, what does that mean in this situation? Is
this message because 1 fragment from every partition is moved and that
is the most that it can do per rebalance because they are technically
the same partition?
When we compare the swift-ring-builder output (partitions per device)
between rebalances we can see some partitions move each time until we
no longer see the push/wait/rebalance message again. So it's not
really moving 100% partitions.

Reid