[Openstack] Rebalancing issue
Morten Møller Riis
mmr at gigahost.dk
Tue Sep 17 05:36:25 UTC 2013
Thank you Clay! That makes sense!
I wasn't aware that zones needed to be about the same size. The first 3 devices are going away. As the names might reveal they are running on software raid partitions which is a bad idea. So I guess some of the problem will correct itself when they are removed completely.
I'm using the official ubuntu packages so I'll have to wait until the commit makes it into those.
At the moment the zones represent different servers (but same rack, DC). Is that a bad idea?
Mvh / Best regards
Morten Møller Riis
Gigahost ApS
mmr at gigahost.dk
On Sep 17, 2013, at 4:11 AM, Clay Gerrard <clay.gerrard at gmail.com> wrote:
> So zones 6 & 7 have drastically more weight than 1, 2, 3, 4, 5 - but since you have three replicas unique-as-possible is trying hard to fill up 3, 4, 5 and leaving 6 & 7 with room to spare and nothing to move.
>
> You want to try to keep your zones roughly the same size overall, and before this change:
>
> https://review.openstack.org/#/c/41802/
>
> ... it was almost a requirement. If possible, can you checkout that change and try to rebalance this .builder with that new code?
>
> If not the easiest thing may be to set device 2's weight to zero - I'm pretty sure that will force those partitions to move. You may also want to combine zones 4 and 5, by setting the weight of the device in one zone to 0 and readding the device in the other zone. Good luck!
>
> -Clay
>
>
>
> On Sun, Sep 15, 2013 at 7:41 PM, Morten Møller Riis <mmr at gigahost.dk> wrote:
> This is an example. It's the same for objects/containers/accounts (I recently lower the weight of the first two to 0 in preparation of replacing them).
>
> $ swift-ring-builder account.builder
> account.builder, build version 56
> 262144 partitions, 3 replicas, 7 zones, 15 devices, 306.67 balance
> The minimum number of hours before a partition can be reassigned is 1
> Devices: id zone ip address port name weight partitions balance meta
> 0 1 10.44.1.101 6002 md2 0.00 0 0.00
> 1 2 10.44.1.102 6002 md2 0.00 0 0.00
> 2 3 10.44.1.103 6002 md2 25.00 26431 306.67
> 5 4 10.44.1.104 6002 sda1 150.00 58928 51.11
> 6 4 10.44.1.104 6002 sdb1 150.00 58928 51.11
> 7 5 10.44.1.105 6002 sda1 150.00 58928 51.11
> 8 5 10.44.1.105 6002 sdb1 150.00 58929 51.11
> 9 6 10.44.1.106 6002 sdc1 300.00 65536 -15.97
> 10 6 10.44.1.106 6002 sdd1 300.00 65536 -15.97
> 11 6 10.44.1.106 6002 sde1 300.00 65536 -15.97
> 12 6 10.44.1.106 6002 sdf1 300.00 65536 -15.97
> 13 7 10.44.1.107 6002 sda1 300.00 65536 -15.97
> 14 7 10.44.1.107 6002 sdb1 300.00 65536 -15.97
> 15 7 10.44.1.107 6002 sdc1 300.00 65536 -15.97
> 16 7 10.44.1.107 6002 sdd1 300.00 65536 -15.97
> $ swift-ring-builder account.builder rebalance
> Cowardly refusing to save rebalance as it did not change at least 1%.
> $
>
>
>
>
> Mvh / Best regards
> Morten Møller Riis
> Gigahost ApS
> mmr at gigahost.dk
>
>
>
>
> On Sep 14, 2013, at 2:03 AM, Clay Gerrard <clay.gerrard at gmail.com> wrote:
>
>> Those two statements do seem in contrast - run `swift-ring-builer account.builder` and check what is the current balance? Can you paste the output? Maybe you have an unbalanced region/zone/server and it's just can't do any better than it is?
>>
>> -Clay
>>
>>
>> On Thu, Sep 12, 2013 at 11:53 PM, Morten Møller Riis <mmr at gigahost.dk> wrote:
>> When adjusting weights and rebalancing I get the message:
>>
>> NOTE: Balance of 306.68 indicates you should push this
>> ring, wait at least 1 hours, and rebalance/repush.
>>
>> However, waiting a couple of hours and running swift-ring-builder account.builder rebalance, it says that it has nothing to rebalance.
>>
>> Cowardly refusing to save rebalance as it did not change at least 1%.
>>
>>
>> What am I getting wrong here?
>>
>>
>> Mvh / Best regards
>> Morten Møller Riis
>> Gigahost ApS
>> mmr at gigahost.dk
>>
>>
>>
>>
>>
>> _______________________________________________
>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> Post to : openstack at lists.openstack.org
>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130917/9a6b21bb/attachment.html>
More information about the Openstack
mailing list