<div dir="ltr">I'm not sure what version ships with the "official ubuntu packages" - but sometime back last-year/Folsom/1.5 Sam landed a change in partition placement [1] and you stopped needing to contort your zones into made up things [2]. You can have a cluster with a single failure domain if that's what you got and partition placement will still keep do the right thing. There's some advantages to using less zones because the builder has more flexibility moving things around; but if you *actually* have fault tolerant partitions in your deployment (beyond a single server) you want to bake that into your data placement so you can take advantage of it should you *actually* loose a whole zone.<div>
<br></div><div>Zone per server is not needed, offers no advantage on newer versions of swift, may not be a problem, but can cause balancing issues.<div><br></div><div>-Clay</div><div><br><div><div>1. generally referred to as "unique-as-possible"</div>
<div>2. <a href="http://swiftstack.com/blog/2013/02/25/data-placement-in-swift/">http://swiftstack.com/blog/2013/02/25/data-placement-in-swift/</a></div></div></div></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Mon, Sep 16, 2013 at 10:36 PM, Morten Møller Riis <span dir="ltr"><<a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Thank you Clay! That makes sense!<div><br></div><div>I wasn't aware that zones needed to be about the same size. The first 3 devices are going away. As the names might reveal they are running on software raid partitions which is a bad idea. So I guess some of the problem will correct itself when they are removed completely.</div>
<div><br></div><div>I'm using the official ubuntu packages so I'll have to wait until the commit makes it into those.</div><div><br></div><div>At the moment the zones represent different servers (but same rack, DC). Is that a bad idea?</div>
<div><div class="im"><br><div>
<span style="border-collapse:separate;border-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div>Mvh / Best regards</div><div>Morten Møller Riis</div><div>Gigahost ApS</div><div><a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a></div><div><br></div></div></span><br></span><br>
</div>
<br></div><div><div class="h5"><div><div>On Sep 17, 2013, at 4:11 AM, Clay Gerrard <<a href="mailto:clay.gerrard@gmail.com" target="_blank">clay.gerrard@gmail.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr">
So zones 6 & 7 have drastically more weight than 1, 2, 3, 4, 5 - but since you have three replicas unique-as-possible is trying hard to fill up 3, 4, 5 and leaving 6 & 7 with room to spare and nothing to move.<div>
<br></div><div>You want to try to keep your zones roughly the same size overall, and before this change:</div><div><br></div><div><a href="https://review.openstack.org/#/c/41802/" target="_blank">https://review.openstack.org/#/c/41802/</a><br>
</div><div><br></div><div>... it was almost a requirement. If possible, can you checkout that change and try to rebalance this .builder with that new code?</div><div><br></div><div>If not the easiest thing may be to set device 2's weight to zero - I'm pretty sure that will force those partitions to move. You may also want to combine zones 4 and 5, by setting the weight of the device in one zone to 0 and readding the device in the other zone. Good luck!</div>
<div><br></div><div>-Clay</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sun, Sep 15, 2013 at 7:41 PM, Morten Møller Riis <span dir="ltr"><<a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word">This is an example. It's the same for objects/containers/accounts (I recently lower the weight of the first two to 0 in preparation of replacing them).<div>
<br></div><div><div><font face="Lucida Console">$ swift-ring-builder account.builder</font></div><div><font face="Lucida Console">account.builder, build version 56</font></div><div><font face="Lucida Console">262144 partitions, 3 replicas, 7 zones, 15 devices, 306.67 balance</font></div>
<div><font face="Lucida Console">The minimum number of hours before a partition can be reassigned is 1</font></div><div><font face="Lucida Console">Devices: id zone ip address port name weight partitions balance meta</font></div>
<div><font face="Lucida Console"> 0 1 10.44.1.101 6002 md2 0.00 0 0.00</font></div><div><font face="Lucida Console"> 1 2 10.44.1.102 6002 md2 0.00 0 0.00</font></div>
<div><font face="Lucida Console"> 2 3 10.44.1.103 6002 md2 25.00 26431 306.67</font></div><div><font face="Lucida Console"> 5 4 10.44.1.104 6002 sda1 150.00 58928 51.11</font></div>
<div><font face="Lucida Console"> 6 4 10.44.1.104 6002 sdb1 150.00 58928 51.11</font></div><div><font face="Lucida Console"> 7 5 10.44.1.105 6002 sda1 150.00 58928 51.11</font></div>
<div><font face="Lucida Console"> 8 5 10.44.1.105 6002 sdb1 150.00 58929 51.11</font></div><div><font face="Lucida Console"> 9 6 10.44.1.106 6002 sdc1 300.00 65536 -15.97</font></div>
<div><font face="Lucida Console"> 10 6 10.44.1.106 6002 sdd1 300.00 65536 -15.97</font></div><div><font face="Lucida Console"> 11 6 10.44.1.106 6002 sde1 300.00 65536 -15.97</font></div>
<div><font face="Lucida Console"> 12 6 10.44.1.106 6002 sdf1 300.00 65536 -15.97</font></div><div><font face="Lucida Console"> 13 7 10.44.1.107 6002 sda1 300.00 65536 -15.97</font></div>
<div><font face="Lucida Console"> 14 7 10.44.1.107 6002 sdb1 300.00 65536 -15.97</font></div><div><font face="Lucida Console"> 15 7 10.44.1.107 6002 sdc1 300.00 65536 -15.97</font></div>
<div><font face="Lucida Console"> 16 7 10.44.1.107 6002 sdd1 300.00 65536 -15.97</font></div></div><div><font face="Lucida Console"><div>$ swift-ring-builder account.builder rebalance</div>
<div>
<div>Cowardly refusing to save rebalance as it did not change at least 1%.</div></div><div>$</div></font></div><div><br></div><div><br></div><div><br></div><div><div><br><div>
<span style="border-collapse:separate;border-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div>Mvh / Best regards</div><div>Morten Møller Riis</div><div>Gigahost ApS</div><div><a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a></div><div><br></div></div></span><br></span><br>
</div>
<br></div><div><div><div><div>On Sep 14, 2013, at 2:03 AM, Clay Gerrard <<a href="mailto:clay.gerrard@gmail.com" target="_blank">clay.gerrard@gmail.com</a>> wrote:</div><br><blockquote type="cite"><div dir="ltr">
Those two statements do seem in contrast - run `swift-ring-builer account.builder` and check what is the current balance? Can you paste the output? Maybe you have an unbalanced region/zone/server and it's just can't do any better than it is?<div>
<br></div><div>-Clay</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 11:53 PM, Morten Møller Riis <span dir="ltr"><<a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>When adjusting weights and rebalancing I get the message:</div><div><br></div><div>
<div><font face="Lucida Console">NOTE: Balance of 306.68 indicates you should push this</font></div><div><font face="Lucida Console"> ring, wait at least 1 hours, and rebalance/repush.</font></div></div><div><br></div>
<div>However, waiting a couple of hours and running swift-ring-builder account.builder rebalance, it says that it has nothing to rebalance.</div><div><br></div><div><font face="Lucida Console">Cowardly refusing to save rebalance as it did not change at least 1%.</font></div>
<div><br></div><div><br></div><div>What am I getting wrong here?</div><div><br></div><br><div>
<span style="border-collapse:separate;border-spacing:0px"><span style="text-indent:0px;letter-spacing:normal;font-variant:normal;text-align:-webkit-auto;font-style:normal;font-weight:normal;line-height:normal;border-collapse:separate;text-transform:none;font-size:medium;white-space:normal;font-family:Helvetica;word-spacing:0px"><div style="word-wrap:break-word">
<div>Mvh / Best regards</div><span><font color="#888888"><div>Morten Møller Riis</div><div>Gigahost ApS</div><div><a href="mailto:mmr@gigahost.dk" target="_blank">mmr@gigahost.dk</a></div><div><br></div></font></span></div>
</span><br></span><br>
</div>
<br></div><br>_______________________________________________<br>
Mailing list: <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
Post to : <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
Unsubscribe : <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br></div></div></div></div></blockquote></div><br></div>
</blockquote></div><br></div></div></div></div></blockquote></div><br></div>