[swift] Rolling upgrade, any version relationships?
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are: - If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)? - If not is there a preferred version to jump to first? - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? regards Mark
Ping! This question seems to have slipped in the global bitstream unnoticed :-) On 11/03/20 3:27 pm, Mark Kirkwood wrote:
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are:
- If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)?
- If not is there a preferred version to jump to first?
- Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time?
regards
Mark
Hi In my case, I upgraded my cluster from ocata to train directly, and It was very successful. - Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time? -> I think.. It's ok but It is better to take a short period. Regards Seongsoo
On Mar 18, 2020, at 1:22 PM, Mark Kirkwood <mark.kirkwood@catalyst.net.nz> wrote:
Ping! This question seems to have slipped in the global bitstream unnoticed :-)
On 11/03/20 3:27 pm, Mark Kirkwood wrote:
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are:
- If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)?
- If not is there a preferred version to jump to first?
- Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time?
regards
Mark
Hi all, flagging this again as it would be great to have a definite answer to these questions! Thanks! On 11/03/20 3:27 pm, Mark Kirkwood wrote:
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are:
- If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)?
- If not is there a preferred version to jump to first?
- Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time?
regards
Mark
On 4/14/20 6:00 PM, Mark Kirkwood wrote:
Hi all, flagging this again as it would be great to have a definite answer to these questions! Thanks!
On 11/03/20 3:27 pm, Mark Kirkwood wrote:
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are:
- If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)?
- If not is there a preferred version to jump to first?
- Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time?
regards
Mark
Hi Mark, It should be perfectly fine upgrading from 2.7.0 to latest. If possible, you'll want to upgrade backend servers (object, container, account) before proxies, though I've done many upgrades where that wasn't possible (because all nodes were running all services). We've never (to my knowledge, anyway) had anything like a checkpoint release. Having a long in-progress window for the upgrade increases the likelihood that you'll uncover some heretofore unknown upgrade bug, but it should be mostly OK. There may be some noisy logging, due to changes in the replication protocol for example, but as long as the cluster was healthy going into the upgrade, I wouldn't worry too much. Shorter is generally better, though, especially if your cluster topology requires that some proxies get upgraded before all the backend servers are upgraded. Tim
On 23/05/20 5:29 am, Tim Burke wrote:
On 4/14/20 6:00 PM, Mark Kirkwood wrote:
Hi all, flagging this again as it would be great to have a definite answer to these questions! Thanks!
On 11/03/20 3:27 pm, Mark Kirkwood wrote:
Hi, we are looking at upgrading our 2.7.0 Swift cluster. In the past I've modeled this on a dev system by upgrading storage nodes one by one (using 2.17 as the target version). This seemed to work well - I deliberately left the cluster half upgraded for an extended period to test for any cross version weirdness (didn't see any). However I'm wanting to check that I have not missed something important. So my questions are:
- If upgrading from 2.7.0 is it safe to just grab the latest version (e.g 2.23)?
- If not is there a preferred version to jump to first?
- Is it ok for the upgrade to take extended time (e.g weeks) and therefore be running with some new and some old storage nodes for that time?
regards
Mark
Hi Mark,
It should be perfectly fine upgrading from 2.7.0 to latest. If possible, you'll want to upgrade backend servers (object, container, account) before proxies, though I've done many upgrades where that wasn't possible (because all nodes were running all services). We've never (to my knowledge, anyway) had anything like a checkpoint release.
Having a long in-progress window for the upgrade increases the likelihood that you'll uncover some heretofore unknown upgrade bug, but it should be mostly OK. There may be some noisy logging, due to changes in the replication protocol for example, but as long as the cluster was healthy going into the upgrade, I wouldn't worry too much. Shorter is generally better, though, especially if your cluster topology requires that some proxies get upgraded before all the backend servers are upgraded.
Thanks Tim, exactly what I needed to know! Yeah, we are able to upgrade all our storage nodes (account, container, object) before the proxies, so fingers crossed.
participants (3)
-
Mark Kirkwood
-
SeongSoo Cho
-
Tim Burke