Open Stack

For those wondering why operators can’t always upgrade sooner, I can add a little bit of color:  In our clouds, we have a couple vendors (one network plugin, one cinder driver) and those vendors typically are 1-3 releases behind ‘cutting edge’.  By the time they support the version we want to go to, that version is almost end-of-life, which can make things interesting.  On the bright side, by then there are usually some helpful articles out there about the issues upgrading from A to B.

As for the planning time required - for us, it mostly boils down to testing or doing it at a time when some amount of disruption is at least somewhat tolerable.  For example, for online retail folks like me, upgrading between October and December would be out of the question due to the busy shopping season that is almost upon us.

I will say that I was very impressed with some of the containerized demos that were given at the Summit last week.  I plan to look into some containerized options next year which hopefully could ease the upgrade process for us.  Still, there is a lot of testing involved, coordination with 3rd parties, and other stars that would still have to align.

At Overstock we have also started maintaining two completely separate production clouds and have orchestration to build/rebuild VMs on either one as needed.  Most the time we spread all our apps across both clouds.  So next year when we get the chance to upgrade cloud A, we can either rebuild things on B, or just shut them down while we rebuild A.  Then we would repeat on cloud B.  Hopefully this eases our upgrade process…at least that’s what we are hoping!

My 2 cents.  Thanks



On Nov 14, 2017, at 4:44 PM, John Dickinson <me at not.mn<mailto:me at not.mn>> wrote:



On 14 Nov 2017, at 15:18, Mathieu Gagné wrote:

On Tue, Nov 14, 2017 at 6:00 PM, Fox, Kevin M <Kevin.Fox at pnnl.gov<mailto:Kevin.Fox at pnnl.gov>> wrote:
The pressure for #2 comes from the inability to skip upgrades and the fact that upgrades are hugely time consuming still.

If you want to reduce the push for number #2 and help developers get their wish of getting features into users hands sooner, the path to upgrade really needs to be much less painful.


+1000

We are upgrading from Kilo to Mitaka. It took 1 year to plan and
execute the upgrade. (and we skipped a version)
Scheduling all the relevant internal teams is a monumental task
because we don't have dedicated teams for those projects and they have
other priorities.
Upgrading affects a LOT of our systems, some we don't fully have
control over. And it can takes months to get new deployment on those
systems. (and after, we have to test compatibility, of course)

So I guess you can understand my frustration when I'm told to upgrade
more often and that skipping versions is discouraged/unsupported.
At the current pace, I'm just falling behind. I *need* to skip
versions to keep up.

So for our next upgrades, we plan on skipping even more versions if
the database migration allows it. (except for Nova which is a huge
PITA to be honest due to CellsV1)
I just don't see any other ways to keep up otherwise.

?!?!

What does it take for this to never happen again? No operator should need to plan and execute an upgrade for a whole year to upgrade one year's worth of code development.

We don't need new policies, new teams, more releases, fewer releases, or anything like that. The goal is NOT "let's have an LTS release". The goal should be "How do we make sure Mattieu and everyone else in the world can actually deploy and use the software we are writing?"

Can we drop the entire LTS discussion for now and focus on "make upgrades take less than a year" instead? After we solve that, let's come back around to LTS versions, if needed. I know there's already some work around that. Let's focus there and not be distracted about the best bureaucracy for not deleting two-year-old branches.


--John



/me puts on asbestos pants


--
Mathieu

_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators


________________________________

CONFIDENTIALITY NOTICE: This message is intended only for the use and review of the individual or entity to which it is addressed and may contain information that is privileged and confidential. If the reader of this message is not the intended recipient, or the employee or agent responsible for delivering the message solely to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify sender immediately by telephone or return email. Thank you.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20171115/6cf091f3/attachment.html>