Open Stack

+1 to " If there are no contributors for an LTS release, there will be
no LTS release. If there *are* contributors, then we'll find a way to
make some sort of LTS model work within the other constraints we
have."

+10000 to let's get some folks who are interested a place to
collaborate and talk to each other and get things up and running. That
would be the first step. If it ends up in LTS and skip-level upgrades
for all projects, we all win!

That's a destination, not the first step!

Thanks,
Dims

On Sun, Nov 12, 2017 at 6:04 AM, Doug Hellmann <doug at doughellmann.com> wrote:
> Excerpts from Clint Byrum's message of 2017-11-11 08:41:15 -0800:
>> Excerpts from Doug Hellmann's message of 2017-11-10 13:11:45 -0500:
>> > Excerpts from Clint Byrum's message of 2017-11-08 23:15:15 -0800:
>> > > Excerpts from Samuel Cassiba's message of 2017-11-08 08:27:12 -0800:
>> > > > On Tue, Nov 7, 2017 at 3:28 PM, Erik McCormick
>> > > > <emccormick at cirrusseven.com> wrote:
>> > > > > Hello Ops folks,
>> > > > >
>> > > > > This morning at the Sydney Summit we had a very well attended and very
>> > > > > productive session about how to go about keeping a selection of past
>> > > > > releases available and maintained for a longer period of time (LTS).
>> > > > >
>> > > > > There was agreement in the room that this could be accomplished by
>> > > > > moving the responsibility for those releases from the Stable Branch
>> > > > > team down to those who are already creating and testing patches for
>> > > > > old releases: The distros, deployers, and operators.
>> > > > >
>> > > > > The concept, in general, is to create a new set of cores from these
>> > > > > groups, and use 3rd party CI to validate patches. There are lots of
>> > > > > details to be worked out yet, but our amazing UC (User Committee) will
>> > > > > be begin working out the details.
>> > > > >
>> > > > > Please take a look at the Etherpad from the session if you'd like to
>> > > > > see the details. More importantly, if you would like to contribute to
>> > > > > this effort, please add your name to the list starting on line 133.
>> > > > >
>> > > > > https://etherpad.openstack.org/p/SYD-forum-upstream-lts-releases
>> > > > >
>> > > > > Thanks to everyone who participated!
>> > > > >
>> > > > > Cheers,
>> > > > > Erik
>> > > > >
>> > > > > __________________________________________________________________________
>> > > > > OpenStack Development Mailing List (not for usage questions)
>> > > > > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> > > >
>> > > > In advance, pardon the defensive tone. I was not in a position to
>> > > > attend, or even be in Sydney. However, as this comes across the ML, I
>> > > > can't help but get the impression this effort would be forcing more
>> > > > work on already stretched teams, ie. deployment-focused development
>> > > > teams already under a crunch as contributor count continues to decline
>> > > > in favor of other projects inside and out of OpenStack.
>> > > >
>> > >
>> > > I suspect if LTS's become a normal part of OpenStack, most deployment
>> > > projects will decline to support the interim releases. We can infer this
>> > > from the way Ubuntu is used. This might actually be a good thing for the
>> > > chef OpenStack community. 3 out of 3.5 of you can focus on the LTS bits,
>> > > and the 0.5 person can do some best effort to cover the weird corner
>> > > case of "previous stable release to master".
>> > >
>> > > The biggest challenge will be ensuring that the skip-level upgrades
>> > > work. The current grenade based upgrade jobs are already quite a bear to
>> > > keep working IIRC. I've not seen if chef or any of the deployment projects
>> > > test upgrades like that.
>> > >
>> > > However, if people can stop caring much about the interim releases and
>> > > just keep "previous LTS to master" upgrades working, then that might be
>> > > good for casual adoption.
>> > >
>> > > Personally I'd rather we make it easier to run "rolling release"
>> > > OpenStack. Maybe we can do that if we stop cutting stable releases every
>> > > 6 months.
>> > >
>> >
>> > We should stop calling what we're talking about "LTS". It isn't
>> > going to match the expectations of anyone receiving LTS releases
>> > for other products, at least at first. Perhaps "Deployer Supported"
>> > or "User Supported" are better terms for what we're talking about.
>> >
>>
>> I believe this state we're in is a stop-gap on the way to the full
>> LTS. People are already getting stuck. We're going to help them stay stuck
>> by upstreaming bug fixes.  We should be mindful of that and provide a way
>> to get less-stuck. The LTS model from other projects has proven quite
>> popular, and it would make sense for us to embrace it if our operators
>> are hurting with the current model, which I believe they are.
>>
>> > In the "LTS" room we did not agree to stop cutting stable releases
>> > or to start supporting upgrades directly from N-2 (or older) to N.
>> > Both of those changes would require modifying the support the
>> > existing contributor base has committed to provide.
>> >
>>
>> Thanks, I am just inferring those things from what was agreed on. However,
>> It would make a lot of sense to discuss the plans for the future, even
>> if we don't have data from the present proposal.
>>
>> > Fast-forward upgrades will still need to run the migration steps
>> > of each release in order, one at a time. The team working on that
>> > is going to produce a document describing what works today so we
>> > can analyze it for ways to improve the upgrade experience, for both
>> > fast-forward and "regular" upgrades.  That was all discussed in a
>> > separate session.
>> >
>>
>> We are what we test. If we're going to test fast-forwards, how far into
>> the past do we test? It makes a lot of sense to pick a release every few
>> cycles and say "we're going to support fast-forwards from this point
>> to master for the next 2 years". Then users can safely clamber their
>> way onto that release, and spend the next 2 years paying down debt,
>> knowing they have a chance for a skip-level upgrade. It's not a model
>> I recommend, but I do think it's the way many companies work today.
>
> That seems like a good approach. When the team working on fast-forward
> upgrades finishes the process document it should become more clear how
> we could potentially start running upgrade tests, and what changes might
> be needed to support doing so. Having some tags for supporting
> fast-forward and skip-level upgrades would be good, too, so deployers
> can understand easily what sort of upgrade models are supported by
> different projects.
>
>> Anyway, let me be clear. I think we should listen to our operators. I
>> wasn't at the forum, so I couldn't do that there. But what I have seen
>> at every org I've been at, is one of two things:
>>
>> 1) Running close to master, merging in and deploying on a regular
>>    cadence. Confidence in the pipeline and actual CD nirvana. This
>>    is rare, and requires commitment and trust at multiple levels. But
>>    it's the way I personally recommend people run OpenStack.
>>
>> 2) Falling 1 release behind the moment they deploy the stable release,
>>    and then falling further and further behind from there.
>>
>> This is my own anecdotal evidence, but we have some data to back it up
>> too. As of the April survey[1] we see that _most_ users were on Mitaka or
>> earlier. Mitaka went EOL in _April_. So users are getting run over by the
>> EOL policy, and no amount of "you should upgrade faster" is going to fix
>> that. Perhaps we should slow down the tagging. Some have suggested going
>> to once per year. That makes some sense to me. Especially if we continue
>> to support CD'ing from master.
>
> The problem with stable maintenance isn't so much the number of
> releases involved. Stable branches become more brittle as they age,
> because the underlying OS changes with upgrades in dependencies.
> We discussed a bunch of different ways to address that, from
> containers to static images to managing a "frozen" repo of packages
> to install during tests. Any decisions on what approach to use will
> be left to the people actually doing the work, along with the folks
> maintaining the tools used (in particular, the infra and QA teams).
>
> In preparation for the summit I went back through all of the notes
> I could find about stable branches from previous summits. The first
> mention of stable branches I found was at the Folsom summit, and
> that was a discussion of doing stable releases "more often," which
> implies that we had the stable branches at least as early as Essex.
> I wasn't around before Folsom, so I'm not sure when they actually
> started.  The first mention of an LTS release I found was the Juno
> summit, which was later than I expected.
>
> No one in the room disputed the assertion that what we're doing for
> stable releases is insufficient.  We are all trying to listen to
> users' needs.  Continuing to just say "we should do LTS releases"
> however doesn't acknowledge the other long standing fact, which is
> that over all of the time we have talked about it we have had *no
> contributors willing to actually support an LTS release model
> upstream.*
>
> We act like people have been saying "no, you are not allowed to
> maintain branches for longer than the stable team says is OK," or
> "no, we'll never provide an LTS release," but that's not how our
> community works. The policies are set by the contributors. If there
> are no contributors for an LTS release, there will be no LTS release.
> If there *are* contributors, then we'll find a way to make some
> sort of LTS model work within the other constraints we have.
>
> It seems now that we have people saying they would do some amount
> of maintenance work (probably less than we try to do for stable
> branches under our current model), if they could. The first change,
> what has been proposed, is to give them a place to do the work.
> Then we'll see if anyone actually does it, and if so we can plan
> further improvements.
>
>> Then we can ask the operators if they would prefer that we stop EOL'ing
>> things out from under them. We can make it a community goal to have a
>> "feature" that is "you can upgrade from the last one" and have "the last
>> one" be something older than 6 months, maybe even older than 1 year.
>>
>> [1] https://www.openstack.org/assets/survey/April2017SurveyReport.pdf
>>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators



-- 
Davanum Srinivas :: https://twitter.com/dims