Open Stack

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

my name was called out here, so I think I need to present my thoughts
on the matter. :) And sorry for writing too many words below.

===

As a disclaimer, I was not the one to support long support term for
stable branches. On the previous summit, I spoke up on sessions about
stable branches to avoid raising the bar even more, and even for
lowering it. One of my arguments against long stable branch support
term is that with 6 month release cycle in mind, we end up with 3, 4,
5, ... stable branches to support in parallel, and it's just too much
to take for anyone. Unfortunately, the session decided to keep 15
months support cycle for Juno.

I really believe we should support one stable release for the most of
our time, leaving ~1 month overlap right after the latest major
release (probably till the first minor .1 update).

That being said, I don't support the way it's proposed here to just
drop a stable branch due to some intermittent issues with it, instead
of fixing it, and the process around it. We made a public statement on
support term, and it would be irresponsible to break it.

===

I also disagree that stable-maint team does not step in to fixing the
issues. There are people who are actively tracking the branches and
doing appropriate fixes here and there. I'm sorry to hear that some of
you still need to step in from time to time to fix stable branches,
but I want to assure you you're not alone in suffering through it.

There were changes in how the team works recently:
- - we actually care about periodic jobs that report issues in project
trees on daily basis;
- - we have a common etherpad to track current stable branch state;
- - we have champions assigned to each of stable branches;
- - we have a generally better fix cycle for issues that arise in stable
branches.

We still have issues to solve, for example the way we handle stable
branch freezes seems to be not optional. We should also be more
proactive to cap library versions right after new release (I'll try to
follow up on that after the current issues are solved). I think we
should also reduce support cycle, especially since there seems to be
little interest from d/s consumers to work collaboratively on those
branches.

===

Part of the problem with stable branches is that we still run them
against bleeding edge. It's true for both external libraries (recent
eventlet, boto issues) and our own projects (clients, oslo libraries,
tempest). This should not be the case, and I'm glad there are people
who are working on pinning pypi libraries on stable branches. Oslo
project now also understand that they should maintain stable branches
for all their libraries (it was not generally the case half a year ago).

I know that sometimes we try to attempt branchlesness, for it's easier
to live without backports, and with due care, it usually works. But it
still fails sometimes, because in reality our backwards compatibility
claims are more of 'best effort' kind, and people do mistakes.

So I hope that once the current issues are solved, we move forward
with pinning all the version for pypi libraries.

I'm also open to help with the effort, though I need to admit that I'm
not involved in current 'qa' program, and so I have no deep
understanding on how it all actually works.

===

I see several problems shown by the gate issues. First, we obviously
lack on communication between 'qa' and 'stable-maint' people. F.e. QA
people were not aware of stable-tracker etherpad maintained by
stable-maint team; while I personally was not aware of who is working
on unwedging the gate, or even whether the issue was critical or even
existed (the first report I got from periodic-checks that showed the
issues comes from today at the night, my local time).

I've updated stable-branch wiki page with contacts of stable branch
champions, so that next time you know whom to reach.

I'd also like to say that you can always reach me for any stable
branch related issues no matter whether it's Icehouse or Juno. The
quicker we get a ping from affected parties, the quicker we start
looking into it.

I'll add #openstack-qa channel into my auto-connect list, I hope it
will help to raise awareness about gate issues on my side, and about
stable process on yours. You're also invited to #openstack-stable were
people show up (sometimes).

===

I would also like to note that stable-maint team lacks a lot of ACLs
to actually resolve any of those issues. Specifically,

- - we don't have +2 for devstack, so we can't merge several patches
that are ready to be pushed to unwedge the branch, specifically:
- - https://review.openstack.org/#/c/154252/
- - https://review.openstack.org/#/c/154217/;

- - we don't have +2 for tempest since it's branch-less, and we are not
cores there;

- - we don't have +2 at requirements/master that sometimes requires
ninja merges in case a gate failure is introduced by a new external
library release.

What we currently have is +2s for project stable branches (neutron,
nova, trove, +requirements). It's not enough to be effective in
solving those kind of issues. So I would be glad if the issues
mentioned above are considered by people who are in charge of those
ACLs, and we actually get +2 hammer there (or at least some of us).

===

PS: it's sad to hear from grenade contributors that they are not
interested in stable branches. If anything, grenade is for those
branches, and so I would expect everyone to be on board with helping
with stable maintenance.

/Ihar
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJU2e05AAoJEC5aWaUY1u57uOMIAOTv6umwEHsSSUkpK8E4Ftx/
xh3fqa6t0+16ate8Tj6M3hreB9YhT0oCMt4wuTI5oGN37aVvPqC12uP5CKnwptAR
ESzN98hhj4j2LouMI7gw1XTYbtqFyzP2zANlrlLfTKCWy215SI2S5g6XZRtE4c4T
e99lnGL4lNcKy2MXOvNiSVXWQcSICZGUx4iIEcfVQyfdnV/A/GTHVRItqU/zN7wj
ZGsR1h+xe4Ccb3J/V7xxTn6P6nj8ZKXpTFbFGywnoIfeQ1QlXcPK5rwjvZC5m3oV
StQGH6IsqYCGoQjBRM7OEWfiE0Y6RbB2PULiEAbmllnw/ZMczgPM2cZgA/CXfro=
=v0Sg
-----END PGP SIGNATURE-----