[dev][infra][qa][tact-sig] Zuul behavior change with Depends-On across queues
For those who haven't seen the more detailed announcement[*] about it, just a quick note that if you get a sudden -2 back from Zuul when approving a change with a Depends-On to a change in a different project which hasn't merged yet, that's likely an indication those projects don't share a dependent queue. It's not a bug, but an intentional clarification of Zuul's enqueuing behavior. For most other Zuul deployments (and even our other Zuul tenants in OpenDev) this is purely cosmetic, but since OpenStack's Zuul tenant is configured to require a positive Verified vote before enqueuing into the gate pipeline, it means some changes may end up unexpectedly needing another pass through check first. It's worth re-evaluating whether or not this "clean check" rule remains a useful requirement for gating. It was added some years ago because a number of gate breaking bugs were traced back to unstable changes being rechecked enough times that eventually they got lucky and were able to merge, and then their instability contributed to destabilizing the integrated gate as a whole. Similarly, changes were being approved without reviewers confirming their jobs were passing first, and this led to additional resource waste. There is a bit of discussion around "blind rechecks" at the PTG this week, and so this topic is related; it might be a good idea to consider it in conjunction with the greater recheck conversation. [*] http://lists.opendev.org/pipermail/service-announce/2022-April/000033.html -- Jeremy Stanley
Hi fungi, a question - is it possible to configure Zuul so that this "clean check" requirement is per-project rather than per-tenant? -yoctozepto On Tue, 5 Apr 2022 at 21:46, Jeremy Stanley <fungi@yuggoth.org> wrote:
For those who haven't seen the more detailed announcement[*] about it, just a quick note that if you get a sudden -2 back from Zuul when approving a change with a Depends-On to a change in a different project which hasn't merged yet, that's likely an indication those projects don't share a dependent queue. It's not a bug, but an intentional clarification of Zuul's enqueuing behavior.
For most other Zuul deployments (and even our other Zuul tenants in OpenDev) this is purely cosmetic, but since OpenStack's Zuul tenant is configured to require a positive Verified vote before enqueuing into the gate pipeline, it means some changes may end up unexpectedly needing another pass through check first. It's worth re-evaluating whether or not this "clean check" rule remains a useful requirement for gating. It was added some years ago because a number of gate breaking bugs were traced back to unstable changes being rechecked enough times that eventually they got lucky and were able to merge, and then their instability contributed to destabilizing the integrated gate as a whole. Similarly, changes were being approved without reviewers confirming their jobs were passing first, and this led to additional resource waste.
There is a bit of discussion around "blind rechecks" at the PTG this week, and so this topic is related; it might be a good idea to consider it in conjunction with the greater recheck conversation.
[*] http://lists.opendev.org/pipermail/service-announce/2022-April/000033.html -- Jeremy Stanley
On 2022-04-06 10:12:08 +0200 (+0200), Radosław Piliszek wrote:
a question - is it possible to configure Zuul so that this "clean check" requirement is per-project rather than per-tenant? [...]
It's part of the gate pipeline's approval requirements here: https://opendev.org/openstack/project-config/src/commit/551b915b8a17cec720ff... I think it could be accomplished by having two gate pipelines, using one for projects which want it but another for projects which don't. Note that any projects sharing a queue would have to rely on the same gate pipeline, and a project could only use one gate or the other (not both). By extension, this also means bifurcation of any project-templates which include a gate pipeline set, so it might result in a lot of duplication for standard templates. For consistency's sake though, it would probably be best for the tenant to have only a single gate pipeline. -- Jeremy Stanley
participants (2)
-
Jeremy Stanley
-
Radosław Piliszek