---- On Fri, 01 Jul 2022 01:46:23 -0500 Matthias Runge <mrunge@matthias-runge.de> wrote ---
On 30/06/2022 20:06, Dan Smith wrote:
Or vice versa, if there are 20 rechecks for 2 patches, even if neither of them are bare, it's still weird and smth worth reconsidering from project perspective.
I think the idea is to create a culture of debugging and record keeping. Yes, I would expect after a few rechecks that maybe the root causes would be addressed in this case, but the first step in doing that is identifying the problem and making note of it.
Right, that is the goal. Asking for a message at least sets the expectation that people are looking at the reasons for the fails. Just because they don't doesn't mean they aren't, or don't care, but I think it helps reinforce the desired behavior. If nothing else, it also helps observers realize "huh, I've seen a bunch of rechecks about $reason lately, maybe we should look at that".
So, what happens with the script, when you add 2 comments, one: "network error during package install, let's try again" and the next message "recheck".
In this case, you can always mentione the "recheck network error during package install, let's try again" or if you have added a lenthy text for failure and then want to recheck you can add a one line sumamry during recheck. Overall idea is not to literally count the bare recheck but to build a habbit among us that we should look at the failure before we just do recheck.
In my understanding, that would count as recheck without reason given. (by the script). Maybe it's worth to document how to give a better proof that someone looked into the logs and tried to get to the root cause of a previous CI failure?
I think Dan has written a nice document about it including how to debug the failure, - https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-tes... We welcome everyone to extend it to have more detail or example if any case specific to projects and it is not covered. -gmann
The other issue I see here is that with CI being flaky, chances seem to get better when doing a recheck.
An extreme example: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/844519 required 8 rechecks, no changes in the patch itself, and no dependencies. The CI failed always in different checks.
Matthias