Re: [Release-job-failures] Release of openstack/governance failed
zuul@openstack.org wrote:
Build failed.
- release-openstack-python http://logs.openstack.org/21/21a5154f6a4d1c317d7b2fe084ecd919ed8cb93e/releas... : POST_FAILURE in 3m 54s - announce-release announce-release : SKIPPED - propose-update-constraints propose-update-constraints : SKIPPED
Issue: ------ The release jobs for openstack/governance failed while processing the recently-added 0.4.0 tag. During collection of the built artifacts, the following command: /usr/bin/rsync --delay-updates -F --compress --archive --rsh=/usr/bin/ssh -S none -o Port=22 --out-format=<<CHANGED>>%i %n%L zuul@104.130.6.123:/home/zuul/zuul-output/artifacts/ /var/lib/zuul/builds/16d67fd9d85342bfabfd82292183e76c/work/artifacts/ failed host key verification: ED25519 host key for 104.130.6.123 has changed and you have requested strict checking. Impact: ------- The 0.4.0 tarball was never copied to tarballs.openstack.org, and it was not uploaded to Pypi either. Once the issue is resolved we should either re-enqueue that reference to re-trigger the job, or push a 0.4.1. -- Thierry Carrez (ttx)
On 2019-03-14 12:10:00 +0100 (+0100), Thierry Carrez wrote: [...]
During collection of the built artifacts, the following command:
/usr/bin/rsync --delay-updates -F --compress --archive --rsh=/usr/bin/ssh -S none -o Port=22 --out-format=<<CHANGED>>%i %n%L zuul@104.130.6.123:/home/zuul/zuul-output/artifacts/ /var/lib/zuul/builds/16d67fd9d85342bfabfd82292183e76c/work/artifacts/
failed host key verification:
ED25519 host key for 104.130.6.123 has changed and you have requested strict checking. [...]
Based on the log this was the Zuul executor trying to rsync files off the job node over SSH, surprised to find the SSH host key on the job node was suddenly different. We see this symptom from time to time when Nova loses track of a VM somewhere in the provider and so the IP address remains in use but also gets reassigned to a new VM for our boot request. This is almost certainly a transient failure and the build was merely "lucky" enough to win at rogue instance roulette. -- Jeremy Stanley
On 2019-03-14 14:18:35 +0000 (+0000), Jeremy Stanley wrote:
On 2019-03-14 12:10:00 +0100 (+0100), Thierry Carrez wrote: [...]
During collection of the built artifacts, the following command:
/usr/bin/rsync --delay-updates -F --compress --archive --rsh=/usr/bin/ssh -S none -o Port=22 --out-format=<<CHANGED>>%i %n%L zuul@104.130.6.123:/home/zuul/zuul-output/artifacts/ /var/lib/zuul/builds/16d67fd9d85342bfabfd82292183e76c/work/artifacts/
failed host key verification:
ED25519 host key for 104.130.6.123 has changed and you have requested strict checking. [...]
Based on the log this was the Zuul executor trying to rsync files off the job node over SSH, surprised to find the SSH host key on the job node was suddenly different. We see this symptom from time to time when Nova loses track of a VM somewhere in the provider and so the IP address remains in use but also gets reassigned to a new VM for our boot request. This is almost certainly a transient failure and the build was merely "lucky" enough to win at rogue instance roulette.
After manually reenqueuing its tag in Zuul, the openstack/governance 0.4.0 release jobs ran to completion successfully as expected, so it seems this was not a systemic issue. -- Jeremy Stanley
participants (2)
-
Jeremy Stanley
-
Thierry Carrez