On 2021-01-08 14:42:14 +0100 (+0100), Herve Beraud wrote: > Also notice that yesterday we met an AFS issue [1] during the merge of the > following patches: > > - https://review.opendev.org/c/openstack/releases/+/769325 > - https://review.opendev.org/c/openstack/releases/+/769322 > - https://review.opendev.org/c/openstack/releases/+/769324 > > The problem was that afs an server got stuck in a pathological way from > 05:50 utc today until 16:10 utc when we hard rebooted the server instance. The patches you reference merged later, so wasn't directly impacted by write failures (the jobs actually succeeded). > The consequence of this is that the related tarballs haven't been published: > > - https://tarballs.opendev.org/openstack/kolla/?C=M;O=D > - https://tarballs.opendev.org/openstack/kolla-ansible/?C=M;O=D > - https://tarballs.opendev.org/openstack/kayobe/?C=M;O=D > > And so the RDO CI fail to build them for train, ussuri and victoria: > > - https://review.rdoproject.org/r/#/c/31499/ > - https://review.rdoproject.org/r/#/c/31498/ > - https://review.rdoproject.org/r/#/c/31497/ > > So I think we need to reenqueue these jobs too. [...] The files were written into the tarballs volume just fine, but the read-only replicas which back the tarballs.opendev.org site hadn't been synchronized. I found a stuck process (waiting since yesterday for a response from the server which had previously died) and killed it to get the periodic synchronization of the read-only volumes working again so, the site is no longer stale and has those releases on it now as of 15:15:40 UTC today. -- Jeremy Stanley -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 833 bytes Desc: not available URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210108/27b9ca02/attachment.sig>