On Tue, Sep 3, 2019 at 10:43 AM Gabriele Cerami <gcerami@redhat.com> wrote:
Hi,

this weekend, and outage caused trunk.rdoprojects.org servers to become
unreachable.
As main effect, all tripleo ci jobs were unable to download and install
dlrn repositories for the needed hashes and failed.

The outage has been resolved yesterday but there's a problem with DNS
propagation and we're still seeing DNS queries returning incorrect IPs,
and as a result, jobs are not consistently passing.
You may see problems dowloading repos, building changes, installing
packages

We would advise to limit the rechecks until we are sure the DNS results
are stable.

Thanks.

Just adding a little more clarity.  You can see the pass rate of TripleO jobs start to drop on 8/31 and recover on 9/3 in the screenshot.   We have not yet fully recovered quite yet, we will update this thread when that is the case.

Thanks!