[TripleO][CI] Outage on the rdoprojects.org server is causing jobs to fail

Wesley Hayutin whayutin at redhat.com
Tue Sep 3 18:14:07 UTC 2019


On Tue, Sep 3, 2019 at 10:43 AM Gabriele Cerami <gcerami at redhat.com> wrote:

> Hi,
>
> this weekend, and outage caused trunk.rdoprojects.org servers to become
> unreachable.
> As main effect, all tripleo ci jobs were unable to download and install
> dlrn repositories for the needed hashes and failed.
>
> The outage has been resolved yesterday but there's a problem with DNS
> propagation and we're still seeing DNS queries returning incorrect IPs,
> and as a result, jobs are not consistently passing.
> You may see problems dowloading repos, building changes, installing
> packages
>
> We would advise to limit the rechecks until we are sure the DNS results
> are stable.
>
> Thanks.


Just adding a little more clarity.  You can see the pass rate of TripleO
jobs start to drop on 8/31 and recover on 9/3 in the screenshot.   We have
not yet fully recovered quite yet, we will update this thread when that is
the case.

Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190903/0cc86f10/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ci_job_pass_fail_rate.png
Type: image/png
Size: 52623 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20190903/0cc86f10/attachment-0001.png>


More information about the openstack-discuss mailing list