[tripleo][ci] container pulls failing

Wesley Hayutin whayutin at redhat.com
Mon Jul 27 21:17:34 UTC 2020


FYI...

If you find your jobs are failing with an error similar to [1], you have
been rate limited by docker.io via the upstream mirror system and have hit
[2].  I've been discussing the issue w/ upstream infra, rdo-infra and a few
CI engineers.

There are a few ways to mitigate the issue however I don't see any of the
options being completed very quickly so I'm asking for your patience while
this issue is socialized and resolved.

For full transparency we're considering the following options.

1. move off of docker.io to quay.io
2. local container builds for each job in master, possibly ussuri
3. parent child jobs upstream where rpms and containers will be build and
host artifacts for the child jobs
4. remove some portion of the upstream jobs to lower the impact we have on
3rd party infrastructure.

If you have thoughts please don't hesitate to share on this thread.  Very
sorry we're hitting these failures and I really appreciate your patience.
I would expect major delays in getting patches merged at this point until
things are resolved.

Thank you!

[1] HTTPError: 429 Client Error: Too Many Requests for url:
http://mirror.ca-ymq-1.vexxhost.opendev.org:8082/v2/tripleotrain/centos-binary-cron/blobs/sha256:76342b0db11c6b5acf33b9f1cbf10b3d2680fb20967ccd7daa9593a39e9e45c0
[2] https://bugs.launchpad.net/tripleo/+bug/1889122
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20200727/b54dc42f/attachment.html>


More information about the openstack-discuss mailing list