[TripleO] Gate blockers - C8 Wallaby & C9 Master|Wallaby
Hey! TripleO team noticed two separate issues which hit gates about 5h ago: * C8 Wallaby https://bugs.launchpad.net/tripleo/+bug/1967943 * C9 Master|Wallaby: https://bugs.launchpad.net/tripleo/+bug/1967945 Please withhold rechecking until further notice. Thanks, Dariusz
On Tue, Apr 5, 2022, at 12:58 PM, Dariusz Smigiel wrote:
Hey! TripleO team noticed two separate issues which hit gates about 5h ago:
* C8 Wallaby https://bugs.launchpad.net/tripleo/+bug/1967943 * C9 Master|Wallaby: https://bugs.launchpad.net/tripleo/+bug/1967945
Please withhold rechecking until further notice.
I think both of these issues are due to the problem where PyPI's CDN will fallback to the backup backend, and that backup backend is stale without newer package releases. OpenStack notices because constraints require specific versions that cannot be satisfied in these situations. Most other users of PyPI end up getting old versions. If you request the package index for the packages experiencing these problems through our mirrors (which are actually just caching proxies) you'll typically find that all the versions are there as this problem is infrequent. Though infrequent it only takes one of the many packages being installed to have a problem to cause everything to break. Others should double check this though as it is possible there is a requirement conflict or some other problem causing this.
Thanks, Dariusz
* C8 Wallaby https://bugs.launchpad.net/tripleo/+bug/1967943 * C9 Master|Wallaby: https://bugs.launchpad.net/tripleo/+bug/1967945
Please withhold rechecking until further notice.
I think both of these issues are due to the problem where PyPI's CDN will fallback to the backup backend, and that backup backend is stale without newer package releases. OpenStack notices because constraints require specific versions that cannot be satisfied in these situations. Most other users of PyPI end up getting old versions.
Clark, I think you hit a nail on the head here: https://status.python.org/incidents/mxgkk3xxr9v7?u=v8pzlr5n28h8 Thanks for that explanation. I learned a new thing with our infra. Dariusz
The issue seems to be resolved by now. We were dealing with outdated pypi cache in rdoproject. After flushing it, we're finally picking up goot content. Thanks, Dariusz On Tue, Apr 5, 2022 at 2:19 PM Dariusz Smigiel <dsmigiel@redhat.com> wrote:
* C8 Wallaby https://bugs.launchpad.net/tripleo/+bug/1967943 * C9 Master|Wallaby: https://bugs.launchpad.net/tripleo/+bug/1967945
Please withhold rechecking until further notice.
I think both of these issues are due to the problem where PyPI's CDN will fallback to the backup backend, and that backup backend is stale without newer package releases. OpenStack notices because constraints require specific versions that cannot be satisfied in these situations. Most other users of PyPI end up getting old versions.
Clark, I think you hit a nail on the head here: https://status.python.org/incidents/mxgkk3xxr9v7?u=v8pzlr5n28h8
Thanks for that explanation. I learned a new thing with our infra.
Dariusz
participants (2)
-
Clark Boylan
-
Dariusz Smigiel