On 5/11/2019 6:57 PM, Mohammed Naser wrote: > is it possible that this is because > `mirror.sjc1.vexxhost.openstack.org` does not actually have AAAA > records that causes this? > > On Thu, May 9, 2019 at 9:58 AM Jeremy Stanley<fungi at yuggoth.org> wrote: >> On 2019-05-09 08:49:35 -0500 (-0500), Eric Fried wrote: >>> Have we tried changing the URI to >>> https://opendev.org/openstack/requirements/raw/branch/master/upper-constraints.txt >>> to avoid the redirecting? >>> >>> On 5/9/19 8:02 AM, Matt Riedemann wrote: >>>> I'm not sure what is causing the bug [1] but it's failing at a really >>>> high rate for about week now. Do we have ideas on the issue? Do we have >>>> thoughts on a workaround? Or should we disable the vexxhost-sjc1 >>>> provider until it's solved? >>>> >>>> [1]http://status.openstack.org/elastic-recheck/#1827083 >> I have to assume the bug report itself is misleading. Jobs should be >> using the on-disk copy of the requirements repository provided by >> Zuul for this and not retrieving that file over the network. However >> the problem is presumably DNS resolution not working at all on those >> nodes, so something is going to break at some point in the job in >> those cases regardless. >> -- >> Jeremy Stanley Just an update on this for those watching at home. Clark switched the vexxhost-sjc1 nodes to IPv4: https://review.opendev.org/#/c/659140/ https://review.opendev.org/#/c/659181/ And early results in logstash [1] show that might have worked around the problem for now. [1] http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22ERROR%3A%20Could%20not%20install%20packages%20due%20to%20an%20EnvironmentError%3A%20HTTPSConnectionPool(host%3D'git.openstack.org'%2C%20port%3D443)%3A%20Max%20retries%20exceeded%20with%20url%3A%20%2Fcgit%2Fopenstack%2Frequirements%2Fplain%2Fupper-constraints.txt%20(Caused%20by%20NewConnectionError('%3Cpip._vendor.urllib3.connection.VerifiedHTTPSConnection%20object%20at%5C%22%20AND%20message%3A%5C%22Failed%20to%20establish%20a%20new%20connection%3A%20%5BErrno%20-3%5D%20Temporary%20failure%20in%20name%20resolution'%2C))%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20voting%3A1&from=1d -- Thanks, Matt