[OpenStack-Infra] FW: [infra] some issue with the osic environment? (as opposed to rax)
Amrith Kumar
amrith at tesora.com
Tue Nov 29 14:28:32 UTC 2016
Re-sending to openstack-infra mailing list
From: Amrith Kumar
Sent: Tuesday, November 29, 2016 9:16 AM
To: Monty Taylor <monty.taylor at gmail.com>; jeremy at openstack.org;
'clark.boylan at gmail.com' <clark.boylan at gmail.com>; 'Andreas Jaeger'
<aj at suse.com>; 'Paul Belanger' <pabelanger at redhat.com>;
'yroblamo at redhat.com' <yroblamo at redhat.com>
Cc: Peter Stachowski <peter at tesora.com>; 'Matt Riedemann'
<mriedem at us.ibm.com>; 'dansmith at redhat.com' <dansmith at redhat.com>
Subject: [infra] some issue with the osic environment? (as opposed to rax)
Importance: High
We've been trying to get all of the trove CI tests to work with neutron and
have made a lot of progress but are now hampered by something that appears
to be out of our control. Please see the latest (say, patch set 52) of
review 356701 [1].
1. Ignore all the Ubuntu-xenial jobs, that is a work in progress at
this point and the failures are known
2. Please only focus on the jobs
gate-trove-functional-dsvm-mysql-ubuntu-trusty and
gate-trove-scenario-dsvm-* -ubuntu-trusty
What we find is that if the job gets scheduled on rax, it works. If it gets
scheduled on osic, it fails. The failure mode indicates that when building a
guest image, we're not able to get to the internet; something is messed up
in the routing on the machine. How and why this works on rax and not on
osic, I know not but this appears to be out of our control.
Failure (from:
http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysq
l-ubuntu-trusty/b9e5197/console.html)
2016-11-29 11:44:42.309943
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_309943> |
2016-11-29 11:44:42.309 | ++
/tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89
: eval curl -v -L -o
/home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w
'%{http_code}' https://bootstrap.pypa.io/get-pip.py
2016-11-29 11:44:42.315614
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_315614> |
2016-11-29 11:44:42.315 | +++
/tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89
: curl -v -L -o
/home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w
'%{http_code}' https://bootstrap.pypa.io/get-pip.py
2016-11-29 11:44:42.317559
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_317559> |
2016-11-29 11:44:42.317 | * Hostname was NOT found in DNS cache
2016-11-29 11:44:42.318830
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_318830> |
2016-11-29 11:44:42.318 | % Total % Received % Xferd Average Speed
Time Time Time Current
2016-11-29 11:44:42.320197
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_320197> |
2016-11-29 11:44:42.319 | Dload Upload
Total Spent Left Speed
2016-11-29 11:44:42.442269
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_442269> |
2016-11-29 11:44:42.442 |
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Trying 151.101.48.175...
2016-11-29 11:46:49.726905
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_726905> |
2016-11-29 11:46:49.724 |
0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:--
0
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:--
0
[.]
0 0 0 0 0 0 0 0 --:--:-- 0:02:06 --:--:--
0* connect to 151.101.48.175 port 443 failed: Connection timed out
2016-11-29 11:46:49.728470
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_728470> |
2016-11-29 11:46:49.728 | * Failed to connect to bootstrap.pypa.io port 443:
Connection timed out
2016-11-29 11:46:49.730314
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_730314> |
2016-11-29 11:46:49.730 | * Closing connection 0
2016-11-29 11:46:49.732029
<http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mys
ql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_732029> |
2016-11-29 11:46:49.731 | curl: (7) Failed to connect to bootstrap.pypa.io
port 443: Connection timed out
Success (from:
http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-si
ngle-ubuntu-trusty/3837026/console.html)
2016-11-29 12:04:16.795230
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_795230> |
2016-11-29 12:04:16.794 | +
/tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:82
: '[' -f
/home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50b
bba586670cca2803e23f614374b482e595f6a -a -s
/home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50b
bba586670cca2803e23f614374b482e595f6a ']'
2016-11-29 12:04:16.798157
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_798157> |
2016-11-29 12:04:16.797 | +
/tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:86
: success='Downloaded and cached https://bootstrap.pypa.io/get-pip.py for
the first time'
2016-11-29 12:04:16.802066
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_802066> |
2016-11-29 12:04:16.801 | ++
/tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89
: eval curl -v -L -o
/home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w
'%{http_code}' https://bootstrap.pypa.io/get-pip.py
2016-11-29 12:04:16.805188
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_805188> |
2016-11-29 12:04:16.804 | +++
/tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89
: curl -v -L -o
/home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w
'%{http_code}' https://bootstrap.pypa.io/get-pip.py
2016-11-29 12:04:16.812077
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_812077> |
2016-11-29 12:04:16.811 | * Hostname was NOT found in DNS cache
2016-11-29 12:04:16.813773
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_813773> |
2016-11-29 12:04:16.813 | % Total % Received % Xferd Average Speed
Time Time Time Current
2016-11-29 12:04:16.815340
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_815340> |
2016-11-29 12:04:16.815 | Dload Upload
Total Spent Left Speed
2016-11-29 12:04:16.937806
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_937806> |
2016-11-29 12:04:16.937 |
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Trying 151.101.44.175...
2016-11-29 12:04:16.939302
<http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-s
ingle-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_939302> |
2016-11-29 12:04:16.939 | * Connected to bootstrap.pypa.io (151.101.44.175)
port 443 (#0)
Would someone in infra please help us figure out what's going on here
please.
We are currently NOT depending on Monty's change (398611). Should we be? If
yes, what's the ETA for a merge there?
Thanks,
-amrith
[1] https://review.openstack.org/#/c/356701/52
[2] https://review.openstack.org/#/c/398611/
--
Amrith Kumar
+1-978-563-9590
amrith at tesora.com <mailto:amrith at amrith.org>
http://www.tesora.com/
GPG: 0x5E48849A9D21A29B
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20161129/a2c81c49/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4805 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-infra/attachments/20161129/a2c81c49/attachment-0001.bin>
More information about the OpenStack-Infra
mailing list