<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
span.EmailStyle20
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.none
{mso-style-name:none;}
span.EmailStyle22
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:703287679;
mso-list-type:hybrid;
mso-list-template-ids:1255322538 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span style='color:#1F497D'>Re-sending to openstack-infra mailing list<o:p></o:p></span></p><p class=MsoNormal><span style='color:#1F497D'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b>From:</b> Amrith Kumar <br><b>Sent:</b> Tuesday, November 29, 2016 9:16 AM<br><b>To:</b> Monty Taylor <monty.taylor@gmail.com>; jeremy@openstack.org; 'clark.boylan@gmail.com' <clark.boylan@gmail.com>; 'Andreas Jaeger' <aj@suse.com>; 'Paul Belanger' <pabelanger@redhat.com>; 'yroblamo@redhat.com' <yroblamo@redhat.com><br><b>Cc:</b> Peter Stachowski <peter@tesora.com>; 'Matt Riedemann' <mriedem@us.ibm.com>; 'dansmith@redhat.com' <dansmith@redhat.com><br><b>Subject:</b> [infra] some issue with the osic environment? (as opposed to rax)<br><b>Importance:</b> High<o:p></o:p></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We’ve been trying to get all of the trove CI tests to work with neutron and have made a lot of progress but are now hampered by something that appears to be out of our control. Please see the latest (say, patch set 52) of review 356701 [1].<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='background:yellow;mso-highlight:yellow'><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span></span><![endif]><span style='background:yellow;mso-highlight:yellow'>Ignore all the Ubuntu-xenial jobs, that is a work in progress at this point and the failures are known<o:p></o:p></span></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo2'><![if !supportLists]><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Please only focus on the jobs gate-trove-functional-dsvm-mysql-ubuntu-trusty and gate-trove-scenario-dsvm-* -ubuntu-trusty<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>What we find is that if the job gets scheduled on rax, it works. If it gets scheduled on osic, it fails. The failure mode indicates that when building a guest image, we’re not able to get to the internet; something is messed up in the routing on the machine. How and why this works on rax and not on osic, I know not but this appears to be out of our control.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Failure (from: <a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html">http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html</a>)<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><pre><a name="_2016-11-29_11_44_42_309943"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_309943">2016-11-29 11:44:42.309943</a><span class=none> | 2016-11-29 11:44:42.309 | ++ /tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89 : eval curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w '%{http_code}' <a href="https://bootstrap.pypa.io/get-pip.py">https://bootstrap.pypa.io/get-pip.py</a><o:p></o:p></span></pre><pre><a name="_2016-11-29_11_44_42_315614"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_315614">2016-11-29 11:44:42.315614</a><span class=none> | 2016-11-29 11:44:42.315 | +++ /tmp/dib_build.7kbGxUCC/hooks/bin/cache-url:main:89 : curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.JYWwOK94 -w '%{http_code}' <a href="https://bootstrap.pypa.io/get-pip.py">https://bootstrap.pypa.io/get-pip.py</a><o:p></o:p></span></pre><pre><a name="_2016-11-29_11_44_42_317559"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_317559">2016-11-29 11:44:42.317559</a><span class=none> | 2016-11-29 11:44:42.317 | * Hostname was NOT found in DNS cache<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_44_42_318830"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_318830">2016-11-29 11:44:42.318830</a><span class=none> | 2016-11-29 11:44:42.318 | % Total % Received % Xferd Average Speed Time Time Time Current<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_44_42_320197"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_320197">2016-11-29 11:44:42.320197</a><span class=none> | 2016-11-29 11:44:42.319 | Dload Upload Total Spent Left Speed<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_44_42_442269"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_44_42_442269">2016-11-29 11:44:42.442269</a><span class=none> | 2016-11-29 11:44:42.442 | <o:p></o:p></span></pre><pre><span class=none> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 151.101.48.175...<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_46_49_726905"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_726905">2016-11-29 11:46:49.726905</a><span class=none> | 2016-11-29 11:46:49.724 | <o:p></o:p></span></pre><pre><span class=none> 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0<o:p></o:p></span></pre><pre><span class=none> 0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0<o:p></o:p></span></pre><pre><span class=none>[…]<o:p></o:p></span></pre><pre><span class=none> 0 0 0 0 0 0 0 0 --:--:-- 0:02:06 --:--:-- 0* connect to 151.101.48.175 port 443 failed: Connection timed out<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_46_49_728470"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_728470">2016-11-29 11:46:49.728470</a><span class=none> | 2016-11-29 11:46:49.728 | * Failed to connect to bootstrap.pypa.io port 443: Connection timed out<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_46_49_730314"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_730314">2016-11-29 11:46:49.730314</a><span class=none> | 2016-11-29 11:46:49.730 | * Closing connection 0<o:p></o:p></span></pre><pre><a name="_2016-11-29_11_46_49_732029"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-functional-dsvm-mysql-ubuntu-trusty/b9e5197/console.html#_2016-11-29_11_46_49_732029">2016-11-29 11:46:49.732029</a><span class=none> | 2016-11-29 11:46:49.731 | curl: (7) Failed to connect to bootstrap.pypa.io port 443: Connection timed out</span><o:p></o:p></pre><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Success (from: <a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html">http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html</a>)<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><pre><a name="_2016-11-29_12_04_16_795230"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_795230">2016-11-29 12:04:16.795230</a><span class=none> | 2016-11-29 12:04:16.794 | + /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:82 : '[' -f /home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50bbba586670cca2803e23f614374b482e595f6a -a -s /home/jenkins/.cache/image-create/source-repositories/pip_and_virtualenv_50bbba586670cca2803e23f614374b482e595f6a ']'<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_798157"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_798157">2016-11-29 12:04:16.798157</a><span class=none> | 2016-11-29 12:04:16.797 | + /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:86 : success='Downloaded and cached <a href="https://bootstrap.pypa.io/get-pip.py">https://bootstrap.pypa.io/get-pip.py</a> for the first time'<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_802066"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_802066">2016-11-29 12:04:16.802066</a><span class=none> | 2016-11-29 12:04:16.801 | ++ /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89 : eval curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w '%{http_code}' <a href="https://bootstrap.pypa.io/get-pip.py">https://bootstrap.pypa.io/get-pip.py</a><o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_805188"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_805188">2016-11-29 12:04:16.805188</a><span class=none> | 2016-11-29 12:04:16.804 | +++ /tmp/dib_build.zLErDYEt/hooks/bin/cache-url:main:89 : curl -v -L -o /home/jenkins/.cache/image-create/source-repositories/.download.LGdrOCXI -w '%{http_code}' <a href="https://bootstrap.pypa.io/get-pip.py">https://bootstrap.pypa.io/get-pip.py</a><o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_812077"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_812077">2016-11-29 12:04:16.812077</a><span class=none> | 2016-11-29 12:04:16.811 | * Hostname was NOT found in DNS cache<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_813773"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_813773">2016-11-29 12:04:16.813773</a><span class=none> | 2016-11-29 12:04:16.813 | % Total % Received % Xferd Average Speed Time Time Time Current<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_815340"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_815340">2016-11-29 12:04:16.815340</a><span class=none> | 2016-11-29 12:04:16.815 | Dload Upload Total Spent Left Speed<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_937806"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_937806">2016-11-29 12:04:16.937806</a><span class=none> | 2016-11-29 12:04:16.937 | <o:p></o:p></span></pre><pre><span class=none> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 151.101.44.175...<o:p></o:p></span></pre><pre><a name="_2016-11-29_12_04_16_939302"></a><a href="http://logs.openstack.org/01/356701/52/check/gate-trove-scenario-dsvm-pxc-single-ubuntu-trusty/3837026/console.html#_2016-11-29_12_04_16_939302">2016-11-29 12:04:16.939302</a><span class=none> | 2016-11-29 12:04:16.939 | * Connected to bootstrap.pypa.io (151.101.44.175) port 443 (#0)</span><o:p></o:p></pre><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Would someone in infra please help us figure out what’s going on here please.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>We are currently NOT depending on Monty’s change (398611). Should we be? If yes, what’s the ETA for a merge there?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Thanks,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>-amrith<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>[1] <a href="https://review.openstack.org/#/c/356701/52">https://review.openstack.org/#/c/356701/52</a><o:p></o:p></p><p class=MsoNormal>[2] <a href="https://review.openstack.org/#/c/398611/">https://review.openstack.org/#/c/398611/</a><o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'>--<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'>Amrith Kumar<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'>+1-978-563-9590<o:p></o:p></span></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'><a href="mailto:amrith@amrith.org">amrith@tesora.com</a><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'><a href="http://www.tesora.com/">http://www.tesora.com/</a><o:p></o:p></span></p><p class=MsoNormal><span style='font-family:Consolas;color:#1F497D'>GPG: 0x5E48849A9D21A29B<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p></div></div></body></html>