<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>Jay,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>This is the Trove commit … <a href="https://review.openstack.org/#q,I85364c6530058e964a8eba7fb515d7deadfd5d72,n,z"><span style='color:windowtext;text-decoration:none'>I85364c6530058e964a8eba7fb515d7deadfd5d72</span></a><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>-amrith<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'><o:p> </o:p></span></p><div style='border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt'><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'>From:</span></b><span style='font-size:11.0pt;font-family:"Calibri",sans-serif'> Jim Rollenhagen [mailto:jim@jimrollenhagen.com] <br><b>Sent:</b> Wednesday, January 18, 2017 7:57 AM<br><b>To:</b> OpenStack Development Mailing List (not for usage questions) <openstack-dev@lists.openstack.org><br><b>Subject:</b> Re: [openstack-dev] [ironic] [infra] Nested KVM + the gate<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>On Tue, Jan 17, 2017 at 6:41 PM, Jay Faulkner <<a href="mailto:jay@jvf.cc" target="_blank">jay@jvf.cc</a>> wrote:<o:p></o:p></p><div><div><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><p class=MsoNormal>Hi all,<br><br>Back in late October, Vasyl wrote support for devstack to auto detect, and when possible, use kvm to power Ironic gate jobs (0036d83b330d98e64d656b156001dd2209ab1903). This has lowered some job time when it works, but has caused failures — how many? It’s hard to quantify as the log messages that show the error don’t appear to be indexed by elastic search. It’s something seen often enough that the issue has become a permanent staple on our gate whiteboard, and doesn’t appear to be decreasing in quantity.<br><br>I pushed up a patch, <a href="https://review.openstack.org/#/c/421581" target="_blank">https://review.openstack.org/#/c/421581</a>, which keeps the auto detection behavior, but defaults devstack to use qemu emulation instead of kvm.<br><br>I have two questions:<br>1) Is there any way I’m not aware of we can quantify the number of failures this is causing? The key log message, "KVM: entry failed, hardware error 0x0”, shows up in logs/libvirt/qemu/node-*.txt.gz.<br>2) Are these failures avoidable or visible in any way?<br><br>IMO, if we can’t fix these failures, in my opinion, we have to do a change to avoid using nested KVM altogether. Lower reliability for our jobs is not worth a small decrease in job run time.<o:p></o:p></p></blockquote><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>+2, especially this late in the cycle, we need our CI to be rock solid.<o:p></o:p></p></div><div><p class=MsoNormal>// jim<o:p></o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal><o:p> </o:p></p></div></div></div></div></body></html>