<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 10/23/2013 05:08 PM, Rochelle.Grober
wrote:<br>
</div>
<blockquote
cite="mid:EE954C2560065B4BBB76E698E2FB112B01848831@sjceml501-mbs.china.huawei.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span
style="color:#1F497D">John Griffith wrote:<o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">On
Wed, Oct 23, 2013 at 8:47 AM, Sean Dague <<a
moz-do-not-send="true" href="mailto:sean@dague.net"
target="_blank">sean@dague.net</a>> wrote:<span
style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"
style="mso-margin-top-alt:0in;margin-right:0in;margin-bottom:12.0pt;margin-left:.5in">On
10/23/2013 10:40 AM, John Griffith wrote:<span
style="color:#1F497D"><o:p></o:p></span></p>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<p class="MsoNormal" style="margin-left:.5in"><br>
<br>
<br>
On Sun, Oct 20, 2013 at 7:38 AM, Sean Dague <<a
moz-do-not-send="true"
href="mailto:sean@dague.net" target="_blank">sean@dague.net</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:.5in"><mailto:<a
moz-do-not-send="true"
href="mailto:sean@dague.net" target="_blank">sean@dague.net</a>>>
wrote:<br>
<br>
Dave Kranz has been building a system so that we
can ensure that<br>
during a Tempest run services don't spew ERRORs
in the logs.<br>
Eventually, we're going to gate on this, because
there is nothing<br>
that Tempest does to the system that should
cause any OpenStack<br>
service to ERROR or stack trace (Errors should
actually be<br>
exceptional events that something is wrong with
the system, not<br>
regular events).<br>
<br>
<br>
So I have to disagree with the approach being taken
here. Particularly<br>
in the case of Cinder and the negative tests that
are in place. When I<br>
read this last week I assumed you actually meant
that "Exceptions" were<br>
exceptional and nothing in Tempest should cause
Exceptions. It turns<br>
out you apparently did mean Errors. I completely
disagree here, Errors<br>
happen, some are recovered, some are expected by the
tests etc. Having<br>
a policy and especially a gate that says NO ERROR
MESSAGE in logs makes<br>
absolutely no sense to me.<br>
<br>
Something like NO TRACE/EXCEPTION MESSAGE in logs I
can agree with, but<br>
this makes no sense to me. By the way, here's a
perfect example:<br>
<a moz-do-not-send="true"
href="https://bugs.launchpad.net/cinder/+bug/1243485"
target="_blank">https://bugs.launchpad.net/cinder/+bug/1243485</a><br>
<br>
As long as we have Tempest tests that do things like
"show non-existent<br>
volume" you're going to get an Error message and I
think that you should<br>
quite frankly.<o:p></o:p></p>
</div>
</blockquote>
<p class="MsoNormal" style="margin-left:.5in"><br>
Ok, I guess that's where we probably need to clarify
what "Not Found" is. Because "Not Found" to me seems
like it should be a request at INFO level, not ERROR.<o:p></o:p></p>
<div>
<p class="MsoNormal" style="margin-left:.5in"><span
style="font-family:"Courier New""><o:p> </o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-left:24.45pt"><br>
ERROR from an admin perspective should really be
something that would suitable for sending an alert to
an administrator for them to come and fix the cloud.<span
style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">From my
perspective as someone who has done Ops in the past,
a “Volume Not Found” can be either info or an
error. It all depends on the context. That said,
we need to be able to test ERROR conditions and
ensure that they report properly as ERROR, else the
poor Ops folks will always be on the spot for not
knowing that there is a problem. A volume that has
gone missing is a problem. Ops would like an
immediate report. They would trigger on the ERROR
statement in the log. On the other hand, if
someone/thing fatfingers an input and requests
something that has never existed, then that’s just
info.</span></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
It is not just a case of fatfingers. Some of the delete apis are
asynchronous and the only way to know that a delete finished is to
check if the object still exists. Tempest does such checks to manage
resource usage, even if there were no negative tests. The logs are
not full of ERRORs because almost all of our apis, including nova,
do not log an ERROR when returning 404.<br>
<br>
I think John's point is that it can be hard or impossible to tell if
an object is not found because it truly no longer exists (or never
existed), or if there is something wrong with the system and the
object really exists but is not being found. But I would argue that
even if this is true we cannot alert the operator every time some
user checks to see if an object is still there. So there has to be
some "thing" that gets put in the log which says "there is a problem
with the system, either a bug or ran out of disk or something". The
appearance of that thing in the log is what an alert should be
triggered on, and what should fail a gate job. That is pretty close
to what ERROR is being used for now.<br>
<blockquote
cite="mid:EE954C2560065B4BBB76E698E2FB112B01848831@sjceml501-mbs.china.huawei.com"
type="cite">
<div class="WordSection1">
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">We need
to be able to test for correctness of errors and
process logs with errors in them as part of the test
verification. Perhaps a switch in the test that
indicates log needs post processing, or a way to
redirect the log during a specific error test, or
some such? The question is, how do we keep test
system logs clean of ERRORs and still test system
logs for intentionally triggered ERRORs?</span></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<br>
<blockquote
cite="mid:EE954C2560065B4BBB76E698E2FB112B01848831@sjceml501-mbs.china.huawei.com"
type="cite">
<div class="WordSection1">
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#1F497D">--Rocky</span></p>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
We might be able to do that in our test framework, but it would not
help operators. IMO the least of evils here by far is to log events
associated with an api call that returns 4xx in a way that is
distinguishable from how we log when we detect a system failure of
some sort.<br>
<br>
-David<br>
<blockquote
cite="mid:EE954C2560065B4BBB76E698E2FB112B01848831@sjceml501-mbs.china.huawei.com"
type="cite">
<div class="WordSection1">
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC
1.0pt;padding:0in 0in 0in
6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><span style="color:#1F497D"><o:p></o:p></span></p>
<br>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>