<div dir="ltr">About fault and alarm, what I was thinking about the causal/deducing chain in root cause analysis.<div><br></div><div>Fault state means the resource is not fully functional and it is evaluated by related indicators. There are alarms on events like power loss or measurands like CPU high, memory low, temperature high. There are also alarms based on deduced state, such as "host fault", "instance fault".</div><div><br></div><div>So an example chain would be</div><div><ul><li>"FAULT: power line cut off" =(monitor)=> "ALARM: host power loss" =(inspect)=> "FAULT: host is unavailable" =(action)=> "ALARM: host fault"</li><li>"FAULT: power line cut off" =(monitor)=> "ALARM: host power loss" =(inspect)=> "FAULT: host is unavailable" =(inspect)=> "FAULT: instance is unavailable" =(action)=> "ALARM: instance fault"</li></ul><div>If we omit the resource, then we get the causal chain as it is in Vitrage</div><div><ul><li>"ALARM: host power loss" =(causes)=> "ALARM: host fault"<br></li><li>"ALARM: host power loss" =(causes)=> "ALARM: instance fault"</li></ul><div>But what the user care about might be there "FAULT: power line cut off" causes all these alarms. What I haven't made clear yet is the equivalence between fault and alarm.</div><div><br></div><div>I may have made it more complex with my <b>immature</b> thoughts. It could be even more complex if we consider multiple upstream causes and downstream outcome. It may be an interesting topic to be discussed in design session.</div><div><div><br></div><div>On Sun, Jan 15, 2017 at 9:21 PM Afek, Ifat (Nokia - IL) <<a href="mailto:ifat.afek@nokia.com">ifat.afek@nokia.com</a>> wrote:<br></div></div></div></div><div><div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg">
<div class="m_8084544798751535514WordSection1 gmail_msg">
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">Hi Yinliyin,<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">There are two use cases:
<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">One is yours, where you have a single monitor that generates “real” alarms, and Vitrage that generates deduced alarms.
<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">Another is where someone has a few monitors, and there might be a collision/equivalence between their alarms.
<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">The solution that you suggested might solve the first use case, but I wouldn’t want to ignore the second one, which is also valid.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">Regarding some of your specific suggestions:<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p></div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg">
<p class="m_8084544798751535514MsoListParagraph gmail_msg" style="margin-left:33.75pt">
<u class="gmail_msg"></u><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><span class="gmail_msg">1.<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">In templates, we only define the alarm entity for the datasource that the alarm is reported by, such as Nagios.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
</div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg"><p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">[Ifat] This will only work for a single monitor.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p></div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg">
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 2. When evaluator deduce an alarm, it would raise the alarm with the type set to be the datasource that would report the alarm, not be vitrage.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
</div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg"><p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">[Ifat] I don’t think this is right. In Vitrage Alarm view in the UI, displaying the deduced alarm as “Nagios” is misleading, since Nagios did not report this alarm.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">I can think of a solution that is specific to the deduced alarms case, where we will replace a Vitrage alarm with a “real” alarm whenever there is a collision. This solution is easier,
but we should carefully examine all use cases to make sure there is no ambiguity. However, for the more general use case I would prefer the option that we discussed in a previous mail, of having two (or more) alarms connected with a ‘equivalent’ relationship.
<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">What do you think?<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">Ifat.<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p class="MsoNormal gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm" class="gmail_msg">
<p class="MsoNormal gmail_msg" style="margin-left:36.0pt"><b class="gmail_msg"><span style="font-family:Calibri;color:black" class="gmail_msg">From:
</span></b><span style="font-family:Calibri;color:black" class="gmail_msg">"<a href="mailto:yinliyin@zte.com.cn" class="gmail_msg" target="_blank">yinliyin@zte.com.cn</a>" <<a href="mailto:yinliyin@zte.com.cn" class="gmail_msg" target="_blank">yinliyin@zte.com.cn</a>><br class="gmail_msg">
<b class="gmail_msg">Date: </b>Saturday, 14 January 2017 at 09:57<br class="gmail_msg">
<br class="gmail_msg">
<u class="gmail_msg"></u><u class="gmail_msg"></u></span></p>
</div>
<div class="gmail_msg">
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">It won’t solve the general problem of two different monitors that raise the same alarm</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p></div></div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg"><div class="gmail_msg">
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> [yinliyin] Generally, we would only deploy one monitor for a same alarm. </span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">It won’t solve possible conflicts of timestamp and severity between different monitors</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> [yinliyin] Please see the following contents.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg">It will make the decision of when to delete the alarm more complex (delete it when the deduced alarm is deleted? When Nagios alarm is deleted? both? And
how to change the timestamp and severity in these cases?)</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:72.0pt" class="gmail_msg"><u class="gmail_msg"></u><span style="font-size:10.0pt;font-family:Symbol" class="gmail_msg"><span class="gmail_msg">·<span style="font:7.0pt "Times New Roman"" class="gmail_msg">
</span></span></span><u class="gmail_msg"></u><span dir="LTR" class="gmail_msg"></span><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> [yinliyin] Please see the following contents.<br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> The following is the basic idea of solving the problem in this situation:</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 1. In templates, we only define the alarm entity for the datasource that the alarm is reported by, such as Nagios.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 2. When evaluator deduce an alarm, it would raise the alarm with the type set to be the datasource that would report the alarm, not be vitrage.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 3. When entity_graph get the events from the "evaluator_queue"(all the alarms in the "evaluator_queue" are deduced alarms), it queries the graph to find out whether there
was a same alarm reported by datasource. If it was true, it would discard the alarm.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 4. When entity_graph get the events from "queue", it queries the graph to find out whether there was a same alarm deduced by evaluator. If it was true, it would replace
the alarm in the graph with the newly arrived alarm reported by the datasource.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 5. When the evaluator deduced that an alarm would be deleted, it deletes the alarm whatever the generation type of the alarm be(Generated by datasource or deduced by evaluator).
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> 6. When datasource reports recover event of an alarm, entity_graph would query graph to find out whether the alarm was exist. If the alarm was not exist, entity_graph would
discard the event.</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> </span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"> </span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:11.0pt;font-family:Calibri" class="gmail_msg"><br class="gmail_msg">
<br class="gmail_msg">
</span><u class="gmail_msg"></u><u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
<div class="gmail_msg">
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:5.5pt;font-family:宋体" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
<p style="margin-left:36.0pt" class="gmail_msg"><span style="font-size:5.5pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></span></p>
</div>
<div class="gmail_msg">
<div class="gmail_msg">
<div id="m_8084544798751535514zwriteHistoryContainer" class="gmail_msg">
<div class="gmail_msg">
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
</div>
</div>
</div>
</div>
<p style="margin-left:36.0pt" class="gmail_msg"><u class="gmail_msg"></u> <u class="gmail_msg"></u></p>
</div></div></div><div bgcolor="white" lang="EN-US" link="blue" vlink="purple" class="gmail_msg"><div class="m_8084544798751535514WordSection1 gmail_msg"><div class="gmail_msg"></div>
</div>
</div>
__________________________________________________________________________<br class="gmail_msg">
OpenStack Development Mailing List (not for usage questions)<br class="gmail_msg">
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" class="gmail_msg" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br class="gmail_msg">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" class="gmail_msg" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="gmail_msg">
</blockquote></div></div></div></div></div>