<div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr">2018년 10월 31일 (수) 오후 5:58, Won <<a href="mailto:wjstk16@gmail.com">wjstk16@gmail.com</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><span class="m_-4965425863663943524gmail-im" style="color:rgb(80,0,80)">Hi,<br></span><div><div class="gmail_quote"><span class="m_-4965425863663943524gmail-im"><blockquote class="gmail_quote" style="color:rgb(80,0,80);margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br></div><div dir="ltr"><div style="color:rgb(34,34,34)">This is strange. I would expect your original definition to work as well, since the alarm key in Vitrage is defined by a combination of the alert name and the instance. We will check it again. </div><div style="color:rgb(34,34,34)">BTW, we solved a different bug related to Prometheus alarms not being cleared [1]. Could it be related?</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div style="color:rgb(80,0,80)"><br></div><div style="color:rgb(80,0,80)">Using the original definition, no matter how different the instances are, the alarm names are recognized as the same alarm in vitrage.</div><div><font color="#500050">And I tried to install the rocky version and the master version on the new server and retest but the problem was not solved. The latest bugfix seems irrelevant.</font><br></div><div><br></div><div><div class="gmail_quote"><span class="m_-4965425863663943524gmail-im"><blockquote class="gmail_quote" style="color:rgb(80,0,80);margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><span style="color:rgb(34,34,34)">Does the wrong timestamp appear if you run 'vitrage alarm list' cli command? please try running 'vitrage alarm list --debug' and send me the output.</span> <br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><br class="m_-4965425863663943524gmail-Apple-interchange-newline"></span></div></div><div><font color="#500050">I have attached 'vitrage-alarm-list.txt.'<br></font></div><div><div><div class="gmail_quote"><span class="m_-4965425863663943524gmail-im"><div> </div><blockquote class="gmail_quote" style="color:rgb(80,0,80);margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div style="color:rgb(34,34,34)">Please send me vitrage-collector.log and vitrage-graph.log from the time that the problematic vm was created and deleted. Please also create and delete a vm on your 'ubuntu' server, so I can check the differences in the log.</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><br class="m_-4965425863663943524gmail-Apple-interchange-newline"></span></div></div><div>I have attached 'vitrage_log_on_compute1.zip' and 'vitrage_log_on_ubuntu.zip' files. </div><div>When creating a vm on computer1, a vitrage-collect log occurred, but no log occurred when it was removed.<br></div><div><br></div><div>Br,</div><div>Won</div><br class="m_-4965425863663943524gmail-Apple-interchange-newline"></div><div><font color="#500050"><br></font></div></span></div></div></div></div></div></div></div></div></div><br><div class="gmail_quote"><div dir="ltr">2018년 10월 30일 (화) 오전 1:28, Ifat Afek <<a href="mailto:ifatafekn@gmail.com" target="_blank">ifatafekn@gmail.com</a>>님이 작성:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>On Fri, Oct 26, 2018 at 10:34 AM Won <<a href="mailto:wjstk16@gmail.com" target="_blank">wjstk16@gmail.com</a>> wrote:<br></div><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><br></div><div dir="ltr"><div>I solved the problem of not updating the Prometheus alarm.</div><div>Alarms with the same Prometheus alarm name are recognized as the same alarm in vitrage.<br></div><div><br></div><div>------- alert.rules.yml</div><div><div>groups:</div><div>- name: alert.rules</div><div> rules:</div><div> - alert: InstanceDown</div><div> expr: up == 0</div><div> for: 60s</div><div> labels:</div><div> severity: warning</div><div> annotations:</div><div> description: '{{ $labels.instance }} of job {{ $labels.job }} has been down</div><div> for more than 30 seconds.'</div><div> summary: Instance {{ $labels.instance }} down</div></div><div>------</div><div>This is the contents of the alert.rules.yml file before I modify it.<br></div><div>This is a yml file that generates an alarm when the cardvisor stops(instance down). Alarm is triggered depending on which instance is down, but all alarms have the same name as 'instance down'. Vitrage recognizes all of these alarms as the same alarm. Thus, until all 'instance down' alarms were cleared, the 'instance down' alarm was recognized as unresolved and the alarm was not extinguished.</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>This is strange. I would expect your original definition to work as well, since the alarm key in Vitrage is defined by a combination of the alert name and the instance. We will check it again. </div><div>BTW, we solved a different bug related to Prometheus alarms not being cleared [1]. Could it be related?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><blockquote class="gmail_quote" style="color:rgb(80,0,80);margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div></div><div><span style="color:rgb(34,34,34)">Can you please show me where you saw the 2001 timestamp? I didn't find it in the log.</span><span style="color:rgb(34,34,34)"> </span></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div> </div><div><img src="cid:ii_jno98j6l0" alt="image.png" style="margin-right:0px" width="1133" height="110"><br></div></div><div>The time stamp is recorded well in log(vitrage-graph,collect etc), but in vitrage-dashboard it is marked 2001-01-01.</div><div>However, it seems that the time stamp is recognized well internally because the alarm can be resolved and is recorded well in log.</div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div><br></div><div>Does the wrong timestamp appear if you run 'vitrage alarm list' cli command? please try running 'vitrage alarm list --debug' and send me the output.<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-gs" style="margin:0px;padding:0px 0px 20px;width:1512px"><div class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-"><div id="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-:pm" class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-ii m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-gt m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-adO" style="direction:ltr;margin:8px 0px 0px;padding:0px"><div id="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-:pl" class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-a3s m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-aXjCH" style="overflow:hidden;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:1.5;outline:none"><div dir="ltr"><div class="gmail_quote"><div style="font-family:Arial,Helvetica,sans-serif;font-size:small"><div><img src="cid:ii_jnoa1gmi1" alt="image.png" width="876" height="154" style="margin-right:0px"><br></div></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small">Host name ubuntu is my main server. I install openstack all in one in this server and i install compute node in host name compute1.</div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small">When i create a new vm in nova(compute1) it immediately appear in the entity graph. But in does not disappear in the entity graph when i delete the vm. No matter how long i wait, it doesn't disappear.</div><div>Afther i execute 'vitrage-purge-data' command and reboot the Openstack(execute reboot command in openstack server(host name ubuntu)), it disappear. Only execute 'vitrage-purge-data' does not work. It need a reboot to disappear.<br></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small">When i create a new vm in nova(ubuntu) there is no problem. </div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div>Please send me vitrage-collector.log and vitrage-graph.log from the time that the problematic vm was created and deleted. Please also create and delete a vm on your 'ubuntu' server, so I can check the differences in the log.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-gs" style="margin:0px;padding:0px 0px 20px;width:1512px"><div class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-"><div id="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-:pm" class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-ii m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-gt m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-adO" style="direction:ltr;margin:8px 0px 0px;padding:0px"><div id="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-:pl" class="m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-a3s m_-4965425863663943524m_7424786759214132764gmail-m_1901580194436723236gmail-m_-3570743932547074883gmail-aXjCH" style="overflow:hidden;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;line-height:1.5;outline:none"><div dir="ltr"><div class="gmail_quote"><div>I implemented the web service of the micro service architecture and applied the RCA. Attached file picture shows the structure of the web service I have implemented. I wonder what data I receive and what can i do when I link vitrage with kubernetes.</div><div>As i know, the vitrage graph does not present information about containers or pods inside the vm. If that is correct, I would like to make the information of the pod level appear on the entity graph.<br></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small"><br></div><div style="font-family:Arial,Helvetica,sans-serif;font-size:small">I follow (<a href="https://docs.openstack.org/vitrage/latest/contributor/k8s_datasource.html" target="_blank">https://docs.openstack.org/vitrage/latest/contributor/k8s_datasource.html</a>) this step. I attached the vitage.conf file and the kubeconfig file. The contents of the Kubeconconfig file are copied from the contents of the admin.conf file on the master node.</div><div>I want to check my settings are right and connected, but I don't know how. It would be very much appreciated if you let me know how.<br></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></div></blockquote><div>Unfortunately, Vitrage does not hold pods and containers information at the moment. We discussed the option of adding it in Stein release, but I'm not sure we will get to do it. </div><div> </div><div>Br,</div><div>Ifat</div><div> </div><div>[1] <a href="https://review.openstack.org/#/c/611258/" target="_blank">https://review.openstack.org/#/c/611258/</a> </div><div> </div></div></div></div></div>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</blockquote></div>
</blockquote></div>