<div dir="ltr"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">> However, it would be great if we could start devising a solution for having<br>
> "health" reports from the various CI systems.<br>
> This report should report the following kind of information:<br>
> - timestamp of last run<br>
> - timestamp of last vote (a system might start job which then get aborted<br>
> for CI infra problems)<br>
> - % of success vs failures (not sure how important is that one but provides<br>
> a metric of comparison with upstream jenkins)<br>
> - % of disagreements with jenkins (this might allow us to quickly spot those<br>
> CI systems which are randomly -1'ing patches)<br>
><br>
> The percentage metrics might be taken over a 48 hours or 7 days interval, or<br>
> both.<br>
> Does this idea sound reasonable?<br>
><br>
This sounds like a very good idea. Now we just need to find someone<br>
with the time to write this. :)</blockquote><div class="gmail_extra"><br></div><div class="gmail_extra">That's exactly what Stackalytics/DriverLog may do! It already collects the latest CI votes and it wouldn't be hard to calculate metrics based on them.<br>
<br></div><div class="gmail_extra">Ilya<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-06-16 18:11 GMT+04:00 Salvatore Orlando <span dir="ltr"><<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On 16 June 2014 15:58, Kyle Mestery <span dir="ltr"><<a href="mailto:mestery@noironetworks.com" target="_blank">mestery@noironetworks.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>On Mon, Jun 16, 2014 at 8:52 AM, Salvatore Orlando <<a href="mailto:sorlando@nicira.com" target="_blank">sorlando@nicira.com</a>> wrote:<br>
> I will probably be unable, as usual, to attend today's CI meeting (falls<br>
> right around my dinner time).<br>
> I think it's a good idea to starting keeping track of the status of the<br>
> various CI systems, but I feel the etherpad will not work very well in the<br>
> long term.<br>
><br>
</div>Agreed. The etherpad was a starting point, I'll move this information<br>
to a wiki page later today.<br>
<div><br>
> However, it would be great if we could start devising a solution for having<br>
> "health" reports from the various CI systems.<br>
> This report should report the following kind of information:<br>
> - timestamp of last run<br>
> - timestamp of last vote (a system might start job which then get aborted<br>
> for CI infra problems)<br>
> - % of success vs failures (not sure how important is that one but provides<br>
> a metric of comparison with upstream jenkins)<br>
> - % of disagreements with jenkins (this might allow us to quickly spot those<br>
> CI systems which are randomly -1'ing patches)<br>
><br>
> The percentage metrics might be taken over a 48 hours or 7 days interval, or<br>
> both.<br>
> Does this idea sound reasonable?<br>
><br>
</div>This sounds like a very good idea. Now we just need to find someone<br>
with the time to write this. :)<br>
<div><br>
> Also, regarding [1]. I agree that more is always better... but I would like<br>
> a minimum required set of tests to be enforced.<br>
> Is this something that can be achieved?<br>
><br>
</div>I think the tests that are in there are the minimum tests I'd like to<br>
see run. I'll clarify the language on the wiki page a bit.<br></blockquote><div><br></div></div></div><div>If the intention is to have the minimum set of test "we'd like to see run" then it's perfect!</div>
<div>
I was trying to say we should impose that set as a minimum requirement...</div><div class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div><div><br>
> Salvatore<br>
><br>
> [1] <a href="https://wiki.openstack.org/wiki/NeutronThirdPartyTesting" target="_blank">https://wiki.openstack.org/wiki/NeutronThirdPartyTesting</a><br>
><br>
><br>
><br>
><br>
> On 16 June 2014 07:07, YAMAMOTO Takashi <<a href="mailto:yamamoto@valinux.co.jp" target="_blank">yamamoto@valinux.co.jp</a>> wrote:<br>
>><br>
>> hi,<br>
>><br>
>> > My initial analysis of Neutron 3rd Party CI is here [1]. This was<br>
>> > somewhat correlated with information from DriverLog [2], which was<br>
>> > helpful to put this together.<br>
>><br>
>> i updated the etherpad for ofagent.<br>
>> currently a single CI system is running tests for both of ofagent and ryu.<br>
>> is it ok?<br>
>><br>
>> YAMAMOTO Takashi<br>
>><br>
>> _______________________________________________<br>
>> OpenStack-dev mailing list<br>
>> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
>> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
><br>
><br>
> _______________________________________________<br>
> OpenStack-dev mailing list<br>
> <a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
> <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
><br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div></div><br></div></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>