<div dir="ltr">Sorry, accidentally hit the wrong key and message went out.....<div><br></div><div style>Was making a mention about the definition of Success. I thought the debate in the meeting was very productive - when a CI posts a +1 that is success, and when a CI posts a -1 (or no vote with comment) is also a success - as this reflects that the CI is doing what it is suppose to do. </div>
<div style><br></div><div style>So, when it comes to stackalytics, it is more critical to show if a given CI is operational or not - and for how long? </div><div style>Another thing we can debate is how to present the +1/-1 votes by a given CI - unless we have some benchmark, it will be hard to consider success/failure. </div>
<div style><br></div><div style>So, I am of the opinion that, initially, we only report on the operational status and duration of the CIs, and a counter of +1 and -1 votes over a period of time. For example, looking at Arista CI, it has casted 7,958 votes so far and it has been operational for past 6 months. This information is not available anywhere - hence, presenting this kind of information on a dashboard created by Ilya would be very useful to the community as well to the vendors..</div>
<div style><br></div><div style>thoughts? </div><div style><br></div><div style>-Sukhdev</div><div style><br></div><div style><br></div><div style><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">
On Mon, Jun 30, 2014 at 12:49 PM, Sukhdev Kapur <span dir="ltr"><<a href="mailto:sukhdevkapur@gmail.com" target="_blank">sukhdevkapur@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Well, Luke, this is collaborative effort by everybody. By having these CI systems in place ensures that one person's code does not break other person's code and vice versa. Therefore, having these CI systems operational and voting 24x7 is a critical step in achieving this goal. <div>
<br></div><div>However, the details as to how and what should be tested is definitely debatable and the team has done excellent job in converging on that. </div><div><br></div><div>Now, as to the issue at hand which Anita is describing, I attended the meeting this morning and was very pleased with the debate that took place and the definition as to Sucess</div>
</div><div class="gmail_extra"><br><br><div class="gmail_quote"><div><div class="h5">On Mon, Jun 30, 2014 at 12:27 PM, Luke Gorrie <span dir="ltr"><<a href="mailto:luke@tail-f.com" target="_blank">luke@tail-f.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<div dir="ltr"><div>On 30 June 2014 21:08, Anita Kuno <span dir="ltr"><<a href="mailto:anteaya@anteaya.info" target="_blank">anteaya@anteaya.info</a>></span> wrote:<br></div><div class="gmail_extra"><div class="gmail_quote">
<div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div><div><span style="color:rgb(34,34,34)">I am disappointed to realize that Ilya (or stackalytics, I don't know</span><br></div></div>
where this is coming from) is unwilling to cease making up definitions<br>
of success for third party ci systems to allow the openstack community<br>
to arrive at its own definition.<br></blockquote><div><br></div></div><div>There is indeed a risk that the new dashboards won't give a meaningful view of whether a 3rd party CI is voting correctly or not.</div><div><br>
</div>
<div>However, there is an elephant in the room and a much more important problem:</div><div><br></div><div>To measure how accurately a CI is voting says much more about a driver author's "Gerrit-fu" ability to operate a CI system than it does about whether the code they have contributed to OpenStack actually works, and the latter is what is actually important.</div>
<div><br></div><div>To my mind the whole 3rd party testing discussion should refocus on helping developers maintain good working code and less on waving "you will be kicked out of OpenStack if you don't keep your swarm of nebulous daemons running 24/7".</div>
<div><br></div><div><br></div></div></div></div>
<br></div></div><div class="">_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org" target="_blank">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></div></blockquote></div><br></div>
</blockquote></div><br></div>