Open Stack

On 06/16/2014 10:25 AM, Ilya Shakhat wrote:
>>
>>> However, it would be great if we could start devising a solution for
>> having
>>> "health" reports from the various CI systems.
>>> This report should report the following kind of information:
>>> - timestamp of last run
>>> - timestamp of last vote (a system might start job which then get aborted
>>> for CI infra problems)
>>> - % of success vs failures (not sure how important is that one but
>> provides
>>> a metric of comparison with upstream jenkins)
>>> - % of disagreements with jenkins (this might allow us to quickly spot
>> those
>>> CI systems which are randomly -1'ing patches)
>>>
>>> The percentage metrics might be taken over a 48 hours or 7 days
>> interval, or
>>> both.
>>> Does this idea sound reasonable?
>>>
>> This sounds like a very good idea. Now we just need to find someone
>> with the time to write this. :)
> 
> 
> That's exactly what Stackalytics/DriverLog may do! It already collects the
> latest CI votes and it wouldn't be hard to calculate metrics based on them.
> 
> Ilya
Hi Ilya:

Can you add an agenda item to today's Third Party meeting agenda so that
you can outline how you address this currently, any bugs you are aware
of and your current direction of development.

Thanks,
Anita.
> 
> 
> 2014-06-16 18:11 GMT+04:00 Salvatore Orlando <sorlando at nicira.com>:
> 
>>
>>
>>
>> On 16 June 2014 15:58, Kyle Mestery <mestery at noironetworks.com> wrote:
>>
>>> On Mon, Jun 16, 2014 at 8:52 AM, Salvatore Orlando <sorlando at nicira.com>
>>> wrote:
>>>> I will probably be unable, as usual, to attend today's CI meeting (falls
>>>> right around my dinner time).
>>>> I think it's a good idea to starting keeping track of the status of the
>>>> various CI systems, but I feel the etherpad will not work very well in
>>> the
>>>> long term.
>>>>
>>> Agreed. The etherpad was a starting point, I'll move this information
>>> to a wiki page later today.
>>>
>>>> However, it would be great if we could start devising a solution for
>>> having
>>>> "health" reports from the various CI systems.
>>>> This report should report the following kind of information:
>>>> - timestamp of last run
>>>> - timestamp of last vote (a system might start job which then get
>>> aborted
>>>> for CI infra problems)
>>>> - % of success vs failures (not sure how important is that one but
>>> provides
>>>> a metric of comparison with upstream jenkins)
>>>> - % of disagreements with jenkins (this might allow us to quickly spot
>>> those
>>>> CI systems which are randomly -1'ing patches)
>>>>
>>>> The percentage metrics might be taken over a 48 hours or 7 days
>>> interval, or
>>>> both.
>>>> Does this idea sound reasonable?
>>>>
>>> This sounds like a very good idea. Now we just need to find someone
>>> with the time to write this. :)
>>>
>>>> Also, regarding [1]. I agree that more is always better...  but I would
>>> like
>>>> a minimum required set of tests to be enforced.
>>>> Is this something that can be achieved?
>>>>
>>> I think the tests that are in there are the minimum tests I'd like to
>>> see run. I'll clarify the language on the wiki page a bit.
>>>
>>
>> If the intention is to have the minimum set of test "we'd like to see run"
>> then it's perfect!
>> I was trying to say we should impose that set as a minimum requirement...
>>
>>
>>>
>>>> Salvatore
>>>>
>>>> [1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
>>>>
>>>>
>>>>
>>>>
>>>> On 16 June 2014 07:07, YAMAMOTO Takashi <yamamoto at valinux.co.jp> wrote:
>>>>>
>>>>> hi,
>>>>>
>>>>>> My initial analysis of Neutron 3rd Party CI is here [1]. This was
>>>>>> somewhat correlated with information from DriverLog [2], which was
>>>>>> helpful to put this together.
>>>>>
>>>>> i updated the etherpad for ofagent.
>>>>> currently a single CI system is running tests for both of ofagent and
>>> ryu.
>>>>> is it ok?
>>>>>
>>>>> YAMAMOTO Takashi
>>>>>
>>>>> _______________________________________________
>>>>> OpenStack-dev mailing list
>>>>> OpenStack-dev at lists.openstack.org
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> OpenStack-dev mailing list
>>>> OpenStack-dev at lists.openstack.org
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>>
>>>
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>> OpenStack-dev at lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>