Open Stack

Edgar:

The status on the wiki page says "Results are not accurate. Needs clarification from Cisco".
Can you please tell me what we are missing?

-Dane

-----Original Message-----
From: Dane Leblanc (leblancd) 
Sent: Tuesday, August 19, 2014 3:05 PM
To: 'Edgar Magana'; OpenStack Development Mailing List (not for usage questions)
Subject: RE: [openstack-dev] [neutron] [third-party] What tests are required to be run

The APIC CI did run tests against that commit (after some queue latency):

http://128.107.233.28:8080/job/apic/1860/
http://cisco-neutron-ci.cisco.com/logs/apic/1860/

But the review comments never showed up on Gerrit. This seems to be an intermittent quirk of Jenkins/Gerrit: We have 3 CIs triggered from this Jenkins/Gerrit server. Whenever we disable another one of our other Jenkins jobs (in this case, we disabled DFA for some rework), the review comments sometimes stop showing up on Gerrit.

-----Original Message-----
From: Edgar Magana [mailto:edgar.magana at workday.com]
Sent: Tuesday, August 19, 2014 1:33 PM
To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

I was looking to one of the most recent Neutron commits:
https://review.openstack.org/#/c/115175/


I could not find the APIC report.

Edgar

On 8/19/14, 9:48 AM, "Dane Leblanc (leblancd)" <leblancd at cisco.com> wrote:

>From which commit is it missing?
>https://review.openstack.org/#/c/114629/
>https://review.openstack.org/#/c/114393/
>
>-----Original Message-----
>From: Edgar Magana [mailto:edgar.magana at workday.com]
>Sent: Tuesday, August 19, 2014 12:28 PM
>To: Dane Leblanc (leblancd); OpenStack Development Mailing List (not 
>for usage questions)
>Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
>required to be run
>
>Dane,
>
>Are you sure about it?
>I just went to this commit and I could not find the APIC tests.
>
>Thanks,
>
>Edgar
>
>On 8/17/14, 8:47 PM, "Dane Leblanc (leblancd)" <leblancd at cisco.com> wrote:
>
>>Edgar:
>>
>>The Cisco APIC should be reporting results for both APIC-related and 
>>non-APIC related changes now.
>>(See http://cisco-neutron-ci.cisco.com/logs/apic/1738/).
>>
>>Will you be updating the wiki page?
>>
>>-Dane
>>
>>-----Original Message-----
>>From: Dane Leblanc (leblancd)
>>Sent: Friday, August 15, 2014 8:18 PM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
>>required to be run
>>
>>Also, you can add me as a contact person for the Cisco VPNaaS driver.
>>
>>-----Original Message-----
>>From: Dane Leblanc (leblancd)
>>Sent: Friday, August 15, 2014 8:14 PM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: RE: [openstack-dev] [neutron] [third-party] What tests are 
>>required to be run
>>
>>Edgar:
>>
>>For the Notes for the Cisco APIC, can you change the comment "results 
>>are fake" to something like "results are only valid for APIC-related 
>>commits"? I think this more accurately represents our current results 
>>(for reasons we chatted about on another thread).
>>
>>Thanks,
>>Dane
>>
>>-----Original Message-----
>>From: Edgar Magana [mailto:edgar.magana at workday.com]
>>Sent: Friday, August 15, 2014 6:36 PM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: Re: [openstack-dev] [neutron] [third-party] What tests are 
>>required to be run
>>Importance: High
>>
>>Team,
>>
>>I did a quick audit on the Neutron CI. Very sad results. Only few 
>>plugins and drivers are running properly and testing all Neutron commits.
>>I created a report here:
>>https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_P
>>l
>>ugi
>>n
>>_and_Drivers
>>
>>
>>We will discuss the actions to take on the next Neutron IRC meeting. 
>>So please, reach me out to clarify what is the status of your CI.
>>I had two commits to quickly verify the CI reliability:
>>
>>https://review.openstack.org/#/c/114393/
>>
>>https://review.openstack.org/#/c/40296/
>>
>>
>>I would expect all plugins and drivers passing on the first one and 
>>failing for the second but I got so many surprises.
>>
>>Neutron code quality and reliability is a top priority, if you ignore 
>>this report that plugin/driver will be candidate to be remove from 
>>Neutron tree.
>>
>>Cheers,
>>
>>Edgar
>>
>>P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty 
>>job!
>>
>>
>>On 8/14/14, 8:30 AM, "Kyle Mestery" <mestery at mestery.com> wrote:
>>
>>>Folks, I'm not sure if all CI accounts are running sufficient tests.
>>>Per the requirements wiki page here [1], everyone needs to be running 
>>>more than just Tempest API tests, which I still see most neutron 
>>>third-party CI setups doing. I'd like to ask everyone who operates a 
>>>third-party CI account for Neutron to please look at the link below 
>>>and make sure you are running appropriate tests. If you have 
>>>questions, the weekly third-party meeting [2] is a great place to ask 
>>>questions.
>>>
>>>Thanks,
>>>Kyle
>>>
>>>[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
>>>[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
>>>
>>>_______________________________________________
>>>OpenStack-dev mailing list
>>>OpenStack-dev at lists.openstack.org
>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>_______________________________________________
>>OpenStack-dev mailing list
>>OpenStack-dev at lists.openstack.org
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>_______________________________________________
>>OpenStack-dev mailing list
>>OpenStack-dev at lists.openstack.org
>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>