Open Stack

Salvatore,

Thank you for your input. I actually took your suggestion about the area to cover for the CI systems and I updated the wiki page for the third party testing in Neutron.
https://wiki.openstack.org/wiki/NeutronThirdPartyTesting

Cheers,

Edgar

From: Salvatore Orlando <sorlando at nicira.com<mailto:sorlando at nicira.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, August 15, 2014 at 4:11 PM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

VMware minesweeper has filters which have been designed to cover the largest possible subset of submissions without add unnecessary load to our scarce resources for CI validation. This is probably why the analysis reveals not all patches are covered.

Therefore our filters exclude neutral changes such as those in README.rst (they're unlikely to crash neutron are they?), or changes in wsgi.py. The latter is because the wsgi framework is shared with all plugins, and when it breaks the NSX plugin it's very likely upstream tests will break too.

I think the areas which are important to cover are:
- neutron/agents/.* (or at least the agents you use)
- neutron/openstack/common/.*
- neutron/notifiers/.* (if your drivers report vif plugging events to nova)
- neutron/db/.*
- neutron/services/${thepluginsyouruninyourci_eveniftheyrenotyours}
- your plugin patch

Salvatore

PS: Openstack proposal bot submission's checks have been suspended for vmware minesweeper about 2 days ago. They will resume soon.


On 16 August 2014 01:02, Edgar Magana <edgar.magana at workday.com<mailto:edgar.magana at workday.com>> wrote:
Ivar,

Very valid point. This is why I need clarification from those CI owner.
I will run a new test with a basic change in the Neutron DB code. That should be covered by almost all CI systems.

Edgar

From: Ivar Lazzaro <ivarlazzaro at gmail.com<mailto:ivarlazzaro at gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Date: Friday, August 15, 2014 at 3:47 PM
To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

Hi Edgar,

Nice shot, to be the inquisitor is not necessarily a bad thing :)

I know some CIs are 'stuck' waiting for bugs to be fixed or certain patches to be merged, but I was wondering if it is a requirement that CIs vote *ALL* the Neutron patches. Some may have missed your call just because of their filters (see [0] section 'what changes to run against').

Cheers,
Ivar.

[0] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting




On Sat, Aug 16, 2014 at 12:35 AM, Edgar Magana <edgar.magana at workday.com<mailto:edgar.magana at workday.com>> wrote:
Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, "Kyle Mestery" <mestery at mestery.com<mailto:mestery at mestery.com>> wrote:

>Folks, I'm not sure if all CI accounts are running sufficient tests.
>Per the requirements wiki page here [1], everyone needs to be running
>more than just Tempest API tests, which I still see most neutron
>third-party CI setups doing. I'd like to ask everyone who operates a
>third-party CI account for Neutron to please look at the link below
>and make sure you are running appropriate tests. If you have
>questions, the weekly third-party meeting [2] is a great place to ask
>questions.
>
>Thanks,
>Kyle
>
>[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
>[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty
>
>_______________________________________________
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140818/e8aa6222/attachment.html>