Open Stack

Yong, if you read Mark's proposal closely, the third party tests will only run when the specific third party code is touched, or when the Jenkins user submits code.

On Nov 17, 2013, at 11:50 PM, Yongsheng Gong <gongysh at unitedstack.com> wrote:

> For third party testing, I am afraid these tests will make the patch merge process much longer since each patch, which even has nothing to do with the specific plugins, will triggers the unwanted third party testing jobs.
> 
> 
> On Fri, Nov 15, 2013 at 4:20 AM, Mark McClain <mark.mcclain at dreamhost.com> wrote:
> tl;dr
> -----
> The Neutron team has experienced tremendous growth in vendor plugins and drivers over the last few cycles.  As a result of the growth, the Neutron team is implementing new requirements for plugin and driver code for Icehouse cycle to ensure continued code quality and stability.
> 
> - Each third party plugin/driver shall designate a point of contact for the coordinated release cycle.
> - To be designated as compatible, a third-party plugin and/or driver code must implement external third party integration testing.
> - Policy is in effect immediately for new plugin/drivers.
> - Existing plugin/drivers have until Icehouse-2 to become compliant.
> 
> Introduction
> -----------
> Ensuring release quality through proper testing is an important tenant of the OpenStack community and Neutron team wants to do our part. We are introducing changes below provide more visibility into the quality and stability of vendor plugin and driver code.  The policies described here are in effect immediately.
> 
> Rationale
> --------
> Code proposals for third party plugins have always presented a review challenge for the Neutron core team.  In the early days, code was often proposed by core project contributors and our review process only validated whether the requirements were met for community coding style and unit testing.  As Neutron has added new resources via extensions, it has become more difficult for Neutron reviewers to ensure the proposed code is functional.  Many of the plugins and/or drivers require proprietary hardware and/or software to conduct such testing.
> 
> In addition to testing changes, the Neutron team is revising the requirements for the point of contact for third party code.  The changes bring the written expectations for contacts in line with current practice.
> 
> Point of Contact Requirements
> -----------------------------
> Each third party plugin and/or driver shall designate a point of contact for each coordinated release cycle.  The contact will serve as a liaison between the Neutron core team and the vendor or community supporting the plugin or driver.  The contact shall:
> 	- Attend weekly Neutron team IRC meetings
> 	- Be an active reviewer and contributor 
> 	- Be an active participant on openstack-dev mailing list
> 	- Assist the core team with triaging bugs specific to the plugin and/or driver
> 	- Ensure OpenStack development deadlines are properly communicated back to their company and/or community
> 
> NOTE: The this information can be maintained here: https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers
> 
> Testing Requirements
> -----------------
> To be designated as compatible, a third-party plugin and/or driver code must implement external third party testing.  The testing should be Tempest executed against a Devstack build with the proposed code changes.  The environment managed by the vendor should be configured to incorporate the plugin and/or driver solution.  The OpenStack Infrastructure team has provided details on how to integrate 3rd party testing at:
> 
> http://ci.openstack.org/third_party.html
> 
> and Tempest can be found at:
> 
> https://github.com/openstack/tempest
> 
> The Neutron team expects that the third party testing will provide a +/-1 verify vote for all changes to a plugin or driver’s code.  In addition, the Neutron team expects that the third party test will also vote on all code submissions by the jenkins user.  The jenkins user regularly submits requirements changes and the Neutron team hopes to catch any possible regressions as early as possible.
> 
> Existing Plugin and Drivers
> -------------------------
> Plugins and drivers currently in the Neutron project repository will be given a grace period until the Icehouse-2 milestone to implement external third party testing.  At that time, the Neutron team will release a list of the compatible plugins and drivers (i.e. those that meet the testing requirements).  Plugins and drivers that do not have external testing will be deprecated at the Icehouse release and will be candidates for removal when the J-release cycle opens.
> 
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev