Open Stack

I've spent some time doing some initial analysis of 3rd Party CI in
Neutron. The tl;dr is that it's a mess, and it needs fixing. And I'm
setting a deadline of Juno-2 for people to address their CI systems
and get them in shape or we will remove plugins and drivers in Juno-3
which do not meet the expectations set out below.

My initial analysis of Neutron 3rd Party CI is here [1]. This was
somewhat correlated with information from DriverLog [2], which was
helpful to put this together.

As you can see from the list, there are a lot of CI systems which are
broken right now. Some have just recently started working again.
Others are working great, and some are in the middle somewhere. The
overall state isn't that great. I'm sending this email to
openstack-dev and BCC;ing CI owners to raise awareness of this issue.
If I have incorrectly labeled your CI, please update the etherpad and
include links to the latest voting/comments your CI system has done
upstream and reply to this thread.

I have documented the 3rd Party CI requirements for Neutron here [3].
I expect people to be following these guidelines for their CI systems.
If there are questions on the guidelines or expectations, please reply
to this thread or reach out to myself in #openstack-neutron on
Freenode. There is also a third-party meeting [4] which is a great
place to ask questions and share your experience setting up a 3rd
party CI system. The infra team has done a great job sponsoring and
running this meeting (thanks Anita!), so please both take advantage of
it and also contribute to it so we can all share knowledge and help
each other.

Owners of plugins/drivers should ensure their CI is matching the
requirements set forth by both infra and Neutron when running tests
and posting results. Like I indicated earlier, we will look at
removing code for drivers which are not meeting these requirements as
set forth in the wiki pages.

The goal of this effort is to ensure consistency across testing
platforms, making it easier for developers to diagnose issues when
third party CI systems fail, and to ensure these drivers are tested
since they are part of the integrated releases we perform. We used to
require a core team member to sponsor a plugin/driver, but we moved to
the 3rd party CI system in Icehouse instead. Ensuring these systems
are running and properly working is the only way we can ensure code is
working when it's part of the integrated release.

Thanks,
Kyle

[1] https://etherpad.openstack.org/p/ZLp9Ow3tNq
[2] http://www.stackalytics.com/driverlog/?project_id=openstack%2Fneutron&vendor=&release_id=
[3] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[4] https://wiki.openstack.org/wiki/Meetings/ThirdParty