[openstack-dev] [cinder][neutron][all] New third-party-ci testing requirements for OpenStack Compatible mark

Chris Hoge chris at openstack.org
Tue Sep 29 15:28:30 UTC 2015


On Sep 29, 2015, at 8:04 AM, Erlon Cruz <sombrafam at gmail.com> wrote:
> 
> Hi Cris,
> 
> There are some questions that came to my mind.
> 
> Cinder has near zero tolerance to backends that does not have a CI running. So, can one assume that all drivers in Cinder will have the "OpenStack Compatible" seal?

One of the reasons we started with Cinder was because they have
have an existing program that is well maintained. Any driver passing
CI becomes eligible for the "OpenStack Compatible” mark. It’s not
automatic, and still needs a signed agreement with the Foundation.

> When you say that the driver have to 'pass' the integration tests, what tests do you consider? All tests in tempest? All patches? Do you have any criteria to determine if a backend is passing or not?

We’re letting the project drive what tests need to be passed. So,
taking a look at this dashboard[1] (it’s one of many that monitor
our test systems) the drivers are running the dsvm-tempest-full
tests. One of the things that the tests exercise, and we’re interested
in from the driver standpoint, are both the user-facing Cinder APIs
as well as the driver-facing APIs.

For Neutron, which we would like to help roll out in the coming year,
this would be a CI run that is defined by the Neutron development
team. We have no interest in dictating to the developers what should
be run. Instead, we want to adopt what the community considers
to be the best-practices and standards for drivers.

> About this "OpenStack Compatible" flag, how does it work? Will you hold a list with the Compatible vendors? Is anything a vendor need to to in order to use this?

“OpenStack Compatible” is one of the trademark programs that is
administered by the Foundation. A company that want to apply the
OpenStack logo to their product needs to sign a licensing agreement,
which gives them the right to use the logo in their marketing materials.

We also create an entry in the OpenStack Marketplace for their
product, which has information about the company and the product, but
also information about tests that the product may have passed. The
best example I can give right now is with the “OpenStack Powered”
program, where we display which Defcore guideline a product has
successfully passed[2].

Chris

[1] http://ci-watch.tintri.com/project?project=cinder&time=24+hours
[2] For example: http://www.openstack.org/marketplace/public-clouds/unitedstack/uos-cloud

> Thanks,
> Erlon
> 
> On Mon, Sep 28, 2015 at 5:55 PM, Kyle Mestery <mestery at mestery.com <mailto:mestery at mestery.com>> wrote:
> The Neutron team also discussed this in Vancouver, you can see the etherpad here [1]. We talked about the idea of creating a validation suite, and it sounds like that's something we should again discuss in Tokyo for the Mitaka cycle. I think a validation suite would be a great step forward for Neutron third-party CI systems to use to validate they work with a release.
> 
> [1] https://etherpad.openstack.org/p/YVR-neutron-third-party-ci-liberty <https://etherpad.openstack.org/p/YVR-neutron-third-party-ci-liberty>
> 
> On Sun, Sep 27, 2015 at 11:39 AM, Armando M. <armamig at gmail.com <mailto:armamig at gmail.com>> wrote:
> 
> 
> On 25 September 2015 at 15:40, Chris Hoge <chris at openstack.org <mailto:chris at openstack.org>> wrote:
> In November, the OpenStack Foundation will start requiring vendors requesting
> new "OpenStack Compatible" storage driver licenses to start passing the Cinder
> third-party integration tests.
> The new program was approved by the Board at
> the July meeting in Austin and follows the improvement of the testing standards
> and technical requirements for the "OpenStack Powered" program. This is all
> part of the effort of the Foundation to use the OpenStack brand to guarantee a
> base-level of interoperability and consistency for OpenStack users and to
> protect the work of our community of developers by applying a trademark backed
> by their technical efforts.
> 
> The Cinder driver testing is the first step of a larger effort to apply
> community determined standards to the Foundation marketing programs. We're
> starting with Cinder because it has a successful testing program in place, and
> we have plans to extend the program to network drivers and OpenStack
> applications. We're going require CI testing for new "OpenStack Compatible"
> storage licenses starting on November 1, and plan to roll out network and
> application testing in 2016.
> 
> One of our goals is to work with project leaders and developers to help us
> define and implement these test programs. The standards for third-party
> drivers and applications should be determined by the developers and users
> in our community, who are experts in how to maintain the quality of the
> ecosystem.
> 
> We welcome and feedback on this program, and are also happy to answer any
> questions you might have.
> 
> Thanks for spearheading this effort.
> 
> Do you have more information/pointers about the program, and how Cinder in particular is 
> paving the way for other projects to follow?
> 
> Thanks,
> Armando
> 
> 
> Thanks!
> 
> Chris Hoge
> Interop Engineer
> OpenStack Foundation
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150929/15d396f4/attachment.html>


More information about the OpenStack-dev mailing list