Open Stack

On 09/29/2015 11:28 AM, Chris Hoge wrote:
> On Sep 29, 2015, at 8:04 AM, Erlon Cruz <sombrafam at gmail.com> wrote:
>>
>> Hi Cris,
>>
>> There are some questions that came to my mind.
>>
>> Cinder has near zero tolerance to backends that does not have a CI running. So, can one assume that all drivers in Cinder will have the "OpenStack Compatible" seal?
> 
> One of the reasons we started with Cinder was because they have
> have an existing program that is well maintained. Any driver passing
> CI becomes eligible for the "OpenStack Compatible” mark. It’s not
> automatic, and still needs a signed agreement with the Foundation.
> 
>> When you say that the driver have to 'pass' the integration tests, what tests do you consider? All tests in tempest? All patches? Do you have any criteria to determine if a backend is passing or not?
> 
> We’re letting the project drive what tests need to be passed. So,
> taking a look at this dashboard[1] (it’s one of many that monitor
> our test systems)

Dashboards, this and any other, aggregate results (build succeeded,
build failed) that are reported back to Gerrit. They don't index the
logs to evaluate if expected test output is present in the logs. If
aggregated results suit your purpose, then fine, this tool is helpful,
but let's not ascribe responsibility to a tool that isn't performing
that action.

Tools usually monitor the stream of comments as they are broadcast from
Gerrit (stream-events). Taking these reported status comments and
aggregating them into a visual digestible form is all they are doing.


> the drivers are running the dsvm-tempest-full
> tests.

Drivers are reporting they are running the tests.

If we are going to be relying on dashboards and aggregating tools for
making decisions lets be sure we are mindful of exactly what information
is being conveyed and what assumptions are being made on top of that
information.

Thank you,
Anita.

> One of the things that the tests exercise, and we’re interested
> in from the driver standpoint, are both the user-facing Cinder APIs
> as well as the driver-facing APIs.
> 
> For Neutron, which we would like to help roll out in the coming year,
> this would be a CI run that is defined by the Neutron development
> team. We have no interest in dictating to the developers what should
> be run. Instead, we want to adopt what the community considers
> to be the best-practices and standards for drivers.
> 
>> About this "OpenStack Compatible" flag, how does it work? Will you hold a list with the Compatible vendors? Is anything a vendor need to to in order to use this?
> 
> “OpenStack Compatible” is one of the trademark programs that is
> administered by the Foundation. A company that want to apply the
> OpenStack logo to their product needs to sign a licensing agreement,
> which gives them the right to use the logo in their marketing materials.
> 
> We also create an entry in the OpenStack Marketplace for their
> product, which has information about the company and the product, but
> also information about tests that the product may have passed. The
> best example I can give right now is with the “OpenStack Powered”
> program, where we display which Defcore guideline a product has
> successfully passed[2].
> 
> Chris
> 
> [1] http://ci-watch.tintri.com/project?project=cinder&time=24+hours
> [2] For example: http://www.openstack.org/marketplace/public-clouds/unitedstack/uos-cloud
> 
>> Thanks,
>> Erlon
>>
>> On Mon, Sep 28, 2015 at 5:55 PM, Kyle Mestery <mestery at mestery.com <mailto:mestery at mestery.com>> wrote:
>> The Neutron team also discussed this in Vancouver, you can see the etherpad here [1]. We talked about the idea of creating a validation suite, and it sounds like that's something we should again discuss in Tokyo for the Mitaka cycle. I think a validation suite would be a great step forward for Neutron third-party CI systems to use to validate they work with a release.
>>
>> [1] https://etherpad.openstack.org/p/YVR-neutron-third-party-ci-liberty <https://etherpad.openstack.org/p/YVR-neutron-third-party-ci-liberty>
>>
>> On Sun, Sep 27, 2015 at 11:39 AM, Armando M. <armamig at gmail.com <mailto:armamig at gmail.com>> wrote:
>>
>>
>> On 25 September 2015 at 15:40, Chris Hoge <chris at openstack.org <mailto:chris at openstack.org>> wrote:
>> In November, the OpenStack Foundation will start requiring vendors requesting
>> new "OpenStack Compatible" storage driver licenses to start passing the Cinder
>> third-party integration tests.
>> The new program was approved by the Board at
>> the July meeting in Austin and follows the improvement of the testing standards
>> and technical requirements for the "OpenStack Powered" program. This is all
>> part of the effort of the Foundation to use the OpenStack brand to guarantee a
>> base-level of interoperability and consistency for OpenStack users and to
>> protect the work of our community of developers by applying a trademark backed
>> by their technical efforts.
>>
>> The Cinder driver testing is the first step of a larger effort to apply
>> community determined standards to the Foundation marketing programs. We're
>> starting with Cinder because it has a successful testing program in place, and
>> we have plans to extend the program to network drivers and OpenStack
>> applications. We're going require CI testing for new "OpenStack Compatible"
>> storage licenses starting on November 1, and plan to roll out network and
>> application testing in 2016.
>>
>> One of our goals is to work with project leaders and developers to help us
>> define and implement these test programs. The standards for third-party
>> drivers and applications should be determined by the developers and users
>> in our community, who are experts in how to maintain the quality of the
>> ecosystem.
>>
>> We welcome and feedback on this program, and are also happy to answer any
>> questions you might have.
>>
>> Thanks for spearheading this effort.
>>
>> Do you have more information/pointers about the program, and how Cinder in particular is 
>> paving the way for other projects to follow?
>>
>> Thanks,
>> Armando
>>
>>
>> Thanks!
>>
>> Chris Hoge
>> Interop Engineer
>> OpenStack Foundation
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe <http://OpenStack-dev-request@lists.openstack.org/?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev <http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev>
>>
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> 
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>