Open Stack

On 06/10/2014 12:32 PM, Sean Dague wrote:
> On 06/10/2014 11:37 AM, Jay Pipes wrote:
>> On 06/10/2014 09:53 AM, Sean Dague wrote:
>>> On 06/10/2014 09:14 AM, Anita Kuno wrote:
>>>> On 06/10/2014 04:33 AM, Mark McLoughlin wrote:
>>>>> On Mon, 2014-06-09 at 20:14 -0400, Doug Hellmann wrote:
>>>>>> On Mon, Jun 9, 2014 at 6:11 PM, Eoghan Glynn
>>>>>> <eglynn at redhat.com> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> Based on the discussion I'd like to propose these
>>>>>>>> options: 1. Cinder-certified driver - This is an
>>>>>>>> attempt to move the "certification" to the project
>>>>>>>> level. 2. CI-tested driver - This is probably the most
>>>>>>>> accurate, at least for what we're trying to achieve for
>>>>>>>> Juno: Continuous Integration of Vendor-specific
>>>>>>>> Drivers.
>>>>>>>
>>>>>>> Hi Ramy,
>>>>>>>
>>>>>>> Thanks for these constructive suggestions.
>>>>>>>
>>>>>>> The second option is certainly a very direct and
>>>>>>> specific reflection of what is actually involved in
>>>>>>> getting the Cinder project's imprimatur.
>>>>>>
>>>>>> I do like "tested."
>>>>>>
>>>>>> I'd like to understand what the foundation is planning for
>>>>>> "certification" as well, to know how big of an issue this
>>>>>> really is. Even if they aren't going to certify drivers, I
>>>>>> have heard discussions around training and possibly other
>>>>>> areas so I would hate for us to introduce confusion by
>>>>>> having different uses of that term in similar contexts.
>>>>>> Mark, do you know who is working on that within the board
>>>>>> or foundation?
>>>>>
>>>>> http://blogs.gnome.org/markmc/2014/05/17/may-11-openstack-foundation-board-meeting/
>>>>>
>>>>>
>>>>>
>>>>>
Boris Renski raised the possibility of the Foundation attaching the
>>>>> trademark to a verified, certified or tested status for
>>>>> drivers. It wasn't discussed at length because board members
>>>>> hadn't been briefed in advance, but I think it's safe to say
>>>>> there was a knee-jerk negative reaction from a number of
>>>>> members. This is in the context of the DriverLog report:
>>>>>
>>>>> http://stackalytics.com/report/driverlog
>>>>>
>>>>> http://www.mirantis.com/blog/cloud-drivers-openstack-driverlog-part-1-solving-driver-problem/
>>>>>
>>>>>
>>>>>
>>>>> http://www.mirantis.com/blog/openstack-will-open-source-vendor-certifications/
>>>>>
>>>>>
>>>>>
>>>>>
AIUI the "CI tested" phrase was chosen in DriverLog to avoid the
>>>>> controversial area Boris describes in the last link above. I
>>>>> think that makes sense. Claiming this CI testing replaces
>>>>> more traditional certification programs is a sure way to bog
>>>>> potentially useful collaboration down in vendor politics.
>>>> Actually FWIW the DriverLog is not posting accurate
>>>> information, I came upon two instances yesterday where I found
>>>> the information "questionable" at best. I know I questioned it.
>>>> Kyle and I have agreed to not rely on the DriverLog information
>>>> as it currently stands as a way of assessing the fitness of
>>>> third party CI systems. I'll add some footnotes for those who
>>>> want more details. [%%], [++], [&&]
>>>>>
>>>>> Avoiding dragging the project into those sort of politics is
>>>>> something I'm really keen on, and why I think the word
>>>>> "certification" is best avoided so we can focus on what we're
>>>>> actually trying to achieve.
>>>>>
>>>>> Mark.
>>>> I agree with Mark, everytime we try to 'abstract' away from
>>>> logs and put an new interface on it, the focus moves to the
>>>> interface and folks stop paying attention to logs. We archive
>>>> and have links to artifacts for a reason and I think we need to
>>>> encourage and support people to access these artifacts and draw
>>>> their own conclusions, which is in keeping with our license.
>>>>
>>>> Copy/pasting Mark here: "Also AIUI "certification" implies some
>>>> level of warranty or guarantee, which goes against the pretty
>>>> clear language "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND"
>>>> in our license :)" [**]
>>>
>>> Honestly, the bigger issue I've got at this point is that
>>> driverlog is horribly inaccurate. Based on DriverLog you'd see
>>> that we don't test KVM or QEMU at all, only XenAPI.
>>
>> Then shouldn't the focus be on both reporting bugs to DriverLog [1]
>> and fixing these inaccuracies? DriverLog doesn't use the term
>> "certified" anywhere, for the record.
>>
>> It is an honest best effort to provide some insight into the
>> testability of various drivers in the OpenStack ecosystem in a more
>> up-to-date way than outdated wiki pages showing matrixes of support
>> for something.
>>
>> It's an alpha project that can and will have bugs. I can
>> absolutely guarantee you that the developers of the DriverLog
>> project are more interested in getting accurate information shown
>> in the interface than with any of the politics around the word
>> "certified".
>
> That seemed like a pretty obvious error. :)

I'd rather have the errors be obvious and correctable than obscure and
hidden behind some admin curtain.

> If we're calling it alpha than perhaps it shouldn't be presented to
> users when they go to stackalytics, which has largely become the
> defacto place where press an analysts go to get project statistics?
>
> I'm fine with it being alpha, and treated as such, off in a corner.
> But it seems to be presented front and center with stackalytics, so
> really needs to be held to a higher standard.

So, if this is all about the placement of the DriverLog button on the
stackalytics page, then we should talk about that separately from a
discussion of the data that is exposed in the interface.

To be plain, we tried to raise awareness of DriverLog and the data
behind it (and how it's all generated), but nobody was all that
interested until someone got upset that a driver that wasn't recently
verified showed up with a green checkbox next to it.

We've been asking for input on the thing for over a month, but have
received little from anyone.

Best,
-jay