[openstack-dev] use of the word certified

Mark McLoughlin markmc at redhat.com
Tue Jun 10 16:53:21 UTC 2014


On Tue, 2014-06-10 at 16:09 +0100, Duncan Thomas wrote:
> On 10 June 2014 15:07, Mark McLoughlin <markmc at redhat.com> wrote:
> 
> > Exposing which configurations are actively "tested" is a perfectly sane
> > thing to do. I don't see why you think calling this "certification" is
> > necessary to achieve your goals.
> 
> What is certification except a formal way of saying 'we tested it'? At
> least when you test it enough to have some degree of confidence in
> your testing.
> 
> That's *exactly* what certification means.

I disagree. I think the word has substantially more connotations than
simply "this has been tested".

http://lists.openstack.org/pipermail/openstack-dev/2014-June/036963.html

> > I don't know what you mean be "others
> > imposing their idea of certification".
> 
> I mean that if some company or vendor starts claiming 'Product X is
> certified for use with cinder',

On what basis would any vendor claim such certification?

>  that is bad for the cinder core team,
> since we didn't define what got tested or to what degree.

That sounds like you mean "Storage technology X is certified for use
with Vendor Y OpenStack?".

i.e. that Vendor Y has certified the driver for use with their version
if OpenStack but the Cinder team has no influence over what that means
in practice?

> Whether we like it or not, when something doesn't work in cinder, it
> is rare for people to blame the storage vendor in their complaints.
> 'Cinder is broken' is what we hear (and I've heard it, even though
> what they meant is 'my storage vendor hasn't tested or updated their
> driver in two releases', that isn't what they /said/).

Presumably people are complaining about that driver not working with
some specific downstream version of OpenStack, right? Not e.g.
stable/icehouse devstack or something?

i.e. even aside from the driver, we're already talking about something
we as an upstream project don't control the quality of.

> Since cinder,
> and therefore cinder-core, is going to get the blame, I feel we should
> try to maintain some degree of control over the claims.

I'm starting to see where you're coming from, but I fear this
"certification" thing will make it even worse.

Right now you can easily shrug off any responsibility for the quality of
a third party driver or an untested in-tree driver. Sure, some people
may have unreasonable expectations about such things, but you can't stop
people being idiots. You can better communicate expectations, though,
and that's excellent.

But as soon as you "certify" that driver cinder-core takes on a
responsibility that I would think is unreasonable even if the driver was
tested. "But you said it's certified!"

Is cinder-core really ready to take on responsibility for every issue
users see with "certified" drivers and downstream OpenStack products?

> If we run our own minimal certification program, which is what we've
> started doing (started with a script which did a test run and tried to
> require vendors to run it, that didn't work out well so we're now
> requiring CI integration instead), then we at least have the option of
> saying 'You're running an non-certified product, go talk to your
> vendor' when dealing with the cases we have no control over. Vendors
> that don't follow the CI & cert requirements eventually get their
> driver removed, that simple.

What about issues with a "certified" driver? Don't talk to the vendor,
talk to us instead?

If it's an out-of-tree driver then we say "talk to your vendor". 

If it's an in-tree driver, those actively maintaining the driver provide
"best effort community support" like anything else.

If it's an in-tree driver and isn't being actively maintained, and "best
effort community support" isn't being provided, then we need a way to
communicate that unmaintained status. The level of testing it receives
is what we currently see as the most important aspect, but it's not the
only aspect.

If the user is actually using a distro or other downstream product
rather than pure upstream, it's completely normal for upstream to say
"talk to your distro maintainers or product vendor".

Upstream projects can only provide limited support for even motivated
and clueful users, particularly when those users are actually using
downstream variants of the project. It certainly makes sense to clarify
that, but a "certification" program will actually raise the expectations
users have about the level of support upstream will provide.

Mark.




More information about the OpenStack-dev mailing list