[openstack-dev] [tempest][nova][defcore] Add option to disable some strict response checking for interop testing

Chris Hoge chris at openstack.org
Tue Jun 21 23:06:51 UTC 2016


> On Jun 20, 2016, at 6:56 AM, Doug Hellmann <doug at doughellmann.com> wrote:
> 
<big snip>
> 
> I'm also, I think, edging away from the "we need to find a compromise"
> camp into the "why is this turning into such a big deal" camp. How did
> we get into a situation where the community has set a clear direction,
> the trademark certification system has a long lead time built in, and
> vendors are still not able to maintain certification?

If I had to make an educated guess, it’s because product managers
have to produce a roadmap with goals and features that consider
both what’s happening upstream, what is currently deployed, and
existing customers.

Just pulling attributes that were once ‘ok’ within the ecosystem and
now aren’t (even with lead time) isn’t as easy as “just change the
response”. It takes time, and given the year-long cycle that DefCore
has adopted for re-testing and the relative youth of the OpenStack
Powered Trademark program, it’s not unsurprising that a few
clouds have been hit by this.

I’ve spoken with all three vendors who are being hit by this, and
two definitely have a longer lead time to work on this. The third
is using the extra responses internally and are currently working
on changing their custom tools to get the extra information in
a different way.

I’ve also spoken with another vendor who is going to be caught
by it, though, and have explained the situation to them and
they are considering their options.

I am now actively telling vendors who are still sending additional
properties to start with with the Nova team upstream to have
their additional properties micro-versioned.

> Are the systems
> running modified versions of newer OpenStack that can't be certified
> with older versions of Tempest?

With the volume of bug fixes, refactoring, and general improvements
from the last year, I’d say no.

-Chris

> 
> Doug

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20160621/e34fe7bc/attachment.html>


More information about the OpenStack-dev mailing list