Open Stack

On 06/12/2014 09:36 AM, Sean Dague wrote:

> This is what I mean by functional testing. If we were directly hitting a
> real database on a set of in tree project tests, I think you could
> discover issues like this. Neutron was headed down that path.
>
> But if we're talking about a devstack / tempest run, it's not really
> applicable.
>
> If someone can point me to a case where we've actually found this kind
> of bug with tempest / devstack, that would be great. I've just *never*
> seen it. I was the one that did most of the fixing for pg support in
> Nova, and have helped other projects as well, so I'm relatively familiar
> with the kinds of fails we can discover. The ones that Julien pointed
> really aren't likely to be exposed in our current system.
>
> Which is why I think we're mostly just burning cycles on the existing
> approach for no gain.

What about

https://bugs.launchpad.net/nova/+bug/1292963

Would this have been caught by strict/traditional mode with mysql?  (Of 
course in this case we didn't actually have tempest testcases for server 
groups yet, not sure if they exist now.)

Also, while we're on the topic of testing databases...I opened a bug a 
while back for the fact that sqlite regexp() doesn't behave like 
mysql/postgres.  Having unit tests that don't behave like a real install 
seems like a bad move.

Chris