Open Stack

On 12/19/2017 01:53 AM, James E. Blair wrote:
> Ian Wienand <iwienand at redhat.com> writes:
> 
>> There's a bunch of stuff that wouldn't show up until live, but we
>> probably could have got a lot of prep work out of the way if the
>> integration tests were doing something.  I didn't realise that although
>> we run the tests, most of our modules don't actually have any tests
>> run ... even something very simple like "apply without failures"
> 
> Don't the apply tests do that?

Not really; since they do a --noop run they find things like syntax
issues, dependency loops, missing statements etc; but this does leave a
lot of room for other failures.

For example, our version of puppet-nodejs was warning on Xenial "this
platform not supported, I'll try to use sensible defaults", which
passed through the apply tests -- but wasn't actually working when it
came to really getting nodejs on the system alongside
etherpad/ethercalc.

I also think there was some false sense of security since (now called)
legacy-puppet-beaker-rspec-infra was working ... even though *it* was
a noop too.

-i