[openstack-dev] [all][tests] Fix it friday! [mock failure in CI]

Thierry Carrez thierry at openstack.org
Fri Jul 10 08:18:52 UTC 2015


Robert Collins wrote:
> Good news everybody, mock 1.1.0 is now out. This backports all the
> improvements over the last couple of years, making it fully
> synchronised with cPython master. Yay.
> 
> Bad news. Lots of unit tests jobs have suffered falled from this.
> 
> But - none of the things I've looked into so far are bugs in mock 1.1.0.
> 
> One of the improvements in mock is to fail when a bad method is called.
> 
> Consider this: https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
> 
> Note the old method: mock_import.assert_called_onec_with(name)
> 
> That method never existed. onec is a typo :).
> 
> mock 1.0.1 silently accepts that - thats part of its job. But, its a
> very fragile API.
> 
> 1.1.0 makes that an error, for methods with assert prefixes - unless
> unsafe is specifically requested. So a big chunk of the failing tests
> are tests that were not testing anything *at all*.
> One common exampled of that is 'assert_called' - another method that
> never ever existed. All our tests using that were testing nothing at
> all.

It's still non-trivial to fix, especially as fixing it may expose a real
bug in the never-actually-tested thing.

Releasing this on a Friday sounds like bad timing, especially without
advance notice that we'd have to rush to fix the dozens of new issues it
would expose.

Now that we have data[1] showing what fails, it's not completely crazy
to pin it and buy us some time ?

[1]
http://lists.openstack.org/pipermail/openstack-stable-maint/2015-July/thread.html

-- 
Thierry Carrez (ttx)



More information about the OpenStack-dev mailing list