Open Stack

I'm looking at Nova unit tests, there are at least 3 issues.

The first one is assert_has_calls *used* to work with a single value

m.assert_has_calls(foo)

The documentation says m.assert_has_calls([foo]) is what you should use,
but the other form used to work. That appears to be tightened up.

It would have been polite to call that out in release notes. Bug filed
here - https://github.com/testing-cabal/mock/issues/263

The second is an issue around the use of autospec (which doesn't do
what's expected any more). Robert figured out a work around -
https://github.com/testing-cabal/mock/issues/264

The last one is still unknown:
https://review.openstack.org/#/c/200500/1/nova/tests/unit/virt/hyperv/test_vhdutils.py,cm

And we're going to just skip that one test until it's resolved. Nova bug
is here - https://bugs.launchpad.net/nova/+bug/1473401 however that
upstream bug should be filed.

Hopefully this set of changes will be helpful with other projects
rushing to try to get tests working again.

	-Sean


On 07/10/2015 03:45 AM, Robert Collins wrote:
> Good news everybody, mock 1.1.0 is now out. This backports all the
> improvements over the last couple of years, making it fully
> synchronised with cPython master. Yay.
> 
> Bad news. Lots of unit tests jobs have suffered falled from this.
> 
> But - none of the things I've looked into so far are bugs in mock 1.1.0.
> 
> One of the improvements in mock is to fail when a bad method is called.
> 
> Consider this: https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
> 
> Note the old method: mock_import.assert_called_onec_with(name)
> 
> That method never existed. onec is a typo :).
> 
> mock 1.0.1 silently accepts that - thats part of its job. But, its a
> very fragile API.
> 
> 1.1.0 makes that an error, for methods with assert prefixes - unless
> unsafe is specifically requested. So a big chunk of the failing tests
> are tests that were not testing anything *at all*.
> One common exampled of that is 'assert_called' - another method that
> never ever existed. All our tests using that were testing nothing at
> all.
> 
> 
> Neutron is failing on a bunch of tests that accessed a private
> function inside mock. I'm surprised reviewers didn't spot this, but
> _patch isn't part of the public API, and never was.
> 
> Tempest seems to be failing due to a different object being returned -
> I haven't dug deep enough to describe the cause in more detail.
> 
> We can probably pin mock back to 1.0.1 locally within projects to gain
> breathing room, but given the API improvements in 1.1.0 (see below[1]
> as readthedocs is failing to build it due to having an old pip in
> their virtualenvs :/) - I think we'll be much better off adopting it
> as quickly as we can. NB: with this release we don't need to use
> 'unittest.mock' for Python3.4 - we can just standardise on using
> 'mock' across the board, which is much simpler and easier to reason
> about (same codebase everywhere[2])
> 
> [2]: Python 2.6 support was dropped in 1.1.0, so we need to use
> markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
> kill those of asap).
> [1]:
> - Issue #23310: Fix MagicMock's initializer to work with __methods__, just
>   like configure_mock().  Patch by Kasia Jachim.
> 
> - Issue #23568: Add rdivmod support to MagicMock() objects.
>   Patch by Håkan Lövdahl.
> 
> - Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.
> 
> - Issue #23326: Removed __ne__ implementations.  Since fixing default __ne__
>   implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***
> 
> - Issue #21270: We now override tuple methods in mock.call objects so that
>   they can be used as normal call attributes.
> 
> - Issue #21256: Printout of keyword args should be in deterministic order in
>   a mock function call. This will help to write better doctests.
> 
> - Issue #21262: New method assert_not_called for Mock.
>   It raises AssertionError if the mock has been called.
> 
> - Issue #21238: New keyword argument `unsafe` to Mock. It raises
>   `AttributeError` incase of an attribute startswith assert or assret.
> 
> - Issue #21239: patch.stopall() didn't work deterministically when the same
>   name was patched more than once.
> 
> - Issue #21222: Passing name keyword argument to mock.create_autospec now
>   works.
> 
> - Issue #17826: setting an iterable side_effect on a mock function created by
>   create_autospec now works. Patch by Kushal Das.
> 
> - Issue #17826: setting an iterable side_effect on a mock function created by
>   create_autospec now works. Patch by Kushal Das.
> 
> - Issue #20968: unittest.mock.MagicMock now supports division.
>   Patch by Johannes Baiter.
> 
> - Issue #20189: unittest.mock now no longer assumes that any object for
>   which it could get an inspect.Signature is a callable written in Python.
>   Fix courtesy of Michael Foord.
> 
> - Issue #17467: add readline and readlines support to mock_open in
>   unittest.mock.
> 
> - Issue #17015: When it has a spec, a Mock object now inspects its signature
>   when matching calls, so that arguments can be matched positionally or
>   by name.
> 
> - Issue #15323: improve failure message of Mock.assert_called_once_with
> 
> - Issue #14857: fix regression in references to PEP 3135 implicit __class__
>   closure variable (Reopens issue #12370)
> 
> - Issue #14295: Add unittest.mock
> 
> 
> -Rob
> 
> 
> 


-- 
Sean Dague
http://dague.net