<div dir="ltr">How do we test to see what is failing in each project with the new version?<div><br></div><div>Also, I'm responsible for the reference to the private mock method in Neutron. That particular reference is to prevent people from patching the same target twice because mock.patch.stopall() unwinds patches in a non-deterministic order in < py34 which leads to leftover monkey patches. These caused completely unrelated unit tests to randomly and inexplicably explode later. More details here: <a href="https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378">https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378</a></div><div><br></div><div>Cheers,</div><div>Kevin Benton</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jul 10, 2015 at 12:45 AM, Robert Collins <span dir="ltr"><<a href="mailto:robertc@robertcollins.net" target="_blank">robertc@robertcollins.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Good news everybody, mock 1.1.0 is now out. This backports all the<br>
improvements over the last couple of years, making it fully<br>
synchronised with cPython master. Yay.<br>
<br>
Bad news. Lots of unit tests jobs have suffered falled from this.<br>
<br>
But - none of the things I've looked into so far are bugs in mock 1.1.0.<br>
<br>
One of the improvements in mock is to fail when a bad method is called.<br>
<br>
Consider this: <a href="https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py" rel="noreferrer" target="_blank">https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py</a><br>
<br>
Note the old method: mock_import.assert_called_onec_with(name)<br>
<br>
That method never existed. onec is a typo :).<br>
<br>
mock 1.0.1 silently accepts that - thats part of its job. But, its a<br>
very fragile API.<br>
<br>
1.1.0 makes that an error, for methods with assert prefixes - unless<br>
unsafe is specifically requested. So a big chunk of the failing tests<br>
are tests that were not testing anything *at all*.<br>
One common exampled of that is 'assert_called' - another method that<br>
never ever existed. All our tests using that were testing nothing at<br>
all.<br>
<br>
<br>
Neutron is failing on a bunch of tests that accessed a private<br>
function inside mock. I'm surprised reviewers didn't spot this, but<br>
_patch isn't part of the public API, and never was.<br>
<br>
Tempest seems to be failing due to a different object being returned -<br>
I haven't dug deep enough to describe the cause in more detail.<br>
<br>
We can probably pin mock back to 1.0.1 locally within projects to gain<br>
breathing room, but given the API improvements in 1.1.0 (see below[1]<br>
as readthedocs is failing to build it due to having an old pip in<br>
their virtualenvs :/) - I think we'll be much better off adopting it<br>
as quickly as we can. NB: with this release we don't need to use<br>
'unittest.mock' for Python3.4 - we can just standardise on using<br>
'mock' across the board, which is much simpler and easier to reason<br>
about (same codebase everywhere[2])<br>
<br>
[2]: Python 2.6 support was dropped in 1.1.0, so we need to use<br>
markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should<br>
kill those of asap).<br>
[1]:<br>
- Issue #23310: Fix MagicMock's initializer to work with __methods__, just<br>
like configure_mock(). Patch by Kasia Jachim.<br>
<br>
- Issue #23568: Add rdivmod support to MagicMock() objects.<br>
Patch by Håkan Lövdahl.<br>
<br>
- Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.<br>
<br>
- Issue #23326: Removed __ne__ implementations. Since fixing default __ne__<br>
implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***<br>
<br>
- Issue #21270: We now override tuple methods in mock.call objects so that<br>
they can be used as normal call attributes.<br>
<br>
- Issue #21256: Printout of keyword args should be in deterministic order in<br>
a mock function call. This will help to write better doctests.<br>
<br>
- Issue #21262: New method assert_not_called for Mock.<br>
It raises AssertionError if the mock has been called.<br>
<br>
- Issue #21238: New keyword argument `unsafe` to Mock. It raises<br>
`AttributeError` incase of an attribute startswith assert or assret.<br>
<br>
- Issue #21239: patch.stopall() didn't work deterministically when the same<br>
name was patched more than once.<br>
<br>
- Issue #21222: Passing name keyword argument to mock.create_autospec now<br>
works.<br>
<br>
- Issue #17826: setting an iterable side_effect on a mock function created by<br>
create_autospec now works. Patch by Kushal Das.<br>
<br>
- Issue #17826: setting an iterable side_effect on a mock function created by<br>
create_autospec now works. Patch by Kushal Das.<br>
<br>
- Issue #20968: unittest.mock.MagicMock now supports division.<br>
Patch by Johannes Baiter.<br>
<br>
- Issue #20189: unittest.mock now no longer assumes that any object for<br>
which it could get an inspect.Signature is a callable written in Python.<br>
Fix courtesy of Michael Foord.<br>
<br>
- Issue #17467: add readline and readlines support to mock_open in<br>
unittest.mock.<br>
<br>
- Issue #17015: When it has a spec, a Mock object now inspects its signature<br>
when matching calls, so that arguments can be matched positionally or<br>
by name.<br>
<br>
- Issue #15323: improve failure message of Mock.assert_called_once_with<br>
<br>
- Issue #14857: fix regression in references to PEP 3135 implicit __class__<br>
closure variable (Reopens issue #12370)<br>
<br>
- Issue #14295: Add unittest.mock<br>
<br>
<br>
-Rob<br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
--<br>
Robert Collins <<a href="mailto:rbtcollins@hp.com">rbtcollins@hp.com</a>><br>
Distinguished Technologist<br>
HP Converged Cloud<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div>Kevin Benton</div></div>
</div>