Open Stack

How do we test to see what is failing in each project with the new version?

Also, I'm responsible for the reference to the private mock method in
Neutron. That particular reference is to prevent people from patching the
same target twice because mock.patch.stopall() unwinds patches in a
non-deterministic order in < py34 which leads to leftover monkey patches.
These caused completely unrelated unit tests to randomly and inexplicably
explode later. More details here:
https://github.com/openstack/neutron/commit/1b60df85ba3ad442c2e4e7e52538e1b9a1bf9378

Cheers,
Kevin Benton

On Fri, Jul 10, 2015 at 12:45 AM, Robert Collins <robertc at robertcollins.net>
wrote:

> Good news everybody, mock 1.1.0 is now out. This backports all the
> improvements over the last couple of years, making it fully
> synchronised with cPython master. Yay.
>
> Bad news. Lots of unit tests jobs have suffered falled from this.
>
> But - none of the things I've looked into so far are bugs in mock 1.1.0.
>
> One of the improvements in mock is to fail when a bad method is called.
>
> Consider this:
> https://review.openstack.org/#/c/200384/1/taskflow/tests/unit/test_engine_helpers.py
>
> Note the old method: mock_import.assert_called_onec_with(name)
>
> That method never existed. onec is a typo :).
>
> mock 1.0.1 silently accepts that - thats part of its job. But, its a
> very fragile API.
>
> 1.1.0 makes that an error, for methods with assert prefixes - unless
> unsafe is specifically requested. So a big chunk of the failing tests
> are tests that were not testing anything *at all*.
> One common exampled of that is 'assert_called' - another method that
> never ever existed. All our tests using that were testing nothing at
> all.
>
>
> Neutron is failing on a bunch of tests that accessed a private
> function inside mock. I'm surprised reviewers didn't spot this, but
> _patch isn't part of the public API, and never was.
>
> Tempest seems to be failing due to a different object being returned -
> I haven't dug deep enough to describe the cause in more detail.
>
> We can probably pin mock back to 1.0.1 locally within projects to gain
> breathing room, but given the API improvements in 1.1.0 (see below[1]
> as readthedocs is failing to build it due to having an old pip in
> their virtualenvs :/) - I think we'll be much better off adopting it
> as quickly as we can. NB: with this release we don't need to use
> 'unittest.mock' for Python3.4 - we can just standardise on using
> 'mock' across the board, which is much simpler and easier to reason
> about (same codebase everywhere[2])
>
> [2]: Python 2.6 support was dropped in 1.1.0, so we need to use
> markers to select 1.0.1 for the remaining 2.6 gate jobs. (We should
> kill those of asap).
> [1]:
> - Issue #23310: Fix MagicMock's initializer to work with __methods__, just
>   like configure_mock().  Patch by Kasia Jachim.
>
> - Issue #23568: Add rdivmod support to MagicMock() objects.
>   Patch by Håkan Lövdahl.
>
> - Issue #23581: Add matmul support to MagicMock. Patch by Håkan Lövdahl.
>
> - Issue #23326: Removed __ne__ implementations.  Since fixing default
> __ne__
>   implementation in issue #21408 they are redundant. *** NOT BACKPORTED ***
>
> - Issue #21270: We now override tuple methods in mock.call objects so that
>   they can be used as normal call attributes.
>
> - Issue #21256: Printout of keyword args should be in deterministic order
> in
>   a mock function call. This will help to write better doctests.
>
> - Issue #21262: New method assert_not_called for Mock.
>   It raises AssertionError if the mock has been called.
>
> - Issue #21238: New keyword argument `unsafe` to Mock. It raises
>   `AttributeError` incase of an attribute startswith assert or assret.
>
> - Issue #21239: patch.stopall() didn't work deterministically when the same
>   name was patched more than once.
>
> - Issue #21222: Passing name keyword argument to mock.create_autospec now
>   works.
>
> - Issue #17826: setting an iterable side_effect on a mock function created
> by
>   create_autospec now works. Patch by Kushal Das.
>
> - Issue #17826: setting an iterable side_effect on a mock function created
> by
>   create_autospec now works. Patch by Kushal Das.
>
> - Issue #20968: unittest.mock.MagicMock now supports division.
>   Patch by Johannes Baiter.
>
> - Issue #20189: unittest.mock now no longer assumes that any object for
>   which it could get an inspect.Signature is a callable written in Python.
>   Fix courtesy of Michael Foord.
>
> - Issue #17467: add readline and readlines support to mock_open in
>   unittest.mock.
>
> - Issue #17015: When it has a spec, a Mock object now inspects its
> signature
>   when matching calls, so that arguments can be matched positionally or
>   by name.
>
> - Issue #15323: improve failure message of Mock.assert_called_once_with
>
> - Issue #14857: fix regression in references to PEP 3135 implicit __class__
>   closure variable (Reopens issue #12370)
>
> - Issue #14295: Add unittest.mock
>
>
> -Rob
>
>
>
> --
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Converged Cloud
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150710/ca072286/attachment.html>