Open Stack

On 12/4/2013 10:16 AM, Nikola Đipanov wrote:
> On 11/19/2013 05:52 PM, Peter Feiner wrote:
>> On Tue, Nov 19, 2013 at 11:19 AM, Chuck Short <chuck.short at canonical.com> wrote:
>>> Hi
>>>
>>>
>>> On Tue, Nov 19, 2013 at 10:43 AM, Peter Feiner <peter at gridcentric.ca> wrote:
>>>>
>>>> A substantive reason for switching from mox to mock is the derelict
>>>> state of mox releases. There hasn't been a release of mox in three
>>>> years: the latest, mox-0.5.3, was released in 2010 [1, 2]. Moreover,
>>>> in the past 3 years, substantial bugs have been fixed in upstream mox.
>>>> For example, with the year-old fix to
>>>> https://code.google.com/p/pymox/issues/detail?id=16, a very nasty bug
>>>> in nova would have been caught by an existing test [3].
>>>>
>>>> Alternatively, a copy of the upstream mox code could be added in-tree.
>>>>
>>> Please no, I think we are in an agreement with mox3 and mock.
>>
>> That's cool. As long as the mox* is phased out, the false-positive
>> test results will be fixed.
>>
>> Of course, there's _another_ alternative, which is to retrofit mox3
>> with the upstream mox fixes (e.g., the bug I cited above exists in
>> mox3). However, the delta between mox3 and upstream mox is pretty huge
>> (I just checked), so effort is probably better spent switching to
>> mock. To that end, I plan on changing the tests I cited above.
>>
>
> Resurrecting this thread because of an interesting review that came up
> yesterday [1].
>
> It seems that our lack of a firm decision on what to do with the mocking
> framework has left people confused. In hope to help - I'll give my view
> of where things are now and what we should do going forward, and
> hopefully we'll reach some consensus on this.
>
> Here's the breakdown:
>
> We should abandon mox:
> * It has not had a release in over 3 years [2] and a patch upstream for 2
> * There are bugs that are impacting the project with it (see above)
> * It will not be ported to python 3
>
> Proposed path forward options:
> 1) Port nova to mock now:
>    * Literally unmanageable - huge review overhead and regression risk
> for not so much gain (maybe) [1]
>
> 2) Opportunistically port nova (write new tests using mock, when fixing
> tests, move them to mock):
>   * Will take a really long time to move to mock, and is not really a
> solution since we are stuck with mock for an undetermined period of time
> - it's what we are doing now (kind of).
>
> 3) Same as 2) but move current codebase to mox3
>   * Buys us py3k compat, and fresher code
>   * Mox3 and mox have diverged and we would need to backport mox fixes
> onto the mox3 three and become de-facto active maintainers (as per Peter
> Feiner's last email - that may not be so easy).
>
> I think we should follow path 3) if we can, but we need to:
>
> 1) Figure out what is the deal with mox3 and decide if owning it will
> really be less trouble than porting nova. To be hones - I was unable to
> even find the code repo for it, only [3]. If anyone has more info -
> please weigh in. We'll also need volunteers
>
> 2) Make better testing guidelines when using mock, and maybe add some
> testing helpers (like we do already have for mox) that will make porting
> existing tests easier. mreidem already put this on this weeks nova
> meeting agenda - so that might be a good place to discuss all the issues
> mentioned here as well.

For anyone interested in seeing the nova meeting agenda notes before the 
actual meeting (since it covers more than just mock/mox), here you go:

https://wiki.openstack.org/wiki/Meetings/Nova

Copying here for reference later if necessary:

- Testing Guides - we need some guide on using mox vs mock; previously 
we were forcing all new tests to use mock but then mox3 was pointed out 
and that seemed to go away, but is anyone using mox3 yet 
(ceilomerclient, heatclient and ironicclient are all moving to mox3).
   - ML on stubs vs mox vs mock: 
http://lists.openstack.org/pipermail/openstack-dev/2013-November/018501.html
   - nova/tests/README.rst is outdated; lots of TBD and last update was 
on 2013/05/16: 
https://github.com/openstack/nova/blob/master/nova/tests/README.rst
     - Note keystone bug 1252454 where the keystone docs were 
referencing the nova testing README
   - Ultimately it seems the best guide is Horizon's: 
http://docs.openstack.org/developer/horizon/topics/testing.html
   - We should have something like the Horizon guide in the overall 
OpenStack hacking guide so the projects can point to a single location, 
http://docs.openstack.org/developer/hacking/, but even that is sparse on 
details for writing unit tests (should almost merge or point to the 
horizon guide), and then the single guide should have notes on stubs vs 
mox vs mock with pros/cons and best practices / pitfalls.

===============

The net problem is we have a lot of various testing guides throughout 
the projects, including hacking guides, readmes, wikis, and devref 
pages, we should really consolidate that into the universal hacking 
guide (IMO) and then integrate Horizon's excellent guide into that along 
with whatever rules we have for mox/mox3/mock.

>
> We should really take a stronger stance on this soon IMHO, as this comes
> up with literally every commit.
>
> Cheers,
>
> Nikola
>
> [1] https://review.openstack.org/#/c/59694/
> [2] https://pypi.python.org/pypi/mox
> [3] https://pypi.python.org/pypi/mox3/0.7.0
>
>> _______________________________________________
>> OpenStack-dev mailing list
>> OpenStack-dev at lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

-- 

Thanks,

Matt Riedemann