<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Jul 1, 2013 at 11:08 PM, Alex Gaynor <span dir="ltr"><<a href="mailto:alex.gaynor@gmail.com" target="_blank">alex.gaynor@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Hi all,</div><div><br></div><div>I suspect many of you don't know me as I've only started to get involved in</div>
<div>OpenStack recently, I work at Rackspace and I'm pretty involved in other Python</div>
<div>open source stuff, notably Django and PyPy, I also serve on the board of the</div><div>PSF. So hi<wave />!</div><div><br></div><div>I'd like to propose an addition to all of the python-client libraries going</div>
<div>forwards (and perhaps a requirement for future ones).</div><div><br></div><div>What I'd like is for each client library, in addition to the actual</div><div>implementation, is that they ship a fake, in-memory, version of the API. The</div>
<div>fake implementations should take the same arguments, have the same return</div><div>values, raise the same exceptions, and otherwise be identical, besides the fact</div><div>that they are entirely in memory and never make network requests.</div>
<div><br></div><div>Why not ``mock.Mock(spec=...)``:</div><div><br></div><div>First, for those not familiar with the distinction between fakes and mocks (and</div><div>doubles and stubs and ...): <a href="http://mumak.net/test-doubles/" target="_blank">http://mumak.net/test-doubles/</a> is a great resource.</div>
<div><a href="https://www.youtube.com/watch?v=Xu5EhKVZdV8" target="_blank">https://www.youtube.com/watch?v=Xu5EhKVZdV8</a> is also a great resource which</div><div>explains much of what I'm about to say, but better.</div>
<div><br></div>
<div>Fakes are better than Mocks, for this because:</div><div><br></div><div>* Mocks tend to be brittle because they're testing for the implementation, and</div><div> not the interface.</div><div>* Each mock tends to grow its own divergent behaviors, which tend to not be</div>
<div> correct.</div><div> <a href="http://stackoverflow.com/questions/8943022/reactor-stops-between-tests-when-using-twisted-trial-unittest/8947354#8947354" target="_blank">http://stackoverflow.com/questions/8943022/reactor-stops-between-tests-when-using-twisted-trial-unittest/8947354#8947354</a></div>
<div> explains how to avoid this with fakes</div><div>* Mocks tend to encourage monkey patching, instead of "just passing objects as</div><div> parameters"</div><div><br></div><div>Again: <a href="https://www.youtube.com/watch?v=Xu5EhKVZdV8" target="_blank">https://www.youtube.com/watch?v=Xu5EhKVZdV8</a> is an amazing resource.</div>
<div><br></div><div><br></div><div>This obviously adds a bit of a burden to development of client libraries, so</div><div>there needs to be a good justification. Here are the advantages I see:</div></div></blockquote><div>
<br></div><div><br></div>Just as FYI, nova currently has a different way of addressing the two points that a fake API server would address.<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><br></div><div>* Helps flesh out the API: by having a simple implementation it helps in</div>
<div> designing a good API.</div></div></blockquote><div><br></div><div style>Having a simple API implementation will not help us with having a good API, as by the time its in code almost too late to change (the window to change is between coded and released). Instead a fake API would help validate that the API follows the specs. Nova has two ways of doing this now, unit tests that exercise the APIs, and compare the results against recorded output (see doc/api_samples/ in nova). And we have tempest API tests.</div>
<div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>* Helps other projects tests: right now any project which uses an openstack</div>
<div> client library has to do something manual in their tests, either add their</div><div> own abstraction layer where they hand write an in-memory implementation, or</div>
<div> they just monkey patch the socket, http, or client library to not make</div><div> request. Either direction requires a bunch of work from each and every</div><div> project using an openstack client. Having these in the core client libraries</div>
<div> would allow downstream authors to simply swap out Connection classes.</div></div></blockquote><div> </div><div style>Instead of a fake API server with no scheduler, no db etc, nova has a fake backend. So the only code path that is different then a real deployment is which virt driver is used (<a href="https://review.openstack.org/#/c/24938/">https://review.openstack.org/#/c/24938/</a>).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>I think these benefits out weigh the disadvantages. I'm not sure what the</div>
<div>procedure for this is going forward. I think to demonstrate this concept it</div>
<div>should start with a few (or even just one) client libraries, particularly ones</div><div>which completely own the resources they serve (e.g. swift, marconi, ceilometer,</div><div>trove), as compared to ones that interact more (e.g. neutrino, cinder, and</div>
<div>nova). This is absolutely something I'm volunteering to work on, but I want to</div><div>ensure this is an idea that has general buy in from the community and existing</div><div>maintainers, so it doesn't wither.</div>
<div><br></div><div>Thanks,</div><div>Alex</div><span class=""><font color="#888888"><div><br></div>-- <br><div dir="ltr">"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)<br>
"The people's good is the highest law." -- Cicero<br><div>GPG Key fingerprint: 125F 5C67 DFE9 4084</div></div>
</font></span></div>
<br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br></div></div>