Open Stack

On Mon, Jul 1, 2013 at 11:08 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:

> Hi all,
>
> I suspect many of you don't know me as I've only started to get involved in
> OpenStack recently, I work at Rackspace and I'm pretty involved in other
> Python
> open source stuff, notably Django and PyPy, I also serve on the board of
> the
> PSF. So hi<wave />!
>
> I'd like to propose an addition to all of the python-client libraries going
> forwards (and perhaps a requirement for future ones).
>
> What I'd like is for each client library, in addition to the actual
> implementation, is that they ship a fake, in-memory, version of the API.
> The
> fake implementations should take the same arguments, have the same return
> values, raise the same exceptions, and otherwise be identical, besides the
> fact
> that they are entirely in memory and never make network requests.
>
> Why not ``mock.Mock(spec=...)``:
>
> First, for those not familiar with the distinction between fakes and mocks
> (and
> doubles and stubs and ...): http://mumak.net/test-doubles/ is a great
> resource.
> https://www.youtube.com/watch?v=Xu5EhKVZdV8 is also a great resource which
> explains much of what I'm about to say, but better.
>
> Fakes are better than Mocks, for this because:
>
> * Mocks tend to be brittle because they're testing for the implementation,
> and
>   not the interface.
> * Each mock tends to grow its own divergent behaviors, which tend to not be
>   correct.
>
> http://stackoverflow.com/questions/8943022/reactor-stops-between-tests-when-using-twisted-trial-unittest/8947354#8947354
>   explains how to avoid this with fakes
> * Mocks tend to encourage monkey patching, instead of "just passing
> objects as
>   parameters"
>
> Again: https://www.youtube.com/watch?v=Xu5EhKVZdV8 is an amazing resource.
>
>
> This obviously adds a bit of a burden to development of client libraries,
> so
> there needs to be a good justification. Here are the advantages I see:
>


Just as  FYI, nova currently has a different way of addressing the two
points that a fake API server would address.



>
> * Helps flesh out the API: by having a simple implementation it helps in
>   designing a good API.
>

Having a simple API implementation will not help us with having a good API,
as by the time its in code almost too late to change (the window to change
is between coded and released).  Instead a fake API would help validate
that the API follows the specs.  Nova has two ways of doing this now, unit
tests that exercise the APIs, and compare the results against recorded
output (see doc/api_samples/ in nova).  And we have tempest API tests.



> * Helps other projects tests: right now any project which uses an openstack
>   client library has to do something manual in their tests, either add
> their
>   own abstraction layer where they hand write an in-memory implementation,
> or
>   they just monkey patch the socket, http, or client library to not make
>   request. Either direction requires a bunch of work from each and every
>   project using an openstack client. Having these in the core client
> libraries
>   would allow downstream authors to simply swap out Connection classes.
>

Instead of a fake API server with no scheduler, no db etc, nova has a fake
backend.  So the only code path that is different then a real deployment is
which virt driver is used (https://review.openstack.org/#/c/24938/).


>
> I think these benefits out weigh the disadvantages. I'm not sure what the
> procedure for this is going forward. I think to demonstrate this concept it
> should start with a few (or even just one) client libraries, particularly
> ones
> which completely own the resources they serve (e.g. swift, marconi,
> ceilometer,
> trove), as compared to ones that interact more (e.g. neutrino, cinder, and
> nova). This is absolutely something I'm volunteering to work on, but I
> want to
> ensure this is an idea that has general buy in from the community and
> existing
> maintainers, so it doesn't wither.
>
> Thanks,
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130703/942c2b82/attachment.html>