<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
On Apr 8, 2014, at 3:04 PM, Jamie Lennox <<a href="mailto:jamielennox@redhat.com">jamielennox@redhat.com</a>> wrote:<br>
<div><br class="Apple-interchange-newline">
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
<br>
----- Original Message -----<br>
<blockquote type="cite">From: "Paul Michali (pcm)" <<a href="mailto:pcm@cisco.com">pcm@cisco.com</a>><br>
To: "OpenStack Development Mailing List (not for usage questions)" <<a href="mailto:openstack-dev@lists.openstack.org">openstack-dev@lists.openstack.org</a>><br>
Cc: <a href="mailto:jamielennox@gmail.com">jamielennox@gmail.com</a><br>
Sent: Wednesday, April 9, 2014 12:09:58 AM<br>
Subject: [openstack-dev] [infra]Requesting consideration of httmock package for test-requirements in Juno<br>
<br>
Reposting this, after discussing with Sean Dague…<br>
<br>
For background, I have developed a REST client lib to talk to a H/W device<br>
with REST server for VPNaaS in Neutron. To support unit testing of this, I<br>
created a UT module and a mock REST server module and used the httmock<br>
package. I found it easy to use, and was able to easily create a sub-class<br>
of my UT to run the same test cases with real H/W, instead of the mock REST<br>
server. See the original email below, for links of the UT and REST mock to<br>
see how I used it.<br>
<br>
<br>
I created a bug under requirements, to propose adding httmock to the<br>
test-requirements. Sean mentioned that there is an existing mock package,<br>
called httpretty , which I found is used in keystone client UTs), and should<br>
petition to see if httmock should replace httpretty, since the two appear to<br>
overlap in functionality.<br>
<br>
I found this link, with a brief comparison of the two:<br>
<a href="http://marekbrzoska.wordpress.com/2013/08/28/mocking-http-requests-in-python/">http://marekbrzoska.wordpress.com/2013/08/28/mocking-http-requests-in-python/</a><br>
<br>
So… I’m wondering if the community is interested in adopting this package<br>
(with the goal of deprecating the httpretty package). Otherwise, I will work<br>
on reworking the UT code I have to try to use httpretty.<br>
<br>
Would be interested in peoples’ thoughts, especially those who have worked<br>
with httpretty.<br>
<br>
Thanks in advance!<br>
</blockquote>
<br>
So I introduced HTTPretty into the requirements and did the work around keystoneclient and am well aware that it has a few warts.<br>
</div>
</blockquote>
<div><br>
</div>
PCM: Great, I grabbed your name from keystone client logs and was hoping you had some knowledge of httpretty.</div>
<div><br>
</div>
<div><br>
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
At the time we were going through the changeover from httplib to requests and httpretty gave a good way to change over the library and ensure that we hadn't actually changed the issued requests at all. If we had already been on requests i don't know if i'd
have made the same choice.<br>
<br>
In general I am in favour of mocking the response layer rather than the client layer - whether we do this with httpretty or httmock doesn't bother me that much. Honestly I don't think a global patch of the requests Session object is that much safer that a global
patch of the socket interface, if anything requests is under development and so this interface is less defined.<br>
</div>
</blockquote>
<div><br>
</div>
PCM: Not sure that httmock can be considered a global patch. It is a context lib that intercepts the call through various decorators where the request can be filtered/processed and if not, will fall through and call the actual library.</div>
<div><br>
</div>
<div>So, with the context lib, you can define several handlers for the request(s). When the call is made, it will try each handler and if they all return None, will call the original function, otherwise they return the value of the mock routine. Here’s an
example front he test cases I cerated:</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> with httmock.HTTMock(csr_request.token, csr_request.put,</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> csr_request.normal_get):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> keepalive_info = {'interval': 60, 'retry': 4}</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> <font color="#b51a00">
self.csr.configure_ike_keepalive</font>(keepalive_info)</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> self.assertEqual(requests.codes.NO_CONTENT, self.csr.status)</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> content =
<font color="#b51a00">self.csr.get_request</font>('vpn-svc/ike/keepalive')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> self.assertEqual(requests.codes.OK, self.csr.status)</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> expected = {'periodic': False}</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> expected.update(keepalive_info)</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> self.assertDictContainsSubset(expected, content)</div>
<div><br>
</div>
<div>The client code (red) does a POST with authentication info to get token, does a PUT with the setting, and then a GET to verify the value. The mock module has these methods created:</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">@httmock.urlmatch</span>(<span style="text-decoration: underline">netloc</span>=r'localhost')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">def</span> token(<span style="text-decoration: underline">url</span>, request):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if '<span style="text-decoration: underline">auth</span>/token-services' in url.path:</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.OK,</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> 'content': {'token-id': 'dummy-token'}}</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">@httmock.urlmatch</span>(<span style="text-decoration: underline">netloc</span>=r'localhost')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">def</span> normal_get(<span style="text-decoration: underline">url</span>, request):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if request.method != 'GET':</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return</div>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if not request.headers.get('X-<span style="text-decoration: underline">auth</span>-token', None):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.UNAUTHORIZED}</div>
</div>
<div><span class="Apple-tab-span" style="white-space:pre"></span>…</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if 'vpn-svc/ike/keepalive' in url.path:</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> content = {u'interval': 60,</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> u'retry': 4,</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> u'periodic': True}</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return httmock.response(requests.codes.OK, content=content)</div>
</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">@httmock.urlmatch(netloc=r'localhost')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">def put(url, request):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if request.method != 'PUT':</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if not request.headers.get('X-auth-token', None):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.UNAUTHORIZED}</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.NO_CONTENT}</div>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Just a few notes….</div>
<div>A) Could have created separate context lib for put vs get.</div>
<div>B) Could have a method for specific URI request matches (I do that in some places, in others I catch all requests).</div>
<div>C) Can filter the requests based on request type or URI.</div>
<div>D) Can catch all URLs or filter.</div>
<div>E) Additional decorators can be created for these handlers.</div>
<div>F) There is lots of flexibility with manipulating the response data.</div>
<div><br>
</div>
<div>For (C) I’ve done things like this:</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">@filter_request(['post'], 'vpn-svc/site-to-site')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">@httmock.urlmatch(netloc=r'localhost')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">def post_missing_ipsec_policy(url, request):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> if not request.headers.get('X-auth-token', None):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.UNAUTHORIZED}</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.BAD_REQUEST}</div>
</div>
<div><br>
</div>
<div>For (D), I have all my handlers set for localhost, and then I have another test module that creates a subclass of the test class, sets the host to an IP of a live system, and runs the same tests, only this time against a live router, instead the mock module.
This saves a lot of coding and allows me to reuse the test code (and make sure it REALLY works with real hardware, in my case).</div>
<div><br>
</div>
<div>For (E), I did this to simulate a timeout in a request and then latter success on retry:</div>
<div><br>
</div>
<div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">@filter_request</span>(['get'], 'global/host-name')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;">@repeat(1)</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">@httmock.urlmatch</span>(<span style="text-decoration: underline">netloc</span>=r'localhost')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"><span style="text-decoration: underline">def</span> expired_request(<span style="text-decoration: underline">url</span>, request):</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> """Simulate access denied failure on first request for this resource.</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco; min-height: 15px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> Intent here is to simulate that the token has expired, by failing</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> the first request to the resource. Because of the repeat=1, this</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> will only be called once, and subsequent calls will not be handled</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> by this function, but instead will access the normal handler and</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> will pass. Currently configured for a GET request, but will work</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> with POST and PUT as well. For DELETE, would need to filter_request on a</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> different resource (e.g. 'global/local-users')</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> """</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco; min-height: 15px;">
<br>
</div>
<div style="margin: 0px; font-size: 11px; font-family: Monaco;"> return {'status_code': requests.codes.UNAUTHORIZED}</div>
</div>
<div><br>
</div>
<div>Ref: <a href="https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py">
https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py</a></div>
<div>Ref: <a href="https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/cisco_csr_mock.py">
https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/cisco_csr_mock.py</a></div>
<div><br>
</div>
<div><br>
</div>
</div>
<div>
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
What i would like to see though is this mocking transferred into fixtures like in<span class="Apple-converted-space"> </span><a href="https://review.openstack.org/#/c/77961/">https://review.openstack.org/#/c/77961/</a><span class="Apple-converted-space"> </span>and
have the actual choice in mock library hidden behind those fixtures. Is this a pattern that httmock can handle? or something else again?<br>
</div>
</blockquote>
<div><br>
</div>
PCM I see your point - it would be nice to be able to have the flexibility to swap the underlying mock libraries. Would have to think about it more, to see if a common fixture can be used for httpretty and httmock. I’m not sure if they are different enough
(one monkey patching, one using context lib) that it would be possible (and I don’t have a strong understanding of httpretty yet).</div>
<div><br>
</div>
<div>Would love to hear peoples’ thoughts no this.</div>
<div><br>
</div>
<div><br>
</div>
<div>
<div>
<div>
<div>PCM (Paul Michali)</div>
<div><br>
</div>
<div>MAIL …..…. <a href="mailto:pcm@cisco.com">pcm@cisco.com</a></div>
<div>IRC ……..… pcm_ (<a href="http://irc.freenode.com">irc.freenode.com</a>)</div>
<div>TW ………... @pmichali</div>
<div>GPG Key … 4525ECC253E31A83</div>
<div>Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83</div>
</div>
<div><br>
</div>
<br class="Apple-interchange-newline">
</div>
<br>
<div></div>
<blockquote type="cite">
<div style="font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">
<br>
<br>
Jamie<br>
<br>
<br>
<blockquote type="cite">PCM (Paul Michali)<br>
<br>
MAIL …..…. <a href="mailto:pcm@cisco.com">pcm@cisco.com</a><br>
IRC ……..… pcm_ ( <a href="http://irc.freenode.com">irc.freenode.com</a> )<br>
TW ………... @pmichali<br>
GPG Key … 4525ECC253E31A83<br>
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83<br>
<br>
<br>
<br>
On Apr 4, 2014, at 10:44 AM, Paul Michali (pcm) < <a href="mailto:pcm@cisco.com">
pcm@cisco.com</a> > wrote:<br>
<br>
<br>
<br>
<br>
I’d like to get this added to the test-requirements for Neutron. It is a very<br>
flexible HTTP mock module that works with the Requests package. It is a<br>
decorator that wraps the Request’s send() method and allows easy mocking of<br>
responses, etc (w/o using a web server).<br>
<br>
The bug is: <a href="https://bugs.launchpad.net/neutron/+bug/1282855">https://bugs.launchpad.net/neutron/+bug/1282855</a><br>
<br>
Initially I had requested both httmock and newer requests, but was requested<br>
to separate them, so this is to target httmock as it is more important (to<br>
me :) to get approval,<br>
<br>
<br>
The review request is: <a href="https://review.openstack.org/#/c/75296/">https://review.openstack.org/#/c/75296/</a><br>
<br>
An example of code that would use this:<br>
<br>
<a href="https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py">https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/notest_cisco_csr_rest.py</a><br>
https://github.com/openstack/neutron/blob/master/neutron/tests/unit/services/vpn/device_drivers/cisco_csr_mock.py<br>
<br>
Looking forward to hearing whether or not we can include this package into<br>
Juno.<br>
<br>
Thanks in advance!<br>
<br>
<br>
PCM (Paul Michali)<br>
<br>
MAIL …..…. pcm@cisco.com<br>
IRC ……..… pcm_ ( irc.freenode.com )<br>
TW ………... @pmichali<br>
GPG Key … 4525ECC253E31A83<br>
Fingerprint .. 307A 96BB 1A4C D2C7 931D 8D2D 4525 ECC2 53E3 1A83<br>
<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br>
<br>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
OpenStack-dev@lists.openstack.org<br>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br>
<br>
</blockquote>
<br>
_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a></div>
</blockquote>
</div>
<br>
</body>
</html>