Open Stack

On 11 October 2016 at 16:23, Clay Gerrard <clay.gerrard at gmail.com> wrote:

> Greetings!
>
> Anyone have any experience to share positive or negative using
> requests-mock?  I see it's been used to replace another dependency that had
> some problems in many of the OpenStack python client libraries:
>
> Added to global requirements -> https://review.openstack.org/#/c/104067
> Added to novaclient -> https://review.openstack.org/#/c/112179/
> Added to cinderclient -> https://review.openstack.org/#/c/106665/
> Added to keystoneclient -> https://review.openstack.org/#/c/106659/
>
> But I'm not sure how folks other than Jamie are getting on with it?  When
> writing new tests do you tend to instinctively grab for requests-mock - or
> do you mostly not notice it's there?  Any unexpected behaviors ever have
> you checking it out with bzr and reading over the source?  Do you recall
> ever having any bumps or bruises in the gate or in your development
> workflow because of a new release from requests-mock?  No presumed fault on
> Jamie!  It seems like he's doing a Herculean one-man job there; but it can
> be difficult go it alone:
>
> https://bugs.launchpad.net/requests-mock/+bug/1616690
>
> It looks like the gate on this project is configured to run nova &
> keystone client tests; so that may be sufficient to catch any sort of issue
> that might come up in something that depends on it?  Presumably once he
> lands a change - he does the update to global-requirements and then all of
> OpenStack get's it from there?
>
> I ask of course because I really don't understand how this works [1] :D
>
> But anyway - Jamie was kind enough to offer to refactor some tests for us
> - but in the process seems to require to bring in these new dependencies -
> so I'm trying to evaluate if I can recommend requiring this code in order
> to develop on swiftclient [2].
>
> Any feedback is greatly appreciated!
>
> -Clay
>
> 1. As you may know (thanks to some recently publicity) swift & swiftclient
> joined OpenStack in the time of dinosaurs with a general policy of trying
> to keep dependencies to a *minimum* - but then one day the policy changed
> to... *always* add dependencies whenever possible?  j/k I'm not acctually
> sure what the OpenStack policy is on dependency hygiene :D  Anyway, I can't
> say *exactly* where that "general policy" came from originally?  Presumably
> crieht or gholt just had some first hand experience that the dependencies
> you choose to add have a HUGE impact on your project over it's lifetime -
> or read something from Joel on Software - http://www.joelonsoftware.com/
> articles/fog0000000007.html - or traveled into the future and read the
> "go proverbs" and google'd "npm breaks internet, again".  Of course they've
> since moved on from OpenStack but the general idea is still something that
> new contributors to swift & swiftclient get acclimated to and the circle of
> confusion continues https://github.com/openstack/swift/blob/master/
> CONTRIBUTING.rst#swift-design-principles - but hey!  maybe I can educate
> myself about the OpenStack policy/process; add this dependency and maybe
> the next one too; then someday break the cycle!?!?
>
> 2. https://review.openstack.org/#/c/360298
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
Clay,

So I'm not going to comment too much on the quality of the library as i
obviously think it's good (and AFAIK we've never broken an API).

The only thing I'd like to clarify is that I'm never coming to clients with
the intent of converting tests to use requests-mock to promote the library.
I'm probably the main person behind most of the keystoneauth work and one
of the biggest problems I've had with trying to convert projects to work
with keystoneauth (and keystoneclient before that) is that mocking out the
request.request function, or the whole keystoneclient Client as swiftclient
does has made the unit tests very strictly locked to the current way things
are done. By converting the clients to a request level mock I've been able
to change the way the clients authenticate whilst proving there are no
change in the requests made.

The same is true of swiftclient, though it's a longer and slower process
there as i've discussed with people in the past - and would like to try
again at summit. Patches [1][2] exists to bring some support of
keystoneauth to swiftclient (which would make the hacking i'm currently
trying to do on glance_store much easier), but ideally i would like to go
further because there is existing and proven support for things like
reauthenticating a token on 401 via keystoneauth rather than the current,
somewhat hacky method, that would greatly simplify the interaction between
the openstack services and swift.

Swiftclient is the last major client to not support keystoneauth, which is
currently a blocker for people experimenting with things such as kerberos
authentication for services, and currently the glance_store/swiftclient
interaction [3] is at the top of my failure list for deprecating the old
ways of consuming keystoneclient [4] - in part at least because it is
implementing it's own connection manager which keystoneauth could do.

Anyway, to summarize, I've started leading with requests-mock changes to a
project simply to decouple the tests from mocking exact function calls with
the bare minimum of implemented attributes which may be called differently
in future. If swiftclient chooses to not accept the dependency we'll find
another way to do keystoneauth integration, it'll probably just involve a
whole lot more hacking with unrelated tests that i personally consider more
dangerous.

Jamie



[1] https://review.openstack.org/#/c/298968/
[2] https://review.openstack.org/#/c/300697/
[3]
https://git.openstack.org/cgit/openstack/glance_store/tree/glance_store/_drivers/swift
[4] https://review.openstack.org/#/c/359707/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20161012/ef08a80d/attachment.html>