Open Stack

On 06/17/2016 04:16 PM, Matt Riedemann wrote:
> I was reviewing this today:
> 
> https://review.openstack.org/#/c/326940/
> 
> And I said to myself, 'self, do we really need to subclass the API
> samples functional tests for this microversion given this change doesn't
> modify the request/response body, it's only adding paging support?'.
> 
> https://review.openstack.org/#/c/326940/6/nova/tests/functional/api_sample_tests/test_hypervisors.py
> 
> 
> The only change here is listing hypervisors, and being able to page on
> those if the microversion is high enough. So the API samples don't
> change at all, they are just running against a different microversion.

Agree. If the samples are the same, I think we shouldn't have that extra
set of tests, and just test the interesting surface. I think part of the
confusion in the code is also that the subclassing to run tests with
different scenarios pattern exists a lot of places, and we use
testscenarios explicitly other places.

> 
> The same goes for the REST API unit tests really:
> 
> https://review.openstack.org/#/c/326940/6/nova/tests/unit/api/openstack/compute/test_hypervisors.py
> 
> 
> I'm not sure if the test subclassing is just done like this for new
> microversions because it's convenient or if it's because of regression
> testing - knowing that we aren't changing a bunch of other REST methods
> in the process, so the subclassed tests aren't testing anything
> different from the microversion that came before them.
> 
> The thing I don't like about the test subclassing is all of the
> redundant testing that goes on, and people might add tests to the parent
> class not realizing it's subclassed and thus duplicating test cases with
> no functional change.
> 
> Am I just having some Friday crazies? Ultimately this doesn't hurt
> anything really but thought I'd ask.

Honestly, I feel like subclassing tests is almost always an
anti-pattern. It looks like you are saving code up front, but it
massively couples things in ways that become super hard to deal with in
the future.

Test code doesn't need to be normalized to within an inch of it's life.

	-Sean

-- 
Sean Dague
http://dague.net