Open Stack

> -----Original Message-----
> From: Matt Riedemann [mailto:mriedem at linux.vnet.ibm.com]
> Sent: 18 June 2016 05:16
> To: OpenStack Development Mailing List (not for usage questions)
> <openstack-dev at lists.openstack.org>
> Subject: [openstack-dev] [nova] Question about redundant API samples
> tests for microversions
> 
> I was reviewing this today:
> 
> https://review.openstack.org/#/c/326940/
> 
> And I said to myself, 'self, do we really need to subclass the API samples
> functional tests for this microversion given this change doesn't modify the
> request/response body, it's only adding paging support?'.
> 
> https://review.openstack.org/#/c/326940/6/nova/tests/functional/api_sam
> ple_tests/test_hypervisors.py
> 
> The only change here is listing hypervisors, and being able to page on those if
> the microversion is high enough. So the API samples don't change at all, they
> are just running against a different microversion.
> 
> The same goes for the REST API unit tests really:
> 
> https://review.openstack.org/#/c/326940/6/nova/tests/unit/api/openstack/
> compute/test_hypervisors.py
> 
> I'm not sure if the test subclassing is just done like this for new microversions
> because it's convenient or if it's because of regression testing - knowing that
> we aren't changing a bunch of other REST methods in the process, so the
> subclassed tests aren't testing anything different from the microversion that
> came before them.
> 
> The thing I don't like about the test subclassing is all of the redundant testing
> that goes on, and people might add tests to the parent class not realizing it's
> subclassed and thus duplicating test cases with no functional change.
> 
> Am I just having some Friday crazies? Ultimately this doesn't hurt anything
> really but thought I'd ask.

Yes, we should not run redundant test case till we are trying to test the regression in previous microversion or really testing the changes does not reflect in previous version or something.
In this case, we should not run the normal hypervisor tests(present in HypervisorsSampleJsonTests) for this new microversion tests (as those does not give real benefit), if we want to tests the pagination in functional tests then it should be separate specific tests. But if we want to tests paging in functional tests, I wonder we end up with having duplicate sample template and sample files. 
Otherwise I too like to tests those bits in unit tests and then in Tempest (volume pagination tests are well implemented [1]).

Tempest has taken care the redundant tests run by mentioning the max_microversion in existing parent test class [2] which make sure tests gets executed once in case of subclass too. Or other way is not to subclass microversion tests class and let is implement and run its own specific tests [3]. Those can be implemented case by case. 

1 .. https://github.com/openstack/tempest/blob/master/tempest/api/volume/v2/test_volumes_list.py#L184-L193 

2 .. https://github.com/openstack/tempest/blob/master/tempest/api/compute/keypairs/test_keypairs.py#L22 

3 .. https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_keypairs_v210.py 

> 
> --
> 
> Thanks,
> 
> Matt Riedemann
> 
> 
> __________________________________________________________
> ________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Thanks
Ghanshyam Mann