Open Stack

On 7/28/2016 11:20 PM, Ghanshyam Mann wrote:
> Yes, I also prefer the approach of capping the tests instead of jobs. But along with that we might need to make sure the same tests coverage Tempest provides if min_microversion is set >2.35 in config.
> For example, if we cap the tests (calls nova-network) with max_microversion = 2.35 then, we might need to implement/modify those tests to start using neutron which can be run if config's min_microversion
> is set > 2.35.
> There are two type of test cases-
>     1. Test only tests nova-network APIs - Example: https://github.com/openstack/tempest/tree/master/tempest/api/compute/floating_ips
>     2. Test testing other scenario using nova-network - Example: https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_rescue.py
>
> 1st case if all ok to cap and leave it skip for >2.35. But for 2nd case, I feel we should not leave them skip if config's min_microversion > 2.35 which mean leaving those scenario untested(if min_microversion >2.35).
> There are 2 options for 2nd case:
>   1. Implement  duplicate tests by using the neutron APIs - This will be duplicate tests but needed because of testing nova-network till newton EOL.
>   2. Or modify those to switch from nova-network to neutron.  - If we do not care about nova-network testing even for stable branches where it is not deprecated.

I don't like either option, but I'd rather go with #1 for two reasons:

a) Once nova-network is gone these tests wouldn't run ever, so we lose 
the coverage.

b) nova-network wasn't deprecated in stable branches so I think we need 
to maintain those tests, so 2.1-2.35 are nova-network, 2.36+ are neutron.

The duplication cost probably wouldn't be all that terrible, we could 
probably abstract a lot of the common network setup parts away for the 
compute API tests for things like creating a floating IP.

>
>
> Thanks
> gmann
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


-- 

Thanks,

Matt Riedemann