---- On Wed, 15 Aug 2018 06:40:57 +0900 Eric K <ekcs.openstack at gmail.com> wrote ---- > Anyone have an example handy of a tempest test conditioning on service > release version (because new features not available in past versions)? > Seems like it could get pretty messy and haphazard, so I'm curious to > see best practices. Thanks lots! Thanks Eric for query. We do it in many times in Tempest and similar approach can be adopt by tempest plugins. There are 2 ways we can handle this- 1. Using feature flag. Tempest documentation is here [1]. Step1- This is simply adding a config options(feature flag) for new/old feature. Example- https://review.openstack.org/#/c/545627/ https://github.com/openstack/tempest/blob/6a8d495192632fd18dce4baf1a4b213f401a0167/tempest/config.py#L242 Step2- Based on that flag you can skip the tests where that feature is not available. Example- https://github.com/openstack/tempest/blob/d5058a8a9c8c1c5383699d04296087b6d5a24efd/tempest/api/identity/base.py#L315 Step3- For gate, devstack plugin on project side (congress is your case [2]) which is branch aware can set that flag to true and false based on which branch that test is running. For tempest we do the same from devstack/lib/tempest Example - https://review.openstack.org/#/c/545680/ https://github.com/openstack-dev/devstack/blob/8c1052001629d62f001d04c182500fa293858f47/lib/tempest#L308 Step4- For cloud testing(non-gate), tester can manually configure the those flag based on what service version they are testing. 2. Detecting service version via version API - If you can get the service version info from API then you can use that while skipping the tests. - One example if for compute where based on microversion, it can be detected that test running against which release. - Example- https://github.com/openstack/tempest/blob/d5058a8a9c8c1c5383699d04296087b6d5a24efd/tempest/api/compute/base.py#L114 [1] https://docs.openstack.org/tempest/latest/HACKING.html#branchless-tempest-considerations [2] https://github.com/openstack/congress/blob/014361c809517661264d0364eaf1e261e449ea80/devstack/plugin.sh#L88 > > Eric Kao > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >