On Tue, Aug 14, 2018 at 8:59 PM, Ghanshyam Mann <gmann at ghanshyammann.com> wrote: > ---- On Wed, 15 Aug 2018 06:40:57 +0900 Eric K <ekcs.openstack at gmail.com> wrote ---- > > Anyone have an example handy of a tempest test conditioning on service > > release version (because new features not available in past versions)? > > Seems like it could get pretty messy and haphazard, so I'm curious to > > see best practices. Thanks lots! > > Thanks Eric for query. We do it in many times in Tempest and similar approach can be adopt by tempest plugins. There are 2 ways we can handle this- > > 1. Using feature flag. Tempest documentation is here [1]. > Step1- This is simply adding a config options(feature flag) for new/old feature. > Example- https://review.openstack.org/#/c/545627/ https://github.com/openstack/tempest/blob/6a8d495192632fd18dce4baf1a4b213f401a0167/tempest/config.py#L242 > Step2- Based on that flag you can skip the tests where that feature is not available. > Example- https://github.com/openstack/tempest/blob/d5058a8a9c8c1c5383699d04296087b6d5a24efd/tempest/api/identity/base.py#L315 > Step3- For gate, devstack plugin on project side (congress is your case [2]) which is branch aware can set that flag to true and false based on which branch that test is running. For tempest we do the same from devstack/lib/tempest > Example - https://review.openstack.org/#/c/545680/ https://github.com/openstack-dev/devstack/blob/8c1052001629d62f001d04c182500fa293858f47/lib/tempest#L308 > Step4- For cloud testing(non-gate), tester can manually configure the those flag based on what service version they are testing. > > 2. Detecting service version via version API > - If you can get the service version info from API then you can use that while skipping the tests. > - One example if for compute where based on microversion, it can be detected that test running against which release. > - Example- https://github.com/openstack/tempest/blob/d5058a8a9c8c1c5383699d04296087b6d5a24efd/tempest/api/compute/base.py#L114 > > > [1] https://docs.openstack.org/tempest/latest/HACKING.html#branchless-tempest-considerations > [2] https://github.com/openstack/congress/blob/014361c809517661264d0364eaf1e261e449ea80/devstack/plugin.sh#L88 > > > > > Eric Kao Thank you so much, Ghanshyam!