Open Stack

On 9/14/2019 12:19 AM, Ghanshyam Mann wrote:
> If you have noticed that stable/ocata gate is blocked where 'legacy-tempest-dsvm-neutron-full/-*' job
> is failing due to the latest Tempest changes.
> 
> Tempest started the JSON schema strict validation for Volume APIs which caught the failure or you can say
> Tempest master cannot be used in Ocata testing. More details-https://bugs.launchpad.net/tempest/+bug/1843762  
> 
> As per the Tempest stable branch testing policy[1], Tempst does not support the stable/ocata (which is EM )in the
> current development cycle. Current supported stable branches by Tempest are Queens, Rocky, Stein and Train-on-going.
> We can keep using Tempest master on EM stable/branches as long as it run successfully and if it start failing which is current
> case of stable/ocata then use Tempest tag to test that EM stable branch.
> 
> To unblock the stable/ocata gate, I am trying to install the Tempest 20.0.0(compatible version for Ocata) in ocata gate
> -https://review.opendev.org/#/c/681950/
> Fix is not working as of now (it still install Tempest master). I will debug that later (my current priority is for Train feature freeze).
> 
> [1]https://docs.openstack.org/tempest/latest/stable_branch_support_policy.html

Thanks for the heads up. I agree that being able to continue to run 
tempest integration jobs on stable/ocata changes, even with a frozen 
tempest version, is better than not running integration testing on 
stable/ocata at all. When I was at IBM and we were supported branches 
downstream that were end of life upstream what I'd do was create an 
internal branch for tempest (stable/ocata in this case) so we'd run 
against that rather than master tempest, just in case we needed to make 
changes and couldn't use a tag (back then tags for tempest were also 
pretty new as I recall). I'm not advocating creating a stable/ocata 
branch for tempest upstream, I'm just giving an example of one 
downstream process for this sort of thing.

Alternatively Cinder could fix the API regression, but that would likely 
be a regression of its own at this point right? Meaning if they added 
something to an API response without a microversion and then removed it 
without a microversion, that's not really helping the situation. As it 
stands clients (in this case tempest) have to deal with the API change.

Another alternative would be putting some kind of compat code in tempest 
for this particular API breakage but if Tempest isn't going to gate on 
stable/ocata then that's not really the responsibility of the QA team to 
carry that compat code.

-- 

Thanks,

Matt