---- On Sun, 15 Sep 2019 02:01:56 +0900 Matt Riedemann mriedemos@gmail.com wrote ----
On 9/14/2019 12:19 AM, Ghanshyam Mann wrote:
If you have noticed that stable/ocata gate is blocked where 'legacy-tempest-dsvm-neutron-full/-*' job is failing due to the latest Tempest changes.
Tempest started the JSON schema strict validation for Volume APIs which caught the failure or you can say Tempest master cannot be used in Ocata testing. More details-https://bugs.launchpad.net/tempest/+bug/1843762
As per the Tempest stable branch testing policy[1], Tempst does not support the stable/ocata (which is EM )in the current development cycle. Current supported stable branches by Tempest are Queens, Rocky, Stein and Train-on-going. We can keep using Tempest master on EM stable/branches as long as it run successfully and if it start failing which is current case of stable/ocata then use Tempest tag to test that EM stable branch.
To unblock the stable/ocata gate, I am trying to install the Tempest 20.0.0(compatible version for Ocata) in ocata gate -https://review.opendev.org/#/c/681950/ Fix is not working as of now (it still install Tempest master). I will debug that later (my current priority is for Train feature freeze).
[1]https://docs.openstack.org/tempest/latest/stable_branch_support_policy.html
Thanks for the heads up. I agree that being able to continue to run tempest integration jobs on stable/ocata changes, even with a frozen tempest version, is better than not running integration testing on stable/ocata at all. When I was at IBM and we were supported branches downstream that were end of life upstream what I'd do was create an internal branch for tempest (stable/ocata in this case) so we'd run against that rather than master tempest, just in case we needed to make changes and couldn't use a tag (back then tags for tempest were also pretty new as I recall). I'm not advocating creating a stable/ocata branch for tempest upstream, I'm just giving an example of one downstream process for this sort of thing.
Thanks for that information. I think creating stable/ocata in Tempest will face the maintenance issue. Let's try with tag first if that work fine.
Alternatively Cinder could fix the API regression, but that would likely be a regression of its own at this point right? Meaning if they added something to an API response without a microversion and then removed it without a microversion, that's not really helping the situation. As it stands clients (in this case tempest) have to deal with the API change.
I am on same page with you on this but there are different opinion on how to change API with microversion. I have started a separate thread on this to discuss the correct way to change API - http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009365...
-gmann
Another alternative would be putting some kind of compat code in tempest for this particular API breakage but if Tempest isn't going to gate on stable/ocata then that's not really the responsibility of the QA team to carry that compat code.
Yeah, as per Extended Maintainance stable branch testing policy, Tempest would not be able to maintain those code. It becomes difficult from maintenance as well as strict verification side also.
-gmann
--
Thanks,
Matt