Hello Everyone, I am summarizing the discussion QA team had in PTG and action items for quick ref. Victoria Retrospective: ----------------------------- We talked about the retrospective of the last cycle, one main challenge we discussed is the slow review in QA due to fewer contributors. Below are the action items that came up from improving things: * Action: - microversion need more work also when you try more combination or latest (gmann). - Triage bugs in the office hour as 30 min -> update agenda - Speed up the review. Office hour schedule: --------------------------- With daylight ending in Nov, we decided to shift the QA office hour by an hour late which is at 14 UTC. Move back the single horizon test in tempest tree from tempest-horizon: ------------------------------------------------------------------------------------------- Tempest and Horizon team decided to bring back the single horizon test to Tempest tree and retire the tempest-horizon plugins which was too much maintenance for single tests. It is unlikely to happen but if Tempest team in future plan to remove this test then it needs to be consulted with Horizon team first. * Action: - Move test back to Tempest - gmann - Deprecate and retire tempest-horizon - amotoki - Check with the release team if any other release needed. (amotoki + gmann) - modify the 'horizon-dsvm-tempest-plugin' job to direct to tests' new location Add run_validation as a centralized point for Tempest scenario tests: ------------------------------------------------------------------------------------- For now, validation resources automation and skip ssh part via run_validation is done in API test only and this proposal is to extend it to scenario tests also. There is no reason not to do that which will help to run tempest scenario tests on images where ssh is not allowed. There is a most possible situation where complete tests need to be skipped but we will see case by case and at least skipping automatically will help tester to avoid explicitly excluding the scenario tests to the skip list. *Action: - Implement it in scenario tests - paras333 - Refactor scenario manager's create_server method to consume common/compute.py - paras333 Patrole stable release: ---------------------------- We are lacking the maintainer in Patrole project which is a big challenge to release a stable version of it. Sonia and doug (already helping) will try to spend more bandwidth in Patrole. Another thing to reduce the Patrole tests execution time. We discussed a few options like policy engine to return the API based on the flag or use fake driver etc but we did not conclude any option yet as they need more investigation. * Action: - Keep eyes on gate and how tests are running and based on that we can make a stable release for Wallaby. - gmann - Keep exploring the options for making patrole lightweight. - gmann, doug Grenade: extending the testing for zero downtime upgrade testing: ------------------------------------------------------------------------------------ This is just a thought. There is TC tag 'assert:supports-zero-downtime-upgrade' which is not used by any of the projects and also we do not have any testing framework which can verify this tag. We talked about it if we can do such testing in grenade or not. Before moving forward on discussion and investigation we checked if anyone can volunteer for this work. As of now, no volunteer. Use different guest image for gate jobs to run tempest tests: --------------------------------------------------------------------------- We only test cirros guest image to create the test VM and all proposal from Paras is to try more images to enhance the different configuration scenarios in upstream testing. This will help to able to catch more failure/scenario at the upstream gate itself compare to the current situation where most of them are reported from downstream testing. * Action: - Add a job as experimental - paras333 Tempest Cred Manager proposed changes: ----------------------------------------------------- 'primary' and 'alt_promary' credentials in Tempest are hardcoded to non-admin and they are assigned configured 'tempest_role'. There is no way we can assign a different role to both of these creds. The idea here is to make 'primary' and 'alt_promary' credentials configurable so that different deployment can configure these with different roles in various combinations. We will be adding two config option similar to 'tempest_role' and default to an empty list so that we continue the default behavior with what we have currently. So that there is no backward-incompatible change instead it is an additional new thing provided. * Action: - Doug needs to propose the patch. Tempest Scenario manager stable version brainstorming: ----------------------------------------------------------------------- We talked about a few method changes to make the scenario manager stable. If any method is used among plugins only and not in Tempest then we do not actually need to move it to Tempest and they can stay on the side of the plugin itself. We ran out of time here and will continue the brainstorming in office hours or so. (this etherpad - https://etherpad.opendev.org/p/tempest-scenario-manager) * Action: - Sonia and Martin will continue working on this. Wallaby Priority & Planning: ------------------------------------ We did not prioritize the things as such but listed all working items in below etherpad along with Victoria cycle backlogs. - https://etherpad.opendev.org/p/qa-wallaby-ptg Detailed discussions are in this Etherpad: https://etherpad.opendev.org/p/qa-wallaby-ptg -gmann