Dear Bauza, Mooney Following up on our discussion during the Weekly Meeting on 2028-01-25 ([1] 16:31:58~) about Arm CCA merging, I'd appreciate your confirmation on a couple of points. The relevant portion of the meeting minutes is included below:
16:35:46 <bauzas> if specific hardware can't be tested on upstream CI, then there are two possibilities : 16:36:35 <bauzas> 1/ either you're able to test nova against fake hardware that reproduces the hardware behaviour 16:36:49 <bauzas> 2/ or you have to provide third-party CI 16:38:37 <sean-k-mooney> while 2 is preferd it much more overhead btu actull tests that the feature works end to end ... 16:44:54 <bauzas> but I strongly suggest you to invest a bit of your time into providing a testing kernel framework that could mock any specific hardware
Following the above comments, I've investigated the tests that can be provided within nova. My proposal and a question are outlined below. Could you please confirm the following? 1. My proposed testing strategy for unreleased hardware We understand that nova's testing framework provides mechanisms to mock kernel functions such as libvirt(using fakelibvirt), kernel files, and so on. However, end-to-end testing, including Arm CCA-enabled hardware emulation, cannot be fully achieved solely within nova. Therefore, to submit end-to-end tests before the hardware release, we will utilize QEMU driver with Arm CCA enabled [2] (The development for CCA is in development now). E2e tests will be executed on these Arm CCA-enabled VMs. I will be able to submit e2e test codes and results, including unit/functional tests leveraging nova's existing mechanisms like fakelibvirt. Please review this approach and provide any feedback. 2. A Question about test strategy after the hardware release The merging schedule for Arm CCA feature will be determined on the next PTG. If the merge is scheduled after the hardware release, what kind of test code and test results need to be submitted?
if its released hardware that is just hard to get hold of because its only available in the latest generation and is expensive or otherwise not widely available provide there is enough unit/functional testing and the author of the code can provide log/test results showing the integration works that might be sufficient.
FUJITSU-MONAKA is likely to be the first to support Arm CCA. So, we will be adding a feature targeted at the newly released hardware. Will e2e tests using the actual Arm CCA-enabled hardware be required in addition to unit/function tests? (Are tests on a hardware-emulated environment unnecessary?) [1] https://meetings.opendev.org/meetings/nova/2025/nova.2025-01-28-16.00.log.ht... [2] https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+... Regards, Taketani