[nova] How to gain approval for a new feature on unavailable hardware
To Whom It May Concern, We are considering to propose and develop a new feature on OpenStack Nova for building Arm Confidential Computing Architecture (Arm CCA)-encrypted instances. I have some questions for doing it. * While specification of Arm CCA is already open(https://developer.arm.com/documentation/den0125/0300/Overview), No Arm CCA-enabled hardware is currently available to our knowledge. Therefore, we plan to conduct testing in an internal environment. * Would a proposal for a feature utilizing publicly unavailable hardware be acceptable to OpenStack? Alternatively, in this case, what steps or preparations are required for approval? * Should the blueprint and specification be submitted simultaneously? * The specification is currently incomplete. Building Arm CCA instances requires libvirt APIs, similar to AMD SEV instances(https://blueprints.launchpad.net/nova/+spec/amd-sev-libvirt-support), but specification of these are not yet determined publicly. We are considering submitting the blueprint first to inform the community of our ongoing development efforts. Regards
On 11/25/24 3:18 PM, taketani.ryo@fujitsu.com wrote:
To Whom It May Concern,
We are considering to propose and develop a new feature on OpenStack Nova for building Arm Confidential Computing Architecture (Arm CCA)-encrypted instances. I have some questions for doing it.
* While specification of Arm CCA is already open(https://developer.arm.com/documentation/den0125/0300/Overview), No Arm CCA-enabled hardware is currently available to our knowledge. Therefore, we plan to conduct testing in an internal environment. * Would a proposal for a feature utilizing publicly unavailable hardware be acceptable to OpenStack? Alternatively, in this case, what steps or preparations are required for approval?
I'd like to ask some opinions from nova team (especially nova cores) but I think having an internal and local testing environment is enough for the beginning. Ideally we may want 3rd party CI which runs tests for every single change to detect regressions but that may not block such features which has really specific hardware requirements. In spec there are some sections to describe the testing plan so once you propose the spec we can discuss details based on the plan described in that section. # For example for SEV-ES support work we agreed with no full functional test coverage in CI, # but only functional tests with fake driver + testing in local environment https://specs.openstack.org/openstack/nova-specs/specs/2025.1/approved/amd-s...
* Should the blueprint and specification be submitted simultaneously? * The specification is currently incomplete. Building Arm CCA instances requires libvirt APIs, similar to AMD SEV instances(https://blueprints.launchpad.net/nova/+spec/amd-sev-libvirt-support), but specification of these are not yet determined publicly. We are considering submitting the blueprint first to inform the community of our ongoing development efforts.
Technically we can start discussions in OpenStack with reference to draft libvirt API, if we have details such as xml format. However approval may be on-hold until support for Arm CCA is completed in libvirt. This is because we should avoid problems caused by the change made in libvirt AFTER design is made in nova. Also nova implements the mechanism to check minimum libvirt/qemu version to use requested features and we may need to know the exact libvirt/qemu version to support CCA to complete the implementation in nova.
Regards
In spec there are some sections to describe the testing plan so once you propose
Technically we can start discussions in OpenStack with reference to draft libvirt API, if we have details such as xml format. However approval may be on-hold until support for Arm CCA is completed in libvirt. This is because we should avoid problems caused by the change made in libvirt AFTER design is made in nova. Also nova implements
Thank you for your response. the spec we can discuss details based on the plan described in that section. I will add the testing methodologies to the "Testing" section (and other relevant sections) of the spec before submitting it. We'll start by implementing an internal and local testing environment. the mechanism to check minimum libvirt/qemu version to use requested features and we may need to know the exact libvirt/qemu version to support CCA to complete the implementation in nova. I understand that patch approval may be on-hold until support for Arm CCA is completes in libvirt. To facilitate discussion of implementation strategies, I will submit the specifications shortly. I will add a note in blueprint and spec that underlying components like libvirt are under development, and their specifications are subject to change . Best regards,
I submitted a spec for Arm CCA instances. In developing my spec, I drew upon aspects of the AMD-SEV ES spec. Thank you very much. https://review.opendev.org/c/openstack/nova-specs/+/938276
in the past we have ask for hardware vendor to provided third party ci and commit to helping to maintain the integration going forward. we don't always require that as it can be a large ask of a new contributor. officially nova required any hardware feature to be supported in a public official release of all software dependencies. That normally means supported by libvirt/qemu but also a released upstream kernel if driver/kernel enablement is needed. We do not allow merging support of functionality for unreleased hardware. you can propose the design and even start working on the implementation but we would hold merging the implementationuntil all the dependencies are met and officially publicly released. the answer also partly comes down to exactly what unavailable hardware means. As noted above if its unavailable because it does not exist yet in the market as a product you can buy then its to early merge the support in nova. We only hardware support for released hardware. if its released hardware that is just hard to get hold of because its only available in the latest generation and is expensive or otherwise not widely available provide there is enough unit/functional testing and the author of the code can provide log/test results showing the integration works that might be sufficient. i stress might since that would need wider discussion in the spec. On 27/12/2024 07:20, taketani.ryo@fujitsu.com wrote:
I submitted a spec for Arm CCA instances. In developing my spec, I drew upon aspects of the AMD-SEV ES spec. Thank you very much.
im not actully back to work until monday but i did see that spec while i was off. i only skimmed it quickly but the general direction seam ok, i.e. reusign the memory_encryption_context resouce class with a new triat for the ARM CCA feature. the main question the will need to be answer is how can we test this this end to end? for something like this unit test would not be sufficient. function tests, which are like unit test but test the interaction betwen multiple nova agents with test fixture to fake out real hardware, may be sufficient. what are the minium qemu/libvirt/linux verisions? how does this impact lifecycle operation like live migration? how does this interact with other features like cinder volumes? virtio-scsi? for SEV we had to add special handeling for virtio devices like the console or virtio disks to make it funciton. does it have any other requirements to function? iommu? firmware=uefi? a specific machine type i.e. virt? realistically the spec approval deadline for this cycle is thrusday the 9th. given that its likly too late to include in 2025.1 (epoxy) but it may make sense to start working on it for 2025.2 regards sean.
Sean Mooney wrote:
the answer also partly comes down to exactly what unavailable hardware means.
As noted above if its unavailable because it does not exist yet in the market as a product you can buy then its to early merge the support in nova. We only hardware support for released hardware.
To our knowledge, no products currently support Arm CCA. FUJITSU-MONAKA [1], scheduled for release by Fujitsu in 2027, may be the first such product. Our goal is to have OpenStack ready for use upon the release of the first Arm CCA-supporting product. Sean Mooney wrote:
officially nova required any hardware feature to be supported in a public official release of all software dependencies.
That normally means supported by libvirt/qemu but also a released upstream kernel if driver/kernel enablement is needed.
We do not allow merging support of functionality for unreleased hardware.
We understand nova's policies. However, we would like to discuss and confirm alternatives. While no CPUs with this feature are currently on the market, It's possible to emulate the Arm hardware, including Arm CCA functionality, using a emulator like QEMU [2]. Even if Arm CCA is available in prerequisite software like qemu/libvirt/linux, and we can develop and test Arm CCA feature for OpenStack on an emulator with fixed prerequisite software stack, is there still no possibility of merging before the release of the actual hardware? We would appreciate the opportunity to share our goals and discuss a potential schedule for merge with you in a suitable meeting such as a nova weekly meeting. Sean Mooney wrote:
the main question the will need to be answer is how can we test this this end to end? for something like this unit test would not be sufficient. function tests, which are like unit test but test the interaction betwen multiple nova agents with test fixture to fake out real hardware, may be sufficient.
We acknowledge that the tests currently specified in the spec are insufficient. We will consider adding tests that simulate real hardware. Sean Mooney wrote:
what are the minium qemu/libvirt/linux verisions?
how does this impact lifecycle operation like live migration?
These are still undetermined and will be updated later. libvirt RFC for Arm CCA has just been submitted. [3] Limitations on certain lifecycle operations will apply to Arm CCA VMs. Sean Mooney wrote:
how does this interact with other features like cinder volumes? virtio-scsi? for SEV we had to add special handeling for virtio devices like the console or virtio disks to make it funciton.
does it have any other requirements to function? iommu? firmware=uefi? a specific machine type i.e. virt?
Thank you for your comments. We will verify these points and add them if necessary. Sean Mooney wrote:
realistically the spec approval deadline for this cycle is thrusday the 9th.
given that its likly too late to include in 2025.1 (epoxy) but it may make sense to start working on it for 2025.2
We submitted this to foster community discussion. However, as several key issues remain to be addressed, we must resubmit a spec for the next release cycle. # I apologize if submitting it to the 2025.1 directory was inappropriate. Regards, [1] https://www.fujitsu.com/downloads/SUPER/topics/isc24/next-arm-based-processo... [2] https://linaro.atlassian.net/wiki/spaces/QEMU/pages/29051027459/Building+an+... [3] https://lists.libvirt.org/archives/list/devel@lists.libvirt.org/thread/Q2AZF... Taketani
participants (3)
-
Sean Mooney
-
Takashi Kajinami
-
taketani.ryo@fujitsu.com