[nova] Help with support to SGX
Hello o/ Anyone already work with Nova to provide VMs with SGX support?
Hi, On 9 Jan 2024, at 20:23, Winicius Allan wrote:
Anyone already work with Nova to provide VMs with SGX support?
as part of the SCS project we’re currently updating the intel patchset (https://github.com/intel/secured-cloud-management-stack/) to work with bobcat/caracal. We’re tracking the work here: https://github.com/SovereignCloudStack/issues/issues/246 The work is here: https://github.com/osism/secured-cloud-management-stack A blogpost outlining the background of our work: https://scs.community/tech/2023/11/21/confidential-computing-in-digital-sove... felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
Cheers Felix. That is quite interesting work and thanks for the links. Are you working only with Intel SGX/TDX or also looking at AMD SEV-SNP? Regards Mahendra On 09/01/2024 22:41, Felix Kronlage-Dammers wrote:
Hi,
On 9 Jan 2024, at 20:23, Winicius Allan wrote:
Anyone already work with Nova to provide VMs with SGX support? as part of the SCS project we’re currently updating the intel patchset (https://github.com/intel/secured-cloud-management-stack/) to work with bobcat/caracal. We’re tracking the work here: https://github.com/SovereignCloudStack/issues/issues/246 The work is here: https://github.com/osism/secured-cloud-management-stack
A blogpost outlining the background of our work: https://scs.community/tech/2023/11/21/confidential-computing-in-digital-sove...
felix
-- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack
Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V.
Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
Hi Mahendra! On 9 Jan 2024, at 23:03, Mahendra Paipuri wrote:
Cheers Felix. That is quite interesting work and thanks for the links. Are you working only with Intel SGX/TDX or also looking at AMD SEV-SNP?
The colleagues from OSISM (who work on the forward porting of the SGX patchset) are looking specifically at the SGX patchset. However that story is part of a larger epic[1] - that has a larger scope. As part of that we will also look at the current (existing[2]) support of SEV. My talk on this subject that I submitted to FOSDEM was sadly not accepted, but I will likely further publish it as a series of blogposts or similar. felix [1] <url: https://github.com/SovereignCloudStack/issues/issues/39> [2] <url: https://docs.openstack.org/nova/latest/admin/sev.html> -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
On Tue, 2024-01-09 at 23:22 +0100, Felix Kronlage-Dammers wrote:
Hi Mahendra!
On 9 Jan 2024, at 23:03, Mahendra Paipuri wrote:
Cheers Felix. That is quite interesting work and thanks for the links. Are you working only with Intel SGX/TDX or also looking at AMD SEV-SNP?
The colleagues from OSISM (who work on the forward porting of the SGX patchset) are looking specifically at the SGX patchset. However that story is part of a larger epic[1] - that has a larger scope. As part of that we will also look at the current (existing[2]) support of SEV.
have you considerd actuly working with the upstream community to supprot this intel has not reached out to the nova comumity to extned the SEV supprot. and the current supprot was intentially design so that it could be extend to intels multi key encypted memory features in the future. https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... the out of tree patches you are maintineing in https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... are going to break release to release. While this may be useful for operator that role there own openstack packages and backport or patch the upstream soruce, in its currnt form the work that is been done with out upstream engagement is never going to make it into a vendor distrobution. if there is interest in enabling SGX i would suggest bringing it up at the next virtual PTG and propsoing it for next cycle. the spec freeze deadline for caracal is tomrrow so we wont have time to review it this cycle. i have only skimed the nova patch but one thing that did jump out at me that would have to change is https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... we do not allwo raw qemu commands in nova upstream and in general they are not stabel across qemu release and useing it taints the libvirt domain which generally renders the vm unsupproted on commeiral distros https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... woudl have to be updated to use something like <memory model='sgx-epc'> <source> <nodemask>0-1</nodemask> </source> <target> <size unit='KiB'>16384</size> <node>0</node> </target> </memory> <memory model='sgx-epc'> <target> <size unit='KiB'>16384</size> </target> </memory> it looks like libvirt gained SGX supprot around 7.9.0 based on https://libvirt.org/formatdomain.html#memory-devices
My talk on this subject that I submitted to FOSDEM was sadly not accepted, but I will likely further publish it as a series of blogposts or similar.
felix
[1] <url: https://github.com/SovereignCloudStack/issues/issues/39> [2] <url: https://docs.openstack.org/nova/latest/admin/sev.html> -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack
Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V.
Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
Hi, thanks for picking this up - am really happy about that, since it will help us to make sure to move this into the proper direction. On 10 Jan 2024, at 14:06, smooney@redhat.com wrote:
The colleagues from OSISM (who work on the forward porting of the SGX patchset) are looking specifically at the SGX patchset. However that story is part of a larger epic[1] - that has a larger scope. As part of that we will also look at the current (existing[2]) support of SEV. have you considerd actuly working with the upstream community to supprot this
yes and to my knowledge the plan is to first update the out of tree patchset so that these work with current openstack and then to properly upstream them. The idea is not to maintain an out-of-tree patchset but instead making sure to get this into upstream.
intel has not reached out to the nova comumity to extned the SEV supprot. and the current supprot was intentially design so that it could be extend to intels multi key encypted memory features in the future. https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0...
thanks for the pointers!
if there is interest in enabling SGX i would suggest bringing it up at the next virtual PTG and propsoing it for next cycle. the spec freeze deadline for caracal is tomrrow so we wont have time to review it this cycle.
very good point, I’ll make sure we do this.
i have only skimed the nova patch but one thing that did jump out at me that would have to change is
https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... we do not allwo raw qemu commands in nova upstream and in general they are not stabel across qemu release
ok. I’ll point the colleagues towards that. felix -- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V. Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
Hi, I'm very interested in this discussion because we are now working in a confidential computing project in my company and have worked on the PoC implementations. We recently managed to launch VMs with SEV-SNP (3rd generation of AMD SEV) enabled on OpenStack using some components(kernel, qemu, ovmf, libvirt and nova) patched. # Some of the bug I recently reported[1] are found in that work. We are looking into SEV-SNP because it provides more integurity protection feature including the way to verify launch digests. At this moment SEV-SNP support is still actively developed and we first need the existing works done by AMD in kernel, qemu and ovmf are merged[2] before we finalize the implementation for SEV-SNP in upper layers (libvirt and OpenStack) but we now have a better view about SEV-ES (2nd generation of AMD SEV) which is already supported by kernel and libvirt and already found a few points we have to discuss to extend the existing memory encryption support in Nova. If you'll discuss the way to extend the memory encryption support for SGX then I'd like to be involved in the discussion and am happy to bring some points we believe are needed for future work to support newer generations of AMD SEV and hopefully start some work to add support of SEV-ES. Thank you, Takashi Kajinami [1] https://bugs.launchpad.net/nova/+bug/2047399 https://bugs.launchpad.net/nova/+bug/2041511 https://bugs.launchpad.net/nova/+bug/2040449 [2] https://github.com/AMDESE On 1/10/24 22:41, Felix Kronlage-Dammers wrote:
Hi,
thanks for picking this up - am really happy about that, since it will help us to make sure to move this into the proper direction.
On 10 Jan 2024, at 14:06, smooney@redhat.com wrote:
The colleagues from OSISM (who work on the forward porting of the SGX patchset) are looking specifically at the SGX patchset. However that story is part of a larger epic[1] - that has a larger scope. As part of that we will also look at the current (existing[2]) support of SEV. have you considerd actuly working with the upstream community to supprot this yes and to my knowledge the plan is to first update the out of tree patchset so that these work with current openstack and then to properly upstream them. The idea is not to maintain an out-of-tree patchset but instead making sure to get this into upstream.
intel has not reached out to the nova comumity to extned the SEV supprot. and the current supprot was intentially design so that it could be extend to intels multi key encypted memory features in the future. https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... thanks for the pointers!
if there is interest in enabling SGX i would suggest bringing it up at the next virtual PTG and propsoing it for next cycle. the spec freeze deadline for caracal is tomrrow so we wont have time to review it this cycle. very good point, I’ll make sure we do this.
i have only skimed the nova patch but one thing that did jump out at me that would have to change is
https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... we do not allwo raw qemu commands in nova upstream and in general they are not stabel across qemu release ok. I’ll point the colleagues towards that.
felix
-- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack
Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V.
Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
just fyi I've created a spec describing my current plan for SEV-ES support[1]. [1] https://review.opendev.org/c/openstack/nova-specs/+/907702 The spec is currently pushed to the 2024.1 directory but my current target is 2024.2 . because I know spec freeze for 2024.1 already passed a few weeks ago. I'll resubmit it to 2024.2 once spec proposal is open and the 2024.2 directory is created. The spec review is currently meant for early reference and feedback. On 1/12/24 14:48, Takashi Kajinami wrote:
Hi,
I'm very interested in this discussion because we are now working in a confidential computing project in my company and have worked on the PoC implementations. We recently managed to launch VMs with SEV-SNP (3rd generation of AMD SEV) enabled on OpenStack using some components(kernel, qemu, ovmf, libvirt and nova) patched. # Some of the bug I recently reported[1] are found in that work.
We are looking into SEV-SNP because it provides more integurity protection feature including the way to verify launch digests.
At this moment SEV-SNP support is still actively developed and we first need the existing works done by AMD in kernel, qemu and ovmf are merged[2] before we finalize the implementation for SEV-SNP in upper layers (libvirt and OpenStack) but we now have a better view about SEV-ES (2nd generation of AMD SEV) which is already supported by kernel and libvirt and already found a few points we have to discuss to extend the existing memory encryption support in Nova.
If you'll discuss the way to extend the memory encryption support for SGX then I'd like to be involved in the discussion and am happy to bring some points we believe are needed for future work to support newer generations of AMD SEV and hopefully start some work to add support of SEV-ES.
Thank you, Takashi Kajinami
[1] https://bugs.launchpad.net/nova/+bug/2047399 https://bugs.launchpad.net/nova/+bug/2041511 https://bugs.launchpad.net/nova/+bug/2040449
On 1/10/24 22:41, Felix Kronlage-Dammers wrote:
Hi,
thanks for picking this up - am really happy about that, since it will help us to make sure to move this into the proper direction.
On 10 Jan 2024, at 14:06, smooney@redhat.com wrote:
The colleagues from OSISM (who work on the forward porting of the SGX patchset) are looking specifically at the SGX patchset. However that story is part of a larger epic[1] - that has a larger scope. As part of that we will also look at the current (existing[2]) support of SEV. have you considerd actuly working with the upstream community to supprot this yes and to my knowledge the plan is to first update the out of tree patchset so that these work with current openstack and then to properly upstream them. The idea is not to maintain an out-of-tree patchset but instead making sure to get this into upstream.
intel has not reached out to the nova comumity to extned the SEV supprot. and the current supprot was intentially design so that it could be extend to intels multi key encypted memory features in the future. https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... https://github.com/openstack/nova-specs/blob/c6b6eab6304203f6fca32dd3ce074b0... thanks for the pointers!
if there is interest in enabling SGX i would suggest bringing it up at the next virtual PTG and propsoing it for next cycle. the spec freeze deadline for caracal is tomrrow so we wont have time to review it this cycle. very good point, I’ll make sure we do this.
i have only skimed the nova patch but one thing that did jump out at me that would have to change is
https://github.com/intel/secured-cloud-management-stack/blob/main/nova-intel... we do not allwo raw qemu commands in nova upstream and in general they are not stabel across qemu release ok. I’ll point the colleagues towards that.
felix
-- Felix Kronlage-Dammers Product Owner IaaS & Operations Sovereign Cloud Stack
Sovereign Cloud Stack — standardized, built and operated by many Ein Projekt der Open Source Business Alliance - Bundesverband für digitale Souveränität e.V.
Tel.: +49-30-206539-205 | Matrix: @fkronlage:matrix.org | fkr@osb-alliance.com
participants (6)
-
Felix Kronlage-Dammers
-
Felix Kronlage-Dammers
-
Mahendra Paipuri
-
smooney＠redhat.com
-
Takashi Kajinami
-
Winicius Allan