[openstack-dev] StarlingX gap analysis to converge with OpenStack master
Hi Stackers, One of the key goals of StarlingX during the current cycle is to converge with the OpenStack projects master branches. In order to accomplish this goal, the Technical Steering Committee put together a gap analysis that shows the specs and patches that need to merge in the different OpenStack projects by the end of Stein. The attached PDF document shows this analysis. Although other projects are involved, most of the work has to be done in Nova, Neutron and Horizon. Hopefully all the involved projects will help StarlingX achieve this important goal. It has to be noted that work is still on-going to refine this gap analysis, so there might be some updates to it in the near future. Best regards Miguel
Thanks for doing this in the open and working with the upstream teams to reduce divergence! On Wed, Nov 21, 2018 at 1:17 PM Miguel Lavalle <miguel@mlavalle.com> wrote:
Hi Stackers,
One of the key goals of StarlingX during the current cycle is to converge with the OpenStack projects master branches. In order to accomplish this goal, the Technical Steering Committee put together a gap analysis that shows the specs and patches that need to merge in the different OpenStack projects by the end of Stein. The attached PDF document shows this analysis. Although other projects are involved, most of the work has to be done in Nova, Neutron and Horizon. Hopefully all the involved projects will help StarlingX achieve this important goal.
It has to be noted that work is still on-going to refine this gap analysis, so there might be some updates to it in the near future.
Best regards
Miguel __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: OpenStack-dev-request@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
-- Mohammed Naser — vexxhost ----------------------------------------------------- D. 514-316-8872 D. 800-910-1726 ext. 200 E. mnaser@vexxhost.com W. http://vexxhost.com
On Wed, 21 Nov 2018 12:11:50 -0600, Miguel Lavalle wrote:
One of the key goals of StarlingX during the current cycle is to converge with the OpenStack projects master branches. In order to accomplish this goal, the Technical Steering Committee put together a gap analysis that shows the specs and patches that need to merge in the different OpenStack projects by the end of Stein. The attached PDF document shows this analysis. Although other projects are involved, most of the work has to be done in Nova, Neutron and Horizon. Hopefully all the involved projects will help StarlingX achieve this important goal.
It has to be noted that work is still on-going to refine this gap analysis, so there might be some updates to it in the near future.
Thanks for sending this out. I'm going to reply about what I know of the status of nova-related planned upstream features. On NUMA-aware live migration, it was identified as the top priority issue in the NFV/HPC pain points forum session [1]. The spec has been approved before in the past for the Rocky cycle, so it's a matter of re-proposing it for re-approval in Stein. We need to confirm with artom and/or stephenfin whether one of them can pick it up this cycle. I don't know as much about the shared/dedicated vCPUs on a single host or the shared vCPU extension, but the cited spec [2] has one +2 already. If we can find a second approver, we can work on this too in Stein. The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen. The HPET support spec was merged about two weeks ago and the implementation patch is under active review in a runway with one +2 now. For vCPU model, I'm not aware of any new proposed spec for Stein from the STX community as of today. Let us know if/when the spec is proposed. For disk performance fixes, the specless blueprint patch is currently under active review in a runway. The extra spec validation spec [3] is under active review now. For the bits that will be addressed using upstream features that are already available, I assume the STX community will take care of this. Please reach out to us in #openstack-nova or on the ML if there are questions/issues. For the bugs, again feel free to reach out to us for reviews/help. Cheers, -melanie [1] https://etherpad.openstack.org/p/BER-nfv-hpc-pain-points [2] https://review.openstack.org/555081 [3] https://review.openstack.org/618542
On 2018-11-21 21:23:51 +0100 (+0100), melanie witt wrote: [...]
The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen. [...]
Thanks, I had somehow missed noticing this spec up to now. I'm curious to follow the implementation and find out how we go about making sure users don't get the impression that an emulated TPM provides anywhere near the same sorts of guarantees as real one. -- Jeremy Stanley
On Wed, 21 Nov 2018 20:34:09 +0000, Jeremy Stanley wrote:
On 2018-11-21 21:23:51 +0100 (+0100), melanie witt wrote: [...]
The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen. [...]
Thanks, I had somehow missed noticing this spec up to now. I'm curious to follow the implementation and find out how we go about making sure users don't get the impression that an emulated TPM provides anywhere near the same sorts of guarantees as real one.
Here's a link to the spec: https://review.openstack.org/571111 and you can follow the implementation (once it's proposed) via the gerrit topic. Cheers, -melanie
sorry to top post but wanted to provide some feedback on the pdf. on future revisions of the pdf the live migration with pinning and numa topology item shoudl be updated to the stein version of the spec. https://review.openstack.org/#/c/599587 the main reson for me to respond however is the "SR-IOV/PT best effort scheduling policy" item. this feature was not completed in queens and is possibly broken. as part of the implementation the flavor extra specs and image metadata items that were part of the spec were removed meaning that neutron sriov port and numa affinity policies for hardware offloaded ovs where lost. the spec was retro activly updated to reflect what lanned in https://review.openstack.org/#/c/555000 the first spec to introduce the concpet of pci numa affintiy was https://specs.openstack.org/openstack/nova-specs/specs/juno/approved/input-o... which clearly calls out that this was inteded for NFV workloads and makes reference to the 2 specs intoducing neutron contoled sriov interface to nova and guest numa topologies for libvirt. at the time my team at intel had our own implemtations of hugepages,vhost user and we hand a complete working implmentation of pci numa affinity with 3 policys (strict affinity, best effort affinity and any device) when the spec was repropsed for kilo the neutron aspect was left out based on feedback that it added noise since neutron controled sriov was just anoter way of useing pci passthough. https://specs.openstack.org/openstack/nova-specs/specs/kilo/implemented/inpu... in queens after an interlude where work on this got out sourced and abandoned twice we finally got around to proposing the policies that where in the original juno code https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/sh... if you look at the current version of that spec you will notice that flavor extras specs and image metadata values are missing but it was updated by https://review.openstack.org/#/c/555000 however it still contians the section decibing what happens if the flavor and image metadata dissagree "If both image and flavor properties are not set (equals None) the legacy policy will be used. If one of image or flavor property is not set (equals None) but the other property is set then the value of the set property will be used. In a case of conflicts between flavor and image properties (both properties are set and they are not equal) an exception will be raised." there have been report at the vacovour summit and more recently on mailing list http://lists.openstack.org/pipermail/openstack-dev/2018-November/136461.html that this fuctionality is broken even for the restrited case of using alias. there has also been a direct request/bug opened to track the neutron support https://bugs.launchpad.net/nova/+bug/1795920. so if we want to close this gap we have to first do an audit of the code an figure out what works/dosent and reprose the queens spec. ill be trying test this over the next two days but i wanted to highlight this a risk. On Wed, 2018-11-21 at 21:42 +0100, melanie witt wrote:
On Wed, 21 Nov 2018 20:34:09 +0000, Jeremy Stanley wrote:
On 2018-11-21 21:23:51 +0100 (+0100), melanie witt wrote: [...]
The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen.
[...]
Thanks, I had somehow missed noticing this spec up to now. I'm curious to follow the implementation and find out how we go about making sure users don't get the impression that an emulated TPM provides anywhere near the same sorts of guarantees as real one.
Here's a link to the spec:
https://review.openstack.org/571111
and you can follow the implementation (once it's proposed) via the gerrit topic.
Cheers, -melanie
On Wed, 2018-11-21 at 20:34 +0000, Jeremy Stanley wrote:
On 2018-11-21 21:23:51 +0100 (+0100), melanie witt wrote: [...]
The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen.
[...]
Thanks, I had somehow missed noticing this spec up to now. I'm curious to follow the implementation and find out how we go about making sure users don't get the impression that an emulated TPM provides anywhere near the same sorts of guarantees as real one. while its out of scope of the proposed spec it does methion the posiblity of passing through a host tpm at some point if there was a demand for that.
on a side note who was aware that out of tree hyperv dirver already support vtpm http://git.openstack.org/cgit/openstack/compute-hyperv/tree/compute_hyperv/n... there was a spec for the hyperv support back to liberty and mitaka. and was completed in tech preview as part of https://blueprints.launchpad.net/nova/+spec/hyper-v-vtpm-devices The current specs referenced in the starlinX gaps is for libvirt specifically but it would be good to see vtpm feature in other driver in the future. i dont know if the hyperv folk still want to get there vTPM feature into the intree driver but it woudl be interesting to see if this will show up in the powervm or vmware drivers at some point.
On Wed, 21 Nov 2018 21:23:51 +0100, Melanie Witt wrote:
On Wed, 21 Nov 2018 12:11:50 -0600, Miguel Lavalle wrote:
One of the key goals of StarlingX during the current cycle is to converge with the OpenStack projects master branches. In order to accomplish this goal, the Technical Steering Committee put together a gap analysis that shows the specs and patches that need to merge in the different OpenStack projects by the end of Stein. The attached PDF document shows this analysis. Although other projects are involved, most of the work has to be done in Nova, Neutron and Horizon. Hopefully all the involved projects will help StarlingX achieve this important goal.
It has to be noted that work is still on-going to refine this gap analysis, so there might be some updates to it in the near future.
Thanks for sending this out. I'm going to reply about what I know of the status of nova-related planned upstream features.
On NUMA-aware live migration, it was identified as the top priority issue in the NFV/HPC pain points forum session [1]. The spec has been approved before in the past for the Rocky cycle, so it's a matter of re-proposing it for re-approval in Stein. We need to confirm with artom and/or stephenfin whether one of them can pick it up this cycle.
Turns out this spec has already been re-proposed for Stein as of Sep 4: https://review.openstack.org/599587 and is under active review now. Apologies for missing this in my previous reply.
I don't know as much about the shared/dedicated vCPUs on a single host or the shared vCPU extension, but the cited spec [2] has one +2 already. If we can find a second approver, we can work on this too in Stein.
The vTPM support spec was merged about two weeks ago and we are awaiting implementation patches from cfriesen.
The HPET support spec was merged about two weeks ago and the implementation patch is under active review in a runway with one +2 now.
For vCPU model, I'm not aware of any new proposed spec for Stein from the STX community as of today. Let us know if/when the spec is proposed.
For disk performance fixes, the specless blueprint patch is currently under active review in a runway.
The extra spec validation spec [3] is under active review now.
For the bits that will be addressed using upstream features that are already available, I assume the STX community will take care of this. Please reach out to us in #openstack-nova or on the ML if there are questions/issues.
For the bugs, again feel free to reach out to us for reviews/help.
Cheers, -melanie
[1] https://etherpad.openstack.org/p/BER-nfv-hpc-pain-points [2] https://review.openstack.org/555081 [3] https://review.openstack.org/618542
participants (5)
-
Jeremy Stanley
-
melanie witt
-
Miguel Lavalle
-
Mohammed Naser
-
Sean Mooney