Open Stack

On 4.1.2017 09:13, Saravanan KR wrote:
> Hello,
>
> The aim of this mail is to ease the DPDK deployment with TripleO. I
> would like to see if the approach of deriving THT parameter based on
> introspection data, with a high level input would be feasible.
>
> Let me brief on the complexity of certain parameters, which are
> related to DPDK. Following parameters should be configured for a good
> performing DPDK cluster:
> * NeutronDpdkCoreList (puppet-vswitch)
> * ComputeHostCpusList (PreNetworkConfig [4], puppet-vswitch) (under review)
> * NovaVcpuPinset (puppet-nova)
>
> * NeutronDpdkSocketMemory (puppet-vswitch)
> * NeutronDpdkMemoryChannels (puppet-vswitch)
> * ComputeKernelArgs (PreNetworkConfig [4]) (under review)
> * Interface to bind DPDK driver (network config templates)
>
> The complexity of deciding some of these parameters is explained in
> the blog [1], where the CPUs has to be chosen in accordance with the
> NUMA node associated with the interface. We are working a spec [2], to
> collect the required details from the baremetal via the introspection.
> The proposal is to create mistral workbook and actions
> (tripleo-common), which will take minimal inputs and decide the actual
> value of parameters based on the introspection data. I have created
> simple workbook [3] with what I have in mind (not final, only
> wireframe). The expected output of this workflow is to return the list
> of inputs for "parameter_defaults",  which will be used for the
> deployment. I would like to hear from the experts, if there is any
> drawbacks with this approach or any other better approach.
>
> This workflow will ease the TripleO UI need to integrate DPDK, as UI
> (user) has to choose only the interface for DPDK [and optionally, the
> number for CPUs required for PMD and Host]. Of-course, the
> introspection should be completed, with which, it will be easy to
> deploy a DPDK cluster.
>
> There is a complexity if the cluster contains heterogeneous nodes, for
> example a cluster having HP and DELL machines with different CPU
> layout, we need to enhance the workflow to take actions based on
> roles/nodes, which brings in a requirement of localizing the above
> mentioned variables per role. For now, consider this proposal for
> homogeneous cluster, if there is a value in this, I will work towards
> heterogeneous clusters too.
>
> Please share your thoughts.
>
> Regards,
> Saravanan KR
>
>
> [1] https://krsacme.github.io/blog/post/dpdk-pmd-cpu-list/
> [2] https://review.openstack.org/#/c/396147/
> [3] https://gist.github.com/krsacme/c5be089d6fa216232d49c85082478419
> [4] https://review.openstack.org/#/c/411797/6/extraconfig/pre_network/host_config_and_reboot.role.j2.yaml
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

We are recently getting quite a lot of requests such as this - of 
bringing up the logic which takes the introspection data and 
pre-populates the parameters with it. This is usable for network 
configuration, storage etc. So as It seems there is a real need for such 
features, TripleO team should discuss general approach on how this logic 
should work. Mistral workflow is an obvious choice, we just need to make 
sure a certain pre-requisities are met.

 From the GUI point of view, we probably don't want this type of 
workflow to happen as part of starting the deployment. That's too late. 
We need to find mechanism which helps us to identify when such workflow 
can run and it should probably be confirmed by user. And when it 
finishes, Used needs to be able to review those parameters and confirm 
that this is the configuration he wants to deploy and should be able to 
make changes to it.

Obviously, as this workflow uses introspection data, user could be 
offered to run it when introspection finishes. Problem is that we need 
to verify, that using this workflow is valid for the deployment setup 
user is creating. For example, If this workflow sets parameters which 
are defined in templates which user won't deploy, it is wrong.

So I think that proper way would be to embed this in environment selection:
Environment selection is a step where user does high level deployment 
decisions - selects environments which are going to be used for 
deployment. We could bring in a mechanism (embedded in environment file 
or capabilities-map.yaml maybe?) which would allow GUI to do: 'hey, 
you've just enabled feature Foo, and you have introspection data 
available. Do you wish to pre-configure this feature using this data?' 
On confirmation the workflow is triggered and configuration is 
populated. User reviews it and does tweaks if he wants.

I'd love to hear feedback on this.

--Jirka