On Fri, Jul 12, 2019 at 3:59 PM Harald Jensås <hjensas@redhat.com> wrote:
I've said this before, but I think we should turn this nova-less around. Now with nova-less we create a bunch of servers, and write up the parameters file to use the deployed-server approach. Effectively we still neet to have the resource group in heat making a server resource for every server. Creating the fake server resource is fast, because Heat does'nt call Nova,Ironic to create any resources. But the stack is equally big, with a stack for every node. i.e not N=1.
What you are doing here, is essentially to say we don't create a resource group that then creates N number of role stacks, one for each overcloud node. You are creating a single generic "server" definition per Role. So we drop the resource group and create OS::Triple::{{Role}}.Server 1-time (once). To me it's backwards to push a large struct with properties for N=many nodes into the creation of that stack.
I'm not entirely following what you're saying is backwards. What I've proposed is that we *don't* have any node specific data in the stack. It sounds like you're saying the way we do it today is backwards. It's correct that what's been proposed with metalsmith currently still requires the full ResourceGroup with a member for each node. With the template changes I'm proposing, that wouldn't be required, so we could actually do the Heat stack first, then metalsmith.
Currently the puppet/role-role.yaml creates all the network ports etc. As you only want to create it once, it instead could simply output the UUID of the networks+subnets. These are identical for all servers in the role. So we end up with a small heat stack.
Once the stack is created we could use that generic "server" role data to feed into something (ansible?, python?, mistral?) that calls metalsmith to build the servers, then create ports for each server in neutron, one port for each network+subnet defined in the role. Then feed that output into the json (hieradata) that is pushed to each node and used during service configuration, all the things we need to configure network interfaces, /etc/hosts and so on. We need a way to keep track of which ports belong to wich node, but I guess something simple like using the node's ironic UUID in either the name, description or tag field of the neutron port will work. There is also the extra filed in Ironic which is json type, so we could place a map of network->port_uuid in there as well.
It won't matter whether we do baremetal provisioning before or after the Heat stack. Heat won't care, as it won't have any expectation to create any servers or that they are already created. We can define where we end up calling the metalsmith piece as it should be independent of the Heat stack if we make these template changes.
Another idea I've been pondering is if we put credentials on the overcloud nodes so that the node itself could make the call to neutron on the undercloud to create ports in neutron. I.e we just push the UUID of the correct network and subnet where the resource should be created, and let the overcloud node do the create. The problem with this is that
I don't think it would be a good idea to have undercloud credentials on the overcloud nodes.
I think the creation of the actual Networks and Subnets can be left in heat, it's typically 5-6 networks and 5-6 subnets so it's not a lot of resources. Even in a large DCN deployment having 50-100 subnets per network or even 50-100 networks I think this is'nt a problem.
Agreed, I'm not specifically proposing we move those pieces at this time. -- -- James Slagle --