<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 23, 2018 at 10:16 AM, Harald Jensås <span dir="ltr"><<a href="mailto:hjensas@redhat.com" target="_blank">hjensas@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="gmail-HOEnZb"><div class="gmail-h5">On Fri, 2018-04-20 at 14:44 +0200, Thomas Herve wrote:<br>
> On Thu, Apr 19, 2018 at 2:59 PM, Harald Jensås <<a href="mailto:hjensas@redhat.com">hjensas@redhat.com</a>><br>
> wrote:<br>
> > Hi,<br>
> <br>
> Hi, thanks for sending this. Responses inline.<br>
> <br>
> > When configuring TripleO deployments with nodes on routed ctlplane<br>
> > networks we need to pass some per-network properties to the<br>
> > NetworkConfig resource[1] in THT. We get the ``ControlPlaneIp``<br>
> > property using get_attr, but the NIC configs need a couple of more<br>
> > parameters[2], for example: ``ControlPlaneSubnetCidr``,<br>
> > ``ControlPlaneDefaultRoute`` and ``DnsServers``.<br>
> > <br>
> > Since queens these templates are jinja templated, to generate<br>
> > things<br>
> > from from network_data.yaml. When using routed ctlplane networks,<br>
> > the<br>
> > parameters ``ControlPlaneSubnetCidr`` and<br>
> > ``ControlPlaneDefaultRoute``<br>
> > will be different. So we need to use static per-role<br>
> > Net::SoftwareConfig templates, and add parameters such as<br>
> > ``<wbr>ControlPlaneDefaultRouteLeafX`<wbr>`.<br>
> > <br>
> > The values the use need to pass in for these are already available<br>
> > in<br>
> > the neutron ctlplane network configuration on the undercloud. So<br>
> > ideally we should not need to ask the user to provide them in<br>
> > parameter_defaults, we should resolve the correct values<br>
> > automatically.<br>
> <br>
> To make it clear, what you want to prevent is the need to add more<br>
> keys in network_data.yaml?<br>
> <br>
> As those had to be provided at some point, I wonder if tripleo can't<br>
> find a way to pass them again on the overcloud deploy.<br>
> <br>
</div></div>No, the networks defined in network_data.yaml is fine, that is the data<br>
used to create the neutron stuff so passing the data from there is<br>
already in place to some extent.<br>
<br>
But, the ctlplane network is not defined in network_data.yaml.<br></blockquote><div><br></div><div>We could add the ControlPlaneDefaultRoute and ControlPlaneSubnetCidr to network_data.yaml, but this would involve some duplication of configuration data, since those are currently defined in undercloud.conf. A more robust solution might be to generate network_data.yaml from that info in undercloud.conf, but currently we don't modify any files in the tripleo-heat-templates package after it gets installed.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div><div class="gmail-h5"><br>
> Inspecting neutron is an elegant solution, though.<br>
> <br>
<br>
<br>
> > : We can get the port ID using get_attr:<br>
> > <br>
> > {get_attr: [<server>, addresses, <network name_or_id>, 0, port]}<br>
> > <br>
> > : From there outside of heat we can get the subnet_id:<br>
> > <br>
> > openstack port show 2fb4baf9-45b0-48cb-8249-<wbr>c09a535b9eda \<br>
> > -f yaml -c fixed_ips<br>
> > <br>
> > fixed_ips: ip_address='172.20.0.10', subnet_id='2b06ae2e-423f-<br>
> > 4a73-<br>
> > 97ad-4e9822d201e5'<br>
> > <br>
> > : And finally we can get the gateway_ip and cidr of the subnet:<br>
> > <br>
> > openstack subnet show 2b06ae2e-423f-4a73-97ad-<wbr>4e9822d201e5 \<br>
> > -f yaml -c gateway_ip -c cidr<br>
> > <br>
> > cidr: <a href="http://172.20.0.0/26" rel="noreferrer" target="_blank">172.20.0.0/26</a><br>
> > gateway_ip: 172.20.0.62<br>
> > <br>
> > <br>
> > The problem is getting there using heat ...<br>
> > a couple of ideas:<br>
> > <br>
> > a) Use heat's ``external_resource`` to create a port resource,<br>
> > and then a external subnet resource. Then get the data<br>
> > from the external resources. We probably would have to make<br>
> > it possible for a ``external_resource`` depend on the server<br>
> > resource, and verify that these resource have the required<br>
> > attributes.<br>
> <br>
> I believe that's a relatively easy fix. It's unclear why we didn't<br>
> allow that in the first place, probably because we were missing a use<br>
> case, but it seems valuable here.<br>
> <br>
> > b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as<br>
> > well probably) to include the data.<br>
> > <br>
> > If we do this we should probably aim to be in parity with<br>
> > what is made available to clients getting the configuration<br>
> > from dhcp. (mtu, dns_domain, dns_servers, prefixlen,<br>
> > gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode<br>
> > etc.)<br>
> <br>
> I'm with you on exposing more neutron data to the Port resource. It<br>
> can be complicated because some of them are implementation specific,<br>
> but we can look into those.<br>
> <br>
> I don't think adding them directly to the Server resource makes a ton<br>
> of sense though.<br>
> <br>
</div></div>In tripleo, the ctlplane interface is an implicit port created by the<br>
server resource. :( (Attempts where made to change this, but upgrades<br>
would'nt work) So the server resource is where I would find it most<br>
useful. (Adding attributes to the port resource, and then using<br>
external resource for the implicit server ports may be a compromise.<br>
Nested dependencies for external_resources might be hard?)<br></blockquote><div><br></div><div>Yes, the port is currently created as part of the Ironic server resource. We would have more flexibility if this were a separate Neutron port, but we need to be able to support upgrades. This would require the ability in Heat to detach the implicit port from the Ironic resource, and attach a Neutron port resource with the same IP to a node without rebuilding the entire node. This isn't currently possible.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<span class="gmail-"><br>
> > c) Create a new heat function to read properties of any<br>
> > openstack resource, without having to make use of the<br>
> > external_resource in heat.<br>
> <br>
> It's an interesting idea, but I think it would look a lot like what<br>
> external resources are supposed to be.<br>
> <br>
> I see a few changes:<br>
> * Allow external resource to depend on other resources<br>
> * Expose more port attributes<br>
> * Expose more subnet attributes<br>
> <br>
> If you can list the attributes you care about that'd be great.<br>
> <br>
<br>
</span>Guess what I envision is a client_config attribute, a map with data<br>
useful to configure a network interface on the client. (I put * on the<br>
ones I believe could be useful for TripleO)<br>
<br>
* /v2.0/networks/{network_id}/<wbr>mtu <br>
/v2.0/networks/{network_id}/<wbr>dns_domain<br>
* /v2.0/subnets/{subnet_id}/dns_<wbr>nameservers <br>
* /v2.0/subnets/{subnet_id}/<wbr>host_routes <br>
/v2.0/subnets/{subnet_id}/ip_<wbr>version<br>
* /v2.0/subnets/{subnet_id}/<wbr>gateway_ip<br>
* /v2.0/subnets/{subnet_id}/cidr<br>
* /v2.0/subnets/{subnet_id}/<wbr>ipv6_address_mode<br>
* /v2.0/subnets/{subnet_id}/<wbr>ipv6_ra_mode<br>
/v2.0/ports/{port_id}/<wbr>description - Why not?<br>
/v2.0/ports/{port_id}/dns_<wbr>assignment<br>
/v2.0/ports/{port_id}/dns_<wbr>domain<br>
/v2.0/ports/{port_id}/dns_name<br>
* /v2.0/ports/{port_id}/fixed_<wbr>ips - We have this already<br>
/v2.0/ports/{port_id}/name - Why not?<br>
<br>
<br>
I've added Dan Sneddon on CC as well. Guess there is the question if<br>
TripleO will/want to continue using heat, neutron, nova etc.<br>
<br>
<br>
<br>
//<br>
<span class="gmail-HOEnZb"><font color="#888888">Harald<br>
</font></span></blockquote></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra">I can't speak to the roadmap of Heat/Neutron/Nova on the undercloud, for the immediate future I don't see us moving away from Heat entirely due to upgrade requirements.</div><div class="gmail_extra"></div></div><br>I can see another use case for this Heat functionality, which is that I would like to be able to generate a report using Heat that lists all the ports in use in the entire deployment. This would be generated post-deployment, and could be used to populate an external DNS server, or simply to report on which IPs belong to which nodes.</div><div class="gmail_extra"><br></div><div class="gmail_extra">-- <br><div class="gmail_signature"><div dir="ltr">Dan Sneddon | Senior Principal OpenStack Engineer<br><a href="mailto:dsneddon@redhat.com" target="_blank">dsneddon@redhat.com</a> | <a href="http://redhat.com/openstack" target="_blank">redhat.com/openstack</a><br>dsneddon:irc | @dxs:twitter<br></div></div>
</div></div>