<div dir="ltr">Hello, <div><br></div><div>We are noticing two issues with these changes:<br><br><b>1</b>. The overrides on the file <font face="monospace">/<b>etc/openstack_deploy/env.d/nova.yml</b></font> are not being honored:<br> <br><font face="monospace">nova_compute_container:<br> belongs_to:<br> - compute_containers<br> - kvm-compute_containers<br> - qemu-compute_containers<br> contains:<br> - neutron_sriov_nic_agent<br> - neutron_ovn_controller<br> - nova_compute<br> properties:<br> is_metal: true<br></font> <br><br>The following block continues to be populated in with compute nodes in <font face="monospace"><b>/etc/openstack_deploy/openstack_inventory.json</b></font> after deleting and recreating the inventory file with <font face="monospace"><b>/opt/openstack-ansible/scripts/inventory-manage.py</b>:</font><br><br><font face="monospace">"neutron_ovn_gateway": {<br> "children": [],<br> "hosts": [<br> "cmp3",<br> "cmp4",<br> "net1",<br> "net2"<br> ]<br> },</font><br> <br><br><b>2</b>. After changing <b><font face="monospace">group_binds </font></b>to <font face="monospace"><b>neutron_ovn_gateway </b></font>instead of the previous <b><font face="monospace">neutron_ovn_controller</font></b>, group binds for <b><font face="monospace">provider_networks </font></b>in <font face="monospace"><b>openstack_user_config.yml</b></font>. Openstack-ansible still wants to create network mappings for compute nodes, which are not part of the <b><font face="monospace">neutron_ovn_gateway </font></b>host group:<br><br>=.=.=.=.=.=.=.=.=<br><font face="monospace">TASK [os_neutron : Setup Network Provider Bridges] **********************************************************************************************************************************************************************************************************************************************************************************************<br><br>fatal: [cmp4]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 1\n\nThe error appears to be in '/etc/ansible/roles/os_neutron/tasks/providers/setup_ovs_ovn.yml': line 55, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Setup Network Provider Bridges\n ^ here\n"}</font><br>=.=.=.=.=.=.=.=.=<br><br>I'll dig deeper to see if I can find anything that helps. But any assistance will be appreciated.<br><br>Thanks<br><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Sep 2, 2023 at 12:08 PM Dmitriy Rabotyagov <<a href="mailto:noonedeadpunk@gmail.com">noonedeadpunk@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div>Hi,<div dir="auto"><br></div><div dir="auto">I think this is known issue which should be fixed with the following patch:</div><div dir="auto"><a href="https://review.opendev.org/c/openstack/openstack-ansible/+/892540" target="_blank">https://review.opendev.org/c/openstack/openstack-ansible/+/892540</a></div><div dir="auto"><br></div><div dir="auto">In the meanwhile you should be able to workaround the issue by creating /etc/openstack_deploy/env.d/nova.yml file with following content:</div><div dir="auto"><br></div><div dir="auto">nova_compute_container:</div><div dir="auto"> belongs_to:</div><div dir="auto"> - compute_containers</div><div dir="auto"> - kvm-compute_containers</div><div dir="auto"> - qemu-compute_containers</div><div dir="auto"> contains:</div><div dir="auto"> - neutron_sriov_nic_agent</div><div dir="auto"> - neutron_ovn_controller</div><div dir="auto"> - nova_compute</div><div dir="auto"> properties:</div><div dir="auto"> is_metal: true</div><div dir="auto"><br></div><div dir="auto">You might also need to remove computes from the inventory using /opt/openstack-ansible/scripts/inventory-manage.py -r cmp03</div><div dir="auto"><br></div>They will be re-added next time running openstack-ansible or dynamic-inventory.py. Removing them is needed to ensure that they're not part of ovn-gateway related group.</div><div dir="auto">You might also need to stop ovn-gateway service on these computes manually, but I'm not sure 100% about that.<br><br><div class="gmail_quote" dir="auto"><div dir="ltr" class="gmail_attr">On Sat, Sep 2, 2023, 17:47 Roger Rivera <<a href="mailto:roger.riverac@gmail.com" target="_blank">roger.riverac@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hello,<br><br>We have deployed an <font face="monospace">openstack-ansible</font> cluster to test it on_metal with OVN and defined <b>dedicated gateway hosts</b> connecting to the external network with the <font face="monospace"><b>network-gateway_hosts</b></font> host group. Unfortunately, we are not able to connect to the external/provider networks. It seems that traffic wants to reach external networks via the hypervisor nodes and not the gateway hosts.</div><div dir="ltr"><br>Any suggestions on changes needed to our configuration will be highly appreciated.<br><br>Environment:<br>-Openstack Antelope<br>-Ubuntu 22 on all hosts<br>-3 infra hosts - 1xNIC (ens1)<br>-2 compute hosts - 1xNIC (ens1)<br>-2 gateway hosts - 2xNIC (ens1 internal, ens2 external)<br>-No linux bridges are created.<br><br>The gateway hosts are the only ones physically connected to the external network via physical interface ens2. Therefore, we need all external provider network traffic to traverse via these gateway hosts.<br><br>Tenant networks work fine and VMs can talk to each other. However, when a VM is spawned with a floating IP to the external network, they are unable to reach the outside network.<br><br>Relevant content from openstack-ansible configuration files:<br><br><br>=.=.=.=.=.=.=.=<br>openstack_user_config.yml</div><div dir="ltr">=.=.=.=.=.=.=.=<br>```<br>...<br>provider_networks:<br> - network:<br> container_bridge: "br-mgmt"<br> container_type: "veth"<br> container_interface: "ens1"<br> ip_from_q: "management"<br> type: "raw"<br> group_binds:<br> - all_containers<br> - hosts<br> is_management_address: true<br> - network:<br> container_bridge: "br-vxlan"<br> container_type: "veth"<br> container_interface: "ens1"<br> ip_from_q: "tunnel"<br> #type: "vxlan"<br> type: "geneve"<br> range: "1:1000"<br> net_name: "geneve"<br> group_binds:<br> - neutron_ovn_controller<br> - network:<br> container_bridge: "br-flat"<br> container_type: "veth"<br> container_interface: "ens1"<br> type: "flat"<br> net_name: "flat"<br> group_binds:<br> - neutron_ovn_controller<br> - network:<br> container_bridge: "br-vlan"<br> container_type: "veth"<br> container_interface: "ens1"<br> type: "vlan"<br> range: "101:300,401:500"<br> net_name: "vlan"<br> group_binds:<br> - neutron_ovn_controller<br> - network:<br> container_bridge: "br-storage"<br> container_type: "veth"<br> container_interface: "ens1"<br> ip_from_q: "storage"<br> type: "raw"<br> group_binds:<br> - glance_api<br> - cinder_api<br> - cinder_volume<br> - nova_compute<br> <br>...<br><br>compute-infra_hosts:<br> inf1:<br> ip: 172.16.0.1<br> inf2:<br> ip: 172.16.0.2<br> inf3:<br> ip: 172.16.0.3<br><br>compute_hosts:<br> cmp4:<br> ip: 172.16.0.21<br> cmp3:<br> ip: 172.16.0.22<br><br>network_hosts:<br> inf1:<br> ip: 172.16.0.1<br> inf2:<br> ip: 172.16.0.2<br> inf3:<br> ip: 172.16.0.3<br><br>network-gateway_hosts:<br> net1:<br> ip: 172.16.0.31<br> net2:<br> ip: 172.16.0.32<br><br>```<br><br><br>=.=.=.=.=.=.=.=<br>user_variables.yml</div><div dir="ltr">=.=.=.=.=.=.=.=<br>```<br>---<br>debug: false<br>install_method: source<br>rabbitmq_use_ssl: False<br>haproxy_use_keepalived: False<br>...<br>neutron_plugin_type: ml2.ovn<br>neutron_plugin_base:<br> - neutron.services.ovn_l3.plugin.OVNL3RouterPlugin<br><br>neutron_ml2_drivers_type: geneve,vlan,flat<br>neutron_ml2_conf_ini_overrides:<br> ml2:<br> tenant_network_types: geneve<br><br>...<br>```<br><br>=.=.=.=.=.=.=.=<br>env.d/neutron.yml</div><div dir="ltr">=.=.=.=.=.=.=.=<br>```<br>component_skel:<br> neutron_ovn_controller:<br> belongs_to:<br> - neutron_all<br> neutron_ovn_northd:<br> belongs_to:<br> - neutron_all<br><br>container_skel:<br> neutron_agents_container:<br> contains: {}<br> properties:<br> is_metal: true<br> neutron_ovn_northd_container:<br> belongs_to:<br> - network_containers<br> contains:<br> - neutron_ovn_northd<br><br>```<br><br>=.=.=.=.=.=.=.=<br>env.d/nova.yml</div><div dir="ltr">=.=.=.=.=.=.=.=<br>```<br>component_skel:<br> nova_compute_container:<br> belongs_to:<br> - compute_containers<br> - kvm-compute_containers<br> - lxd-compute_containers<br> - qemu-compute_containers<br> contains:<br> - neutron_ovn_controller<br> - nova_compute<br> properties:<br> is_metal: true<br>```<br><br>=.=.=.=.=.=.=.=<br>group_vars/network_hosts</div><div dir="ltr">=.=.=.=.=.=.=.=<br>```<br>openstack_host_specific_kernel_modules:<br> - name: "openvswitch"<br> pattern: "CONFIG_OPENVSWITCH"<br>```<br><br>The nodes layout is like this:<br><br> <img src="cid:ii_lm26wmpy0" alt="image.png" width="563" height="436"><br><br><br>Any guidance on what we have wrong or how to improve this configuration will be appreciated. We need to make external traffic for VMs to go out via the gateway nodes and not the compute/hypervisor nodes.<br><br>Thank you.<br><br>Roger<br></div></div>
</blockquote></div></div></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"><div><b>Roger Rivera</b></div></span></div></div>