Hi,
On 9 Jul 2020, at 17:35, Johannes Scheuermann <joh.scheuer@gmail.com> wrote:
Hi together,
currently we exploring how we can reboot a compute node without any interruptions for the networking stack. We run Openstack Train with ml2 driver Linux bridge and dnsmasq for DHCP and internal DNS. The DHCP setup runs as high availability setup with 3 replicas. During our tests we identified the following challenges:
1.)
If we reboot the machine without doing anything on the network layer all ports will be rescheduled. Also the networks will be removed from the (dead) agent and will be reassigned to another agent. But for each reboot we have some leftover ports with the device-id "reserved_dhcp_port". These ports can safely deleted (we haven't figured out where the issue in the neutron code is).
It’s done here https://opendev.org/openstack/neutron/src/branch/master/neutron/db/agentsche... and it’s done by purpose. The issue may be that this reserved port should be used on the new agent so we should check why it isn’t and why new port is created for new agent.
2.)
If we disable the network agent like described here: https://docs.openstack.org/neutron/train/admin/config-dhcp-ha.html and then remove the disabled agent from all networks we have an even worse behaviour since the neutron scheduler doesn't reschedule the network to a different agent.
So what is the correct way to ensure that the reboot of a node has no (or only small) interruptions to the networking service? The current issue is that if we remove one agent we might remove the port that is the first entry in the clients (VM's) resolv.conf which means that each request will be delayed by the default timeout.
And is there any option to "migrate" a network from one agent to another?
You can manually remove networks from one agent with command like: $ neutron dhcp-agent-network-remove <old_agent_id> <network_id> And then add it to the new one with: $ neutron dhcp-agent-network-add <new_agent_id> <network_id>
Thanks in advance,
Johannes Scheuermann
— Slawek Kaplonski Principal software engineer Red Hat