Hey!
Thanks for writing this up! It looks like a really nice guide and totally gonna be useful. And I totally will be recommending it for read in case of questions regarding such migration.
I have some questions though regarding the process and setup you're testing against. It's not that you need to answer all of them, but wondering about also direction you are going as a whole.
1. Do you have network nodes on the same hosts as computes, controllers or standalone ones? This one does not seem to cover L3 migration yet, and I assume it will be described in the next article in series?
It is an openstack ansible setup with three controllers. The neutron dhcp agents are running on each controller.. L3 migration is not tested, as we dont use routers. It is mainly
provider based vlans mapped to the physical dc networks.
2. It seems that your approach is to rollout changes to OVN across all compute nodes at once, and then reboot VMs, but then you see an unexpected behavior regarding DHCP/Metadata/etc. I was wondering if you've tried to run the playbook with a `--limit` to convert per-compute basis? And then live migrate instances between computes, to allow them to be re-wired? At least I've heard about this approach during one of Summit sessions hold by CERN. But then I'm having the same concerns about how to migrate L3 and FIPs...
Let me clear the confusion. We do not upgrade the computes via playbook. We have a pre prepared os image with all the required upgraded version packages, so that,
when we just reboot the compute nodes, they come up with that os image, after which, post boot scripts configure the neutron and nova services. This is seems to be much
faster compared to the playbook upgrade of computes, especially given the huge scale of compute nodes.
This method specifically addresses a particular scenario, where we do not have enough downtime to upgrade all the compute nodes in a single change window of 6 hours.
So, if we complete in part (a few nodes), rest of the computes would still be in lxb plugin, until we fully upgrade during another downtime, which causes this issue of metadata
proxy not being created inside the qdhcp namespaces even though we restart the neutron-dhcp-agent service. As a result, any vm reboots, new workloads on these lxb
computes, would fail to get their IP leases and nova metadata access. So, the ssh keys and other data would not be pushed into the vms.
3. I was wondering if you played with neutron-ovn-db-sync-util [1] to migrate the state and populate OVN NB/SB databases?
Yes. This is part of the migration process.
Hope this helps...