I ended up figuring it out.  All the interfaces were created correctly.  However, for some reason the DHCP options for this subnet were incorrect.  The static route for the subnet was set to the .6 address but neutron had setup the metadata interface on .16.  Once I modified the DHCP options and built new instances everything worked fine.

Thanks
Chris

On Mon, Feb 21, 2022 at 6:49 AM Rodolfo Alonso Hernandez <ralonsoh@redhat.com> wrote:
Hello Chris:

10.210.65.6 is the IP address of the metadata namespace interface. The metadata service is on 169.254.169.254 address; this route is redirecting this traffic. Check the network ID and then the corresponding metadata:
root@u20ovn:~# ip netns
ovnmeta-5274fa45-c33a-4c66-be3a-b8ecd294c769 (id: 3)
ovnmeta-ee1597ee-579d-49dc-9b90-fbb8f98baef9 (id: 2)

root@u20ovn:~# ip netns exec $meta ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: tap5274fa45-c1@if62: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether fa:16:3e:dd:2b:2c brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.0.10.3/26 brd 10.0.10.63 scope global tap5274fa45-c1
       valid_lft forever preferred_lft forever
    inet 169.254.169.254/32 brd 169.254.169.254 scope global tap5274fa45-c1


Check if the metadata agent logs and verify you are not hitting [1]. Check you have [2].

Regards.

[1]https://bugs.launchpad.net/neutron/+bug/1952550
[2]https://review.opendev.org/c/openstack/neutron/+/822669


On Thu, Feb 17, 2022 at 8:14 PM Satish Patel <satish.txt@gmail.com> wrote:
Disregard my above email, I didn't see that you created vm using a
fixed IP address.  When you create a network/subnet during that time
OVN creates this special localport IP. In my case its 10.1.1.5

I believe you shouldn't use that IP address for VM.

port 0e36e2b1-12d2-4eb2-a9ea-556644a73067
        type: localport
        addresses: ["fa:16:3e:6e:36:f9 10.1.1.5"]

On Thu, Feb 17, 2022 at 2:00 PM Satish Patel <satish.txt@gmail.com> wrote:
>
> 10.210.64.6 must be your computer node IP, it will redirect metadata
> requests to your controller node.  169.254.169.254 is a special IP
> address for meta services.
>
> On Thu, Feb 17, 2022 at 1:55 PM Chris DiLorenzo <cdilorenzo@gmail.com> wrote:
> >
> > Hello,
> >
> > I am having a weird issue.  I am running openstack Victoria with OVN.  Generally, everything is working.  I created a VM using a fixed port with an IP of 10.210.64.6.  There are no issues with this VM.  When I create a new VM, I get this route info:
> >
> > [   29.363664] cloud-init[564]: ci-info: ++++++++++++++++++++++++++++++++Route IPv4 info++++++++++++++++++++++++++++++++
> > [   29.368568] cloud-init[564]: ci-info: +-------+-----------------+-------------+-----------------+-----------+-------+
> > [   29.369945] cloud-init[564]: ci-info: | Route |   Destination   |   Gateway   |     Genmask     | Interface | Flags |
> > [   29.372441] cloud-init[564]: ci-info: +-------+-----------------+-------------+-----------------+-----------+-------+
> > [   29.376434] cloud-init[564]: ci-info: |   0   |     0.0.0.0     | 10.210.64.1 |     0.0.0.0     |    ens3   |   UG  |
> > [   29.377820] cloud-init[564]: ci-info: |   1   |   10.210.64.0   |   0.0.0.0   | 255.255.255.192 |    ens3   |   U   |
> > [   29.380417] cloud-init[564]: ci-info: |   2   | 169.254.169.254 | 10.210.64.6 | 255.255.255.255 |    ens3   |  UGH  |
> > [   29.384448] cloud-init[564]: ci-info: +-------+-----------------+-------------+-----------------+-----------+-------+
> >
> > The IP is set correctly, but you can see I am getting a gateway of 10.210.64.6 for the metadata ser. ice. This is preventing the VM from reaching the service so the SSH key is not being added preventing me from logging in.  As far as I can tell, there are no error and every VM I build is getting this .6 IP for this route.
> >
> > Thanks,
> > Chris
> >
> >