<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 23, 2022 at 5:37 PM Satish Patel <<a href="mailto:satish.txt@gmail.com">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Luis,</div><div><br></div><div>/cc - openstack discuss mailing list</div><div><br></div><div>Thank you so much for clearing my doubts. I have a counter question. </div><div><br></div><div> BGP Mode:</div><div> - In bgp mode where i should run ovn-bgp-agent ? Network node or Compute node? </div></div></blockquote><div><br></div><div>It need to be running in all the nodes, unless you are only interested in a limited functionality:</div><div>- For FIP and VMs on the provider network you would only need them in the compute nodes</div><div>- For the expose_tenant_network flag, you would need it on the network nodes, as the traffic is exposed through the ovn router gateway port</div><div>- For octavia load balancer with ovn provider (where the VIP is on the provider network and the members in a tenant network) you will also need the agent running on the networker node, as the traffic is also injected into the OVN overlay at the networker node (through the ovn router gateway port)<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>EVPN Mode:</div><div> - In evpn mode where i should run ovn-bgp-agent? </div></div></blockquote><div><br></div><div>Only in the networker nodes is enough, as this is where the IPs gets exposed and where the traffic gets injected into the OVN overlay through the ovn router gateway port <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div> - In my lab i am using your heck "How to use it without BGPVPN" to create vni mapping, do you think because of that i am not seeing vni pushed out to FRR automatically? <a href="https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/" target="_blank">https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/</a> </div></div></blockquote><div><br></div><div>indeed, I think the hack may be a bit outdated, and now you need to set up both the VNI and the AS number:</div><div><a href="https://opendev.org/x/ovn-bgp-agent/src/branch/master/ovn_bgp_agent/drivers/openstack/watchers/evpn_watcher.py#L92-L93">https://opendev.org/x/ovn-bgp-agent/src/branch/master/ovn_bgp_agent/drivers/openstack/watchers/evpn_watcher.py#L92-L93</a></div><div><br></div><div>Otherwise the event won't match and will be dismissed. Try also doing:</div><div><pre class="gmail-wp-block-code"><code>ovn-nbctl set logical_switch_port XXX external_ids:"neutron_bgpvpn\:as"=65000</code></pre></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div> - In your demo link I can see you have bridge two different deployment clouds using the same VNI, In this scenario we need a router (or vrf) somewhere correct to bridge to different subnet IPs, correct? or is this L2 stretch? </div></div></blockquote><div><br></div><div>Yes, in my demo I have:</div><div>- One spine/leaf deployment configured with frr where and OpenStack deployment is connected to the leafs</div><div>- One devstack deployment with Kuryr and Kubernetes (to use neutron ports for kubernetes pods, therefore connecting neutron ports to neutron ports under the hood)</div><div>- One router VM where both spines are connected to, as well as the devstack VM</div><div><br></div><div>So, it is all L3, with different networks on each side. In the router VM I have the next frr configuration:</div><div></div><div><pre style="line-height:16.25px;margin:0px;padding:5px 0px;font-family:"Bitstream Vera Sans Mono",monospace;font-size:13px;color:rgb(0,0,0);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;word-spacing:0px;text-decoration-style:initial;text-decoration-color:initial">...<br>router bgp 65000
bgp log-neighbor-changes
bgp graceful-shutdown
neighbor downlink peer-group
neighbor downlink remote-as internal
neighbor downlink bfd
neighbor eth0 interface peer-group downlink
neighbor eth1 interface peer-group downlink
neighbor uplink peer-group
neighbor uplink remote-as external
neighbor uplink bfd
neighbor eth2 interface peer-group uplink
address-family ipv4 unicast
redistribute connected
neighbor downlink default-originate
neighbor downlink prefix-list only-host-prefixes in
neighbor uplink prefix-list only-default-host-prefixes in
exit-address-family
address-family l2vpn evpn
neighbor uplink activate
neighbor downlink activate
neighbor downlink route-reflector-client
exit-address-family<br>...</pre></div><div> </div><div>Where I define the connectivity to spines as internal and the one to devstack VM as external. And I just activated the l2vpn evpn so that the vni information is considered.<br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>I am going to open a new thread to discuss issues related to expose_tenant_networks=True flag issue.<br></div></div></blockquote><div><br></div><div>Thanks <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Aug 23, 2022 at 3:14 AM Luis Tomas Bolivar <<a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Satish! See inline<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 22, 2022 at 5:22 PM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Luis,<div><br></div><div>Welcome back from your vacation, sorry i didn't know you are on vacation. Hope you had a wonderful time. </div><div><br></div><div>How do i start conversion on opendev thread? is there a mailing list or are you saying open thread here - <a href="https://storyboard.openstack.org/#!/project/x/ovn-bgp-agent" target="_blank">https://storyboard.openstack.org/#!/project/x/ovn-bgp-agent</a></div></div></blockquote><div><br></div><div>Actually, you did it right (sending it to openstack-discuss), but it seems at some point we stop adding it on our replies... probably my fault due to replying on the phone while on vacation. <br></div><div><br></div><div>For the next issue we can open a different thread and I'll try not to drop the openstack-discuss list! xD</div><div><br></div><div>It would be great though if you can sent a follow up to the previous thread with the final outcome (fixed by updating, or adding config X, ..), so that community can see the project is alive, and other people facing similar issues can try your solution/config. </div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>I have an update for you, after upgrading the OVN/OVS version to the latest and that resolved lots of issues. As you mentioned earlier running the latest code is very important to get proper functionality. <br></div></div></blockquote><div><br></div><div>Awesome! Yeah, it is a new project, so functionality/fixes are being added regularly, therefore using the latest version is the best approach. The plan is to soon create a more stable version/release, once we have enough testing coverage and main functionality covered<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div><br></div><div>You are saying in BGP mode only FIP will get exposed but not the tenant VM ( What is this flag for expose_tenant_network=True ?)</div></div></blockquote><div><br></div><div>Main idea was:</div><div>- BGP mode for exposing VMs/LBs on provider networks, or with FIPs attached</div><div>- EVPN mode for tenant networks</div><div><br></div><div>To give you some more context, we started with the BGP mode, and did the expose_tenant_network to expose also tenant IPs. However, the BGP mode is lacking an API to decide what to expose. So we moved to the EVPN mode (with networking-bgpvpn as the API) to expose the tenant networks.</div><div><br></div><div>That said, you are right, and the expose_tenant_network flag is intended for the BGP mode, to expose all tenant networks (so you need to ensure no overlapping CIDRs). And, if that is not working (it may be, as there is no testing coverage for it), it is something we can definitely look at and fix. So, feel free to open a new thread on openstack-discuss, or bug in storyboard (whatever works for you better), and I'll try to fix asap.</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>You are saying in EVPN mode only tenant VM ip will get exposed but not FIP but in this design what is the use of getting tenant VM exposed but you can't expose FIP then how does external people access VMs? ( I am totally confused in EVPN design and use case because your tenant VM is already talking to each other using geneve tunnel so why do we need EVPN/L2VPN to expose them?) </div></div></blockquote><div><br></div><div>EVPN mode is indeed only for tenant VMs (and loadbalancer). Idea for this mode is to connect (N/S traffic, as you mention E/W is still using the normal geneve path) tenant networks between different OpenStack clusters. EVPN will create a vxlan tunnel connecting them. So, the vni/vxlan id selected (with networking-bgpvpn) is the vxlan tunnel encap id being used to connect both tenant networks. <br></div><div><br></div><div>So, the idea behind the EVPN mode is not to make your VMs in a tenant network publicly accessible, but being able to access them from a different tenant network in a different cloud (it actually does not need to be an OpenStack cloud, anything connected to the same vxlan id. Perhaps this is a bit more clear in this demo: <a href="https://ltomasbo.wordpress.com/2021/10/01/ovn-bgp-agent-interconnecting-kubernetes-pods-and-openstack-tenant-vms-with-evpn/" target="_blank">https://ltomasbo.wordpress.com/2021/10/01/ovn-bgp-agent-interconnecting-kubernetes-pods-and-openstack-tenant-vms-with-evpn/</a></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div> This is current status of my LAB</div><div><br></div><div>ovn-bgp-agent in BGP Mode: <br>I am successfully able to expose FIP and provider IPs (as you mentioned) but when i use expose_tenant_network=True in config then getting error and vm tenant ips not getting exposed to bgp so i am not sure what is the use case of that flag. <br></div></div></blockquote><div><br></div><div>Yep, most probably we broke something here due to lack of upstream testing for this flag. I'll check asap and try to fix it <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br>ovn-bgp-agent in EVPN Mode: <br>In this mode everything is working and tenant vm also gets exposed but when I attach FIP to vm those FIP ip are not getting exposed in BGP (as you mentioned in your reply). </div></div></blockquote><div>Yes, EVPN mode is not intended for FIPs or VMs on provider networks. Only for tenants<br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>But i am seeing one bug here where vni config not getting inserted in FRR <a href="https://opendev.org/x/ovn-bgp-agent/src/branch/master/ovn_bgp_agent/drivers/openstack/utils/frr.py#L26" target="_blank">https://opendev.org/x/ovn-bgp-agent/src/branch/master/ovn_bgp_agent/drivers/openstack/utils/frr.py#L26</a> <br></div><div><br></div><div>Who will trigger that code and when will that code get triggered to configure FRR for vrf/vni ? </div></div></blockquote><div><br></div><div>This is being added by networking-bgpvpn. You can see some details about the integration into my blogpost (<a href="https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/" target="_blank">https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/</a>) or in the upstream documentation: <a href="https://opendev.org/x/ovn-bgp-agent/src/branch/master/doc/source/contributor/evpn_mode_design.rst" target="_blank">https://opendev.org/x/ovn-bgp-agent/src/branch/master/doc/source/contributor/evpn_mode_design.rst</a> <br></div><div><br></div><div>Note though, to make networking-bgpvpn to work with this integration, this patch is needed (which btw I should include into the documentation): <a href="https://review.opendev.org/c/openstack/networking-bgpvpn/+/803161" target="_blank">https://review.opendev.org/c/openstack/networking-bgpvpn/+/803161</a></div><div><br></div><div>Cheers,</div><div>Luis<br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 22, 2022 at 5:44 AM Luis Tomas Bolivar <<a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Hi Satish,</div><div><br></div><div>Sorry I was on vacation. Trying to get through my inbox.</div><div><br></div><div>I'm not sure what is the current status, note there is a difference between EVPN and BGP mode. BGP mode is for FIPs and VMs/LBs on the provider network or with FIPs, while EVPN is for VMs on the tenant networks, without FIPs. Right now you cannot mix them and need to choose either EVPN or BGP mode. I'm planning to give it a try to make it multidriver, but I haven't even started yet.</div><div><br></div><div>BTW, as you did with the initial email, perhaps it is worth to have conversations on the opendev thread, so that anyone else facing the same issues can get some hints (or even provide feedback), that was one of the main ideas about moving it in there. To be able to have better support.</div><div><br></div><div>As for the questions:</div><div><div># # on rack-2-host-1 </div><div># when i created vm1 which endup on rack-2-host-1 but it doesn't expose the vm ip address. Is that normal behavior? <br></div><div> </div><div>This should be exposed in the node with cr-lrp port, i.e., rack1-host2, and it should be as simple as adding the IP to the lo-2001 dummy device<br></div><div><br></div><div># When I attach a floating ip to vm2 then why does my floating ip address not get exposed in BGP? <br></div><div>This is what I mentioned before, you cannot merge BGP and EVPN mode. <br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 8, 2022 at 1:41 PM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Hi Luis,<div><br></div><div>Sorry for bugging. I’m almost there now. Do you have any thought of following question <br><br><div dir="ltr">Sent from my iPhone</div><div dir="ltr"><br><blockquote type="cite">On Aug 5, 2022, at 1:16 AM, Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr">Good morning Luis,<div><br></div><div>Quick question, I have following deployment as per your lab</div><div><br></div><div>rack-1-host-1 (controller)</div><div>rack-1-host-2 (compute1 - This is hosting cr-lrp ports, inshort router)</div><div>rack-2-host-1 (compute2)</div><div><br></div><div>I have created two vms</div><div><br></div><div>vagrant@rack-1-host-1:~$ nova list<br>nova CLI is deprecated and will be a removed in a future release<br>+--------------------------------------+------+--------+------------+-------------+--------------------------------------+<br>| ID | Name | Status | Task State | Power State | Networks |<br>+--------------------------------------+------+--------+------------+-------------+--------------------------------------+<br>| aecb4f10-c46f-4551-b112-44e4dc007e88 | vm1 | ACTIVE | - | Running | private-test=10.0.0.105 |<br>| ceae14b9-70c2-4dbc-8071-0d64d9a0ca84 | vm2 | ACTIVE | - | Running | private-test=10.0.0.86, 172.16.1.200 |<br>+--------------------------------------+------+--------+------------+-------------+--------------------------------------+<br></div><div><br></div><div># on rack-1-host-2 </div><div>when i spun up vm2 which endup on rack-1-host-2 hence it created vrf-2001 on dummy lo-2001 interface and exposed vm2 ip address <a href="http://10.0.0.86/32" target="_blank">10.0.0.86/32</a></div><div><br></div><div>96: vrf-2001: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state UP group default qlen 1000<br> link/ether 22:cc:25:b3:7b:96 brd ff:ff:ff:ff:ff:ff<br>97: br-2001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf-2001 state UP group default qlen 1000<br> link/ether 0a:c3:23:7a:8f:0c brd ff:ff:ff:ff:ff:ff<br> inet6 fe80::851:67ff:fe64:b2c3/64 scope link<br> valid_lft forever preferred_lft forever<br>98: vxlan-2001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-2001 state UNKNOWN group default qlen 1000<br> link/ether 0a:c3:23:7a:8f:0c brd ff:ff:ff:ff:ff:ff<br> inet6 fe80::8c3:23ff:fe7a:8f0c/64 scope link<br> valid_lft forever preferred_lft forever<br>99: lo-2001: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf-2001 state UNKNOWN group default qlen 1000<br> link/ether d6:60:da:91:2e:6d brd ff:ff:ff:ff:ff:ff<br> inet <a href="http://10.0.0.86/32" target="_blank">10.0.0.86/32</a> scope global lo-2001<br> valid_lft forever preferred_lft forever<br> inet6 fe80::d460:daff:fe91:2e6d/64 scope link<br> valid_lft forever preferred_lft forever</div><div><br></div><div><br></div><div># on rack-2-host-1 </div><div>when i created vm1 which endup on rack-2-host-1 but it doesn't expose the vm ip address. Is that normal behavior? </div><div><br></div><div>When I attach a floating ip to vm2 then why does my floating ip address not get exposed in BGP? </div><div><br></div><div>Thank you in advance</div><div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 4, 2022 at 2:30 PM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Update: Good news, I found what was wrong.<div><br></div><div>After adding AS it works. In your doc you only added VNI but look like it is required to add BGP AS. Or may be your doc is little older and new code required AS.</div><div><br></div><div>vagrant@rack-1-host-1:~$ sudo ovn-nbctl set logical_switch_port c32dcd90-7820-44bd-894f-416e44b36aa0 external_ids:"neutron_bgpvpn\:as"=64999<br>vagrant@rack-1-host-1:~$ sudo ovn-nbctl set logical_switch_port f55c1d1e-4b5b-4d8c-b922-3ad4a9700c81 external_ids:"neutron_bgpvpn\:as"=64999<br></div><div><br></div><div>Now i can see it created VRF and exposed VM tenant ip in lo-2001 </div><div><br></div><div>73: vrf-2001: <NOARP,MASTER,UP,LOWER_UP> mtu 65575 qdisc noqueue state UP group default qlen 1000<br> link/ether b2:29:65:3b:ac:db brd ff:ff:ff:ff:ff:ff<br>74: br-2001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf-2001 state UP group default qlen 1000<br> link/ether d6:0d:61:2d:b7:29 brd ff:ff:ff:ff:ff:ff<br> inet6 fe80::6830:47ff:febc:b10b/64 scope link<br> valid_lft forever preferred_lft forever<br>75: vxlan-2001: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-2001 state UNKNOWN group default qlen 1000<br> link/ether d6:0d:61:2d:b7:29 brd ff:ff:ff:ff:ff:ff<br> inet6 fe80::d40d:61ff:fe2d:b729/64 scope link<br> valid_lft forever preferred_lft forever<br>76: lo-2001: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue master vrf-2001 state UNKNOWN group default qlen 1000<br> link/ether fe:a1:a0:87:76:7c brd ff:ff:ff:ff:ff:ff<br> inet <a href="http://10.0.0.83/32" target="_blank">10.0.0.83/32</a> scope global lo-2001<br> valid_lft forever preferred_lft forever<br> inet6 fe80::fca1:a0ff:fe87:767c/64 scope link<br></div><div><br></div><div><br></div><div>I am continuing doing testing and seeing if I hit any other bug.. </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 4, 2022 at 11:58 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Luis,<div><br></div><div>I am following your doc, tell me if that doc is outdated or not <a href="https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/" target="_blank">https://ltomasbo.wordpress.com/2021/06/25/openstack-networking-with-evpn/</a><br><div><br></div><div>I have upgraded pyroute2 to 0.7.2 but still getting the same error, if you look at carefully following logs you will see an agent saying I can't find VNI but it's there, i can see in ovn-nbctl list logical_switch_port. </div><div><br></div><div>pyroute2 0.7.2<br>pyroute2.core 0.6.13<br>pyroute2.ethtool 0.6.13<br>pyroute2.ipdb 0.6.13<br>pyroute2.ipset 0.6.13<br>pyroute2.ndb 0.6.13<br>pyroute2.nftables 0.6.13<br>pyroute2.nslink 0.6.13<br></div><div><br></div><div><br></div><div>ovn-bgp-agent logs</div><div><br></div><div>2022-08-04 15:52:52.898 396714 DEBUG ovn_bgp_agent.drivers.openstack.utils.ovn [-] Either "neutron_bgpvpn:vni" or "neutron_bgpvpn:as" were not found or have an invalid value in the port f55c1d1e-4b5b-4d8c-b922-3ad4a9700c81 external_ids {'neutron:cidrs': '<a href="http://172.16.1.132/24" target="_blank">172.16.1.132/24</a>', 'neutron:device_id': '43b2c756-d92c-4fe5-b17b-32d5ab9b1f37', 'neutron:device_owner': 'network:router_gateway', 'neutron:network_name': 'neutron-0d82e6b0-bf9f-484d-85bd-0ba38aab508d', 'neutron:port_name': '', 'neutron:project_id': '', 'neutron:revision_number': '4', 'neutron:security_group_ids': '', 'neutron_bgpvpn:vni': '2001'} get_evpn_info /usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/utils/ovn.py:251<br><br></div><div>2022-08-04 15:52:52.899 396714 DEBUG ovn_bgp_agent.drivers.openstack.ovn_evpn_driver [-] No EVPN information for CR-LRP Port with IPs ['<a href="http://172.16.1.132/24'" target="_blank">172.16.1.132/24'</a>]. Not exposing it. _expose_ip /usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/ovn_evpn_driver.py:220<br></div><div><br></div><div><br></div><div>In OVN i can see vni number </div><div><br></div><div>_uuid : 3473d0ce-1348-4423-a2dc-6d4df4c06f74<br>addresses : [router]<br>dhcpv4_options : []<br>dhcpv6_options : []<br>dynamic_addresses : []<br>enabled : true<br>external_ids : {"neutron:cidrs"="<a href="http://172.16.1.132/24" target="_blank">172.16.1.132/24</a>", "neutron:device_id"="43b2c756-d92c-4fe5-b17b-32d5ab9b1f37", "neutron:device_owner"="network:router_gateway", "neutron:network_name"=neutron-0d82e6b0-bf9f-484d-85bd-0ba38aab508d, "neutron:port_name"="", "neutron:project_id"="", "neutron:revision_number"="4", "neutron:security_group_ids"="", "neutron_bgpvpn:vni"="2001"}<br>ha_chassis_group : []<br>name : "f55c1d1e-4b5b-4d8c-b922-3ad4a9700c81"<br>options : {exclude-lb-vips-from-garp="true", nat-addresses=router, router-port=lrp-f55c1d1e-4b5b-4d8c-b922-3ad4a9700c81}<br>parent_name : []<br>port_security : []<br>tag : []<br>tag_request : []<br>type : router<br>up : true<br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 4, 2022 at 11:39 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi Luis,<div><br></div><div>This is what I have installed on the compute and controller nodes. </div><div><br></div><div>pyroute2 0.6.13<br>pyroute2.core 0.6.13<br>pyroute2.ethtool 0.6.13<br>pyroute2.ipdb 0.6.13<br>pyroute2.ipset 0.6.13<br>pyroute2.ndb 0.6.13<br>pyroute2.nftables 0.6.13<br>pyroute2.nslink 0.6.13<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Aug 3, 2022 at 10:10 AM Luis Tomas Bolivar <<a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">What version of pyroute2 are you using? Maybe it is related to some bug in there in an old version (I hit quite a few). You can try to upgrade that. Also, as it is on the resync, you can also disable that by setting a very long time in there.<div><br><br>On Wednesday, August 3, 2022, Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Any thoughts Luis. Thanks <br><br><div dir="ltr">Sent from my iPhone</div><div dir="ltr"><br><blockquote type="cite">On Jul 27, 2022, at 11:53 AM, Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr">Luis,<div><br></div><div>If you look at this logs - <a href="https://paste.opendev.org/show/buRbY415guvHFUtSapFK/" target="_blank">https://paste.opendev.org/show/buRbY415guvHFUtSapFK/</a> </div><div><br></div><div><font face="arial, sans-serif">I am able to expose the tenant ip when I set "<span style="color:rgb(0,0,0);font-size:13px">expose_tenant_networks=True" in the ovn-bgp-agent.conf file. But interestingly it exposes ip for the first time but when i delete vm and re-create new vm then its throws following error which you can see full track in above link </span></font></div><div><font face="arial, sans-serif"><span style="color:rgb(0,0,0);font-size:13px"><br></span></font></div><div><pre style="line-height:16.25px;margin-top:0px;margin-bottom:0px;padding:5px 0px;font-family:"Bitstream Vera Sans Mono",monospace;font-size:13px;color:rgb(0,0,0)">ERROR oslo_service.periodic_task [-] Error during BGPAgent.sync: KeyError: 'object does not exists'</pre></div><div><font face="arial, sans-serif" color="#000000"><br></font></div><div><font face="arial, sans-serif" color="#000000">I have just 1x controller and 1x compute machine at present. As you said something is broken when setting up "</font><span style="color:rgb(0,0,0);font-family:arial,sans-serif;font-size:13px">expose_tenant_networks=True" Let me know if you need more info or logs etc. i will continue to poke and see if anything else I can find. </span><font face="arial, sans-serif" color="#000000"><br></font><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 27, 2022 at 10:35 AM Luis Tomas Bolivar <<a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Satish,sorry for the delay in replies, I'm on vacations and have limited connectivity.</div><div><br></div><div>See soon comments/replies inline<br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 27, 2022 at 4:58 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Luis,<div><br></div><div>Just checking incase you missed my last email. If you are on vacation then ignore it :) Thank you. <br><div><br></div><div><br></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Jul 23, 2022 at 12:27 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Hi Luis,</div><div><br></div><div>As you suggested, checkout 5 commits back to avoid the Load_Balancer issue that made good progress. </div></div></div></blockquote></div></blockquote><div><br></div><div>Great to hear!</div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br></div><div>But now i am seeing very odd behavior where floating IP can get exposed to BGP but when i am trying to expose VM Tenant IP then I get a strange error. Here is the full logs output; <a href="https://paste.opendev.org/show/buRbY415guvHFUtSapFK/" target="_blank">https://paste.opendev.org/show/buRbY415guvHFUtSapFK/</a> </div></div></div></blockquote></div></blockquote><div><br></div><div>I was using the evpn mode for the tenant networks, so it has been a while since I tested the vm tenant ip with plain bgp. Perhaps we broke it with some of the new features/reshapes.</div><div><br></div><div>Regarding the error, two things:</div><div>- Do you have several hosts or just one? The VM IP should be exposed in the host holding the ovn cr-lrp port (ovn router gateway port connecting the router to the provider network).</div><div>- The error there seems to be related to the re-sync task. I've hit some issues with pyroute2 in the past, please be sure to use a recent enough version (minimum supported is 0.6.4)</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 22, 2022 at 8:55 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Hi Luis,<div><br></div><div>Thank you for reply,</div><div><br></div><div>Let me tell you that your blog is wonderful. I have used your method to install frr without any docker container. </div><div><br></div><div>What is the workaround here? Can I tell ovn-bgp-agent to not look for container of frr?</div></div></blockquote></div></div></blockquote></div></blockquote><div><br></div><div>You just need to point where the frr socket is on the host.</div><div><br></div><div> <br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div><br></div><div>I have notice one more thing that it’s trying to run “copy /tmp/blah running-config” but that command isn’t supported. </div><div><br></div><div>I have tried to run copy command manually on vtysh shell to see what options are available for copy command but there is only one command available with copy which is “copy running-config startup-config” do you think I have wrong version of frr running? I’m running 7.2. </div></div></blockquote></div></div></blockquote></div></blockquote><div><br></div><div>I think the minimum version I tried was either 7.4 or 7.5. <br></div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div><br></div><div>Please let me know if any workaround here. Thank you </div></div></blockquote></div></div></blockquote></div></blockquote><div><br></div><div>No easy workaround for this, as it is relying on that way to reconfigure the frr.conf <br></div><div><br></div><div>Cheers,</div><div>Luis<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div><br><div dir="ltr">Sent from my iPhone</div><div dir="ltr"><br><blockquote type="cite">On Jul 22, 2022, at 2:41 AM, Luis Tomas Bolivar <<a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a>> wrote:<br><br></blockquote></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div>Hi Satish,</div><div><br></div><div>The one to use should be <a href="https://opendev.org/x/ovn-bgp-agent" target="_blank">https://opendev.org/x/ovn-bgp-agent</a>. The one on my personal github repo was the initial PoC for it. But the opendev one is the upstream effort to develop it, and is the one being maintained/updated.</div><div><br></div><div>Looking at your second logs, it seems you are missing FRR (and its shell, vtysh) in the node.</div><div><br></div><div>Actually, thinking about this:</div><div>"Unexpected error while running command.<br>Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config"</div><div><br></div><div>The ovn-bgp-agent has been developed with "deploying on containers" in mind, meaning it is assuming there is a frr container running, and the container running the agent is trying to connect to the same socket so that it can run the vtysh commands. Perhaps in your case the frr socket is in a different location than /run/frr/<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 22, 2022 at 6:27 AM Satish Patel <<a href="mailto:satish.txt@gmail.com" target="_blank">satish.txt@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Folks,<br><div><br></div><div>I am trying to create lab of of ovn-bgp-agent using this blog <a href="https://ltomasbo.wordpress.com/2021/02/04/ovn-bgp-agent-testing-setup/" target="_blank">https://ltomasbo.wordpress.com/2021/02/04/ovn-bgp-agent-testing-setup/</a> </div><div><br></div><div>So far everything went well but I'm stuck at the bgp-agent installation and I encounter following error when running bgp-agent. Any suggestions? </div><div><br></div><div>root@rack-1-host-2:/home/vagrant/bgp-agent# bgp-agent<br>2022-07-22 04:02:39.123 111551 INFO bgp_agent.config [-] Logging enabled!<br>2022-07-22 04:02:39.475 111551 CRITICAL bgp-agent [-] Unhandled error: AssertionError<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent Traceback (most recent call last):<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/bin/bgp-agent", line 10, in <module><br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent sys.exit(start())<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/agent.py", line 76, in start<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent bgp_agent_launcher = service.launch(config.CONF, BGPAgent())<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/agent.py", line 44, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent self.agent_driver = driver_api.AgentDriverBase.get_instance(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/driver_api.py", line 25, in get_instance<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent agent_driver = stevedore_driver.DriverManager(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/driver.py", line 54, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(DriverManager, self).__init__(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/named.py", line 78, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent extensions = self._load_plugins(invoke_on_load,<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/extension.py", line 221, in _load_plugins<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent ext = self._load_one_plugin(ep,<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/named.py", line 156, in _load_one_plugin<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent return super(NamedExtensionManager, self)._load_one_plugin(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/stevedore/extension.py", line 257, in _load_one_plugin<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent obj = plugin(*invoke_args, **invoke_kwds)<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/ovn_bgp_driver.py", line 64, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent self._sb_idl = ovn.OvnSbIdl(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/utils/ovn.py", line 62, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(OvnSbIdl, self).__init__(<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/bgp_agent/platform/osp/utils/ovn.py", line 31, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent super(OvnIdl, self).__init__(remote, schema)<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 283, in __init__<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent schema = schema_helper.get_idl_schema()<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 2323, in get_idl_schema<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent self._keep_table_columns(schema, table, columns))<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent File "/usr/local/lib/python3.8/dist-packages/ovs/db/idl.py", line 2330, in _keep_table_columns<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent assert table_name in schema.tables<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent AssertionError<br>2022-07-22 04:02:39.475 111551 ERROR bgp-agent<br></div><div><br></div><div><br></div><div><br></div><div><br></div><div>After googling I found one more agent at <a href="https://opendev.org/x/ovn-bgp-agent" target="_blank">https://opendev.org/x/ovn-bgp-agent</a> and its also throwing an error. Which agent should I be using?</div><div><br></div><div>root@rack-1-host-2:~# ovn-bgp-agent<br>2022-07-22 04:04:36.780 111761 INFO ovn_bgp_agent.config [-] Logging enabled!<br>2022-07-22 04:04:37.247 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopped<br>2022-07-22 04:04:37.248 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' starting<br>2022-07-22 04:04:37.248 111761 INFO ovn_bgp_agent.drivers.openstack.utils.frr [-] Add VRF leak for VRF ovn-bgp-vrf on router bgp 64999<br>2022-07-22 04:04:37.248 111761 INFO oslo.privsep.daemon [-] Running privsep helper: ['sudo', 'privsep-helper', '--privsep_context', 'ovn_bgp_agent.privileged.vtysh_cmd', '--privsep_sock_path', '/tmp/tmp4cie9eiz/privsep.sock']<br>2022-07-22 04:04:37.687 111761 INFO oslo.privsep.daemon [-] Spawned new privsep daemon via rootwrap<br>2022-07-22 04:04:37.598 111769 INFO oslo.privsep.daemon [-] privsep daemon starting<br>2022-07-22 04:04:37.613 111769 INFO oslo.privsep.daemon [-] privsep process running with uid/gid: 0/0<br>2022-07-22 04:04:37.617 111769 INFO oslo.privsep.daemon [-] privsep process running with capabilities (eff/prm/inh): CAP_NET_ADMIN|CAP_SYS_ADMIN/CAP_NET_ADMIN|CAP_SYS_ADMIN/none<br>2022-07-22 04:04:37.617 111769 INFO oslo.privsep.daemon [-] privsep daemon running as pid 111769<br>2022-07-22 04:04:37.987 111769 ERROR ovn_bgp_agent.privileged.vtysh [-] Unable to execute vtysh with ['/usr/bin/vtysh', '--vty_socket', '/run/frr/', '-c', 'copy /tmp/tmpiz5s_wvs running-config']. Exception: Unexpected error while running command.<br>Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config<br>Exit code: 1<br>Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n'<br>Stderr: ''<br>Traceback (most recent call last):<br> File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/privileged/vtysh.py", line 30, in run_vtysh_config<br> return processutils.execute(*full_args)<br> File "/usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py", line 438, in execute<br> raise ProcessExecutionError(exit_code=_returncode,<br>oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.<br>Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config<br>Exit code: 1<br>Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n'<br>Stderr: ''<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service [-] Error starting thread.: oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.<br>Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config<br>Exit code: 1<br>Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n'<br>Stderr: ''<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Traceback (most recent call last):<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_service/service.py", line 806, in run_service<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service service.start()<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/agent.py", line 50, in start<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service self.agent_driver.start()<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/ovn_bgp_driver.py", line 73, in start<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service frr.vrf_leak(constants.OVN_BGP_VRF, CONF.bgp_AS, CONF.bgp_router_id)<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/utils/frr.py", line 110, in vrf_leak<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service _run_vtysh_config_with_tempfile(vrf_config)<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/ovn_bgp_agent/drivers/openstack/utils/frr.py", line 93, in _run_vtysh_config_with_tempfile<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service ovn_bgp_agent.privileged.vtysh.run_vtysh_config(<a href="http://f.name" target="_blank">f.name</a>)<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/priv_context.py", line 271, in _wrap<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service return self.channel.remote_call(name, args, kwargs,<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service File "/usr/local/lib/python3.8/dist-packages/oslo_privsep/daemon.py", line 215, in remote_call<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service raise exc_type(*result[2])<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service oslo_concurrency.processutils.ProcessExecutionError: Unexpected error while running command.<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Command: /usr/bin/vtysh --vty_socket /run/frr/ -c copy /tmp/tmpiz5s_wvs running-config<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Exit code: 1<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Stdout: '% Unknown command: copy /tmp/tmpiz5s_wvs running-config\n'<br></div><div>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service Stderr: ''<br>2022-07-22 04:04:37.990 111761 ERROR oslo_service.service<br>2022-07-22 04:04:37.993 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopping<br>2022-07-22 04:04:37.994 111761 INFO ovn_bgp_agent.agent [-] Service 'BGPAgent' stopped<br></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div></div></div>
</div></blockquote></div></div></blockquote></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div></div></div>
</blockquote></div>
</div></blockquote></div></blockquote></div><br><br>-- <br><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div><br>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</blockquote></div>
</div></blockquote></div></div></blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr"><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div></div></div>
</blockquote></div></div>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>LUIS TOMÁS BOLÍVAR<br>Principal Software Engineer<br>Red Hat<br>Madrid, Spain<br><a href="mailto:ltomasbo@redhat.com" target="_blank">ltomasbo@redhat.com</a> <br> <br></div></div></div></div>