[neutron][octavia] New deployment / suggestions for network
Hi, We have been deploying OpenStack for quite some time, using Kolla-Ansible, and typically choose DVR and OVS with Amphorae deployed by Octavia for load balancing. With the issues that DVR has with Octavia's Amphorae and Virtual IPs, with essentially non-functional automated fail-over, we have always wanted to move to OVN since it appears to be the popular approach now. I have also read that OVN appears to work properly with allowed-address-pairs correctly, whereas with DVR, OVS does not, and thus some of the issues with Amphorae Virtual IPs. However, OVN, from what I understand, has issues with, or doesn't support, VPNaaS, which we use extensively. Plus, it only supports Layer 4 load balancing, whereas with Amphorae, we get Layer 7 load balancing - also used extensively. I'm not sure, though - maybe OVN with Octavia still supports Amphorae if we need Layer 7 load balancing? Am I wrong regarding any of the comments above? What is the best back-end networking architecture that provides scalability (so, not VLANs), Layer 7 load balancing with Octavia, along with VPNaaS, in a brand new install with the latest version of OpenStack? Note that we used Midonet long long ago, and it seemed to have everything we wanted, but shortly after purchasing it, Midokura immediately decided to abandon support for OpenStack and went the Kubernetes route. Not sure if they still do this, but needless to say, Midonet isn't a valid solution unfortunately. Tungsten Fabric appears like an alternate solution to Midonet, but that project is sunsetting in 2024, so that's dead too. :( Thank you for any suggestions! Eric
Hi Eric, Greetings from a fellow Midonet refugee. On Wed, Oct 18, 2023, 9:53 PM Eric K. Miller <emiller@genesishosting.com> wrote:
Hi,
We have been deploying OpenStack for quite some time, using Kolla-Ansible, and typically choose DVR and OVS with Amphorae deployed by Octavia for load balancing.
With the issues that DVR has with Octavia's Amphorae and Virtual IPs, with essentially non-functional automated fail-over, we have always wanted to move to OVN since it appears to be the popular approach now.
Just FYI there's no reason you need to use DVR with OVS. You could run vanilla OVS with L3-HA and Octavia Amphorae work great and do proper failover. Is DVR a requirement? I don't have any issues with allowed-address-pairs and OVS but I've never tried to mix in DVR.
I have also read that OVN appears to work properly with
allowed-address-pairs correctly, whereas with DVR, OVS does not, and thus some of the issues with Amphorae Virtual IPs.
However, OVN, from what I understand, has issues with, or doesn't support, VPNaaS, which we use extensively. Plus, it only supports Layer 4 load balancing, whereas with Amphorae, we get Layer 7 load balancing - also used extensively. I'm not sure, though - maybe OVN with Octavia still supports Amphorae if we need Layer 7 load balancing?
Am I wrong regarding any of the comments above? What is the best back-end networking architecture that provides scalability (so, not VLANs), Layer 7 load balancing with Octavia, along with VPNaaS, in a brand new install with the latest version of OpenStack?
Using Amphorae with an OVN fabric should still work fine. They are VMs and don't really care what they attached to so long as Neutron responds to the appropriate requests. The OVN Octavia driver has the benefit of native load balancers without VMs, but you are correct that you'd lose your L7 functionality. Note that we used Midonet long long ago, and it seemed to have
everything we wanted, but shortly after purchasing it, Midokura immediately decided to abandon support for OpenStack and went the Kubernetes route. Not sure if they still do this, but needless to say, Midonet isn't a valid solution unfortunately. Tungsten Fabric appears like an alternate solution to Midonet, but that project is sunsetting in 2024, so that's dead too. :(
Thank you for any suggestions!
Eric
-Erik
Hi Erik, Thanks for the quick response!
Just FYI there's no reason you need to use DVR with OVS. You could run vanilla OVS with L3-HA and Octavia Amphorae work great and do proper failover. Is DVR a requirement? I don't have any issues with allowed-address-pairs and OVS but I've never tried to mix in DVR.
I guess I hadn't thought of not using DVR. The benefit seemed significant back in the day, and I thought it was required for VXLAN configurations, but maybe not? I guess that all traffic that is sent to a router would go through a centralized router instead, which, I think means that all floating IP traffic would flow through network nodes (where the centralized routers are located). I will review and report back what I find since it has been a while since I looked into this.
Using Amphorae with an OVN fabric should still work fine. They are VMs and don't really care what they attached to so long as Neutron responds to the appropriate requests. The OVN Octavia driver has the benefit of native load balancers without VMs, but you are correct that you'd lose your L7 functionality.
I suspected that it should, theoretically, but I didn't know if anyone had experience with this before we attempt it. Maybe three is a way to determine which load balancer type is chosen using Octavia, OVN or Amphorae. Eric
Hi, Not for all your problems perhaps but there is a massive patch for vpnaas to work with OVN: https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353 Please check, and leave comments if you can test it, would be really useful to have feedback from possible users. I hope next week PTG we can discuss this topic also to push the review of such changes. Best wishes Lajos Katona (lajoskatona) Eric K. Miller <emiller@genesishosting.com> ezt írta (időpont: 2023. okt. 19., Cs, 3:56):
Hi,
We have been deploying OpenStack for quite some time, using Kolla-Ansible, and typically choose DVR and OVS with Amphorae deployed by Octavia for load balancing.
With the issues that DVR has with Octavia's Amphorae and Virtual IPs, with essentially non-functional automated fail-over, we have always wanted to move to OVN since it appears to be the popular approach now.
I have also read that OVN appears to work properly with allowed-address-pairs correctly, whereas with DVR, OVS does not, and thus some of the issues with Amphorae Virtual IPs.
However, OVN, from what I understand, has issues with, or doesn't support, VPNaaS, which we use extensively. Plus, it only supports Layer 4 load balancing, whereas with Amphorae, we get Layer 7 load balancing - also used extensively. I'm not sure, though - maybe OVN with Octavia still supports Amphorae if we need Layer 7 load balancing?
Am I wrong regarding any of the comments above? What is the best back-end networking architecture that provides scalability (so, not VLANs), Layer 7 load balancing with Octavia, along with VPNaaS, in a brand new install with the latest version of OpenStack?
Note that we used Midonet long long ago, and it seemed to have everything we wanted, but shortly after purchasing it, Midokura immediately decided to abandon support for OpenStack and went the Kubernetes route. Not sure if they still do this, but needless to say, Midonet isn't a valid solution unfortunately. Tungsten Fabric appears like an alternate solution to Midonet, but that project is sunsetting in 2024, so that's dead too. :(
Thank you for any suggestions!
Eric
Not for all your problems perhaps but there is a massive patch for vpnaas to work with OVN: https://review.opendev.org/c/openstack/neutron-vpnaas/+/765353
Thanks Lajos! We are planning a new test cluster soon, so will try this patch and report back. Might be a good 3 to 4 weeks from now, though. Eric
Hi Eric, Yes, DVR in neutron has bugs that have not been fixed relating to ARP and protocols such as VRRP (which Octavia Amphora use). Moving to OVN is a good solution to get rid of the DVR issues. Also note, Octavia Amphora based load balancers can still be used with the OVN ML2 in neutron. You do not need to use the neutron OVN provider driver in Octavia. Michael On Wed, Oct 18, 2023 at 6:55 PM Eric K. Miller <emiller@genesishosting.com> wrote:
Hi,
We have been deploying OpenStack for quite some time, using Kolla-Ansible, and typically choose DVR and OVS with Amphorae deployed by Octavia for load balancing.
With the issues that DVR has with Octavia's Amphorae and Virtual IPs, with essentially non-functional automated fail-over, we have always wanted to move to OVN since it appears to be the popular approach now.
I have also read that OVN appears to work properly with allowed-address-pairs correctly, whereas with DVR, OVS does not, and thus some of the issues with Amphorae Virtual IPs.
However, OVN, from what I understand, has issues with, or doesn't support, VPNaaS, which we use extensively. Plus, it only supports Layer 4 load balancing, whereas with Amphorae, we get Layer 7 load balancing - also used extensively. I'm not sure, though - maybe OVN with Octavia still supports Amphorae if we need Layer 7 load balancing?
Am I wrong regarding any of the comments above? What is the best back-end networking architecture that provides scalability (so, not VLANs), Layer 7 load balancing with Octavia, along with VPNaaS, in a brand new install with the latest version of OpenStack?
Note that we used Midonet long long ago, and it seemed to have everything we wanted, but shortly after purchasing it, Midokura immediately decided to abandon support for OpenStack and went the Kubernetes route. Not sure if they still do this, but needless to say, Midonet isn't a valid solution unfortunately. Tungsten Fabric appears like an alternate solution to Midonet, but that project is sunsetting in 2024, so that's dead too. :(
Thank you for any suggestions!
Eric
Yes, DVR in neutron has bugs that have not been fixed relating to ARP and protocols such as VRRP (which Octavia Amphora use).
Moving to OVN is a good solution to get rid of the DVR issues.
Also note, Octavia Amphora based load balancers can still be used with the OVN ML2 in neutron. You do not need to use the neutron OVN provider driver in Octavia.
Thanks Michael! Much appreciated for the feedback. I did see that there is a "type" parameter when creating load balancers, and the default appears to be amphora, even if OVN is used. Using amphorae is actually a bit more desired, even for more basic Layer 4 balancing, since the hashes available are more flexible than what OVN provides. So good to know what we can basically disable the use of OVN with Octavia. Thanks again! Eric
participants (4)
-
Eric K. Miller
-
Erik McCormick
-
Lajos Katona
-
Michael Johnson