On Tue, Feb 11, 2025 at 3:27 PM Thomas Goirand <zigo@debian.org> wrote:
Hi there!
In our setup, we offer to our customers, on top of floating IPS, access to an OpenStack internal network that contains public IPs. The idea is that they can, using this network, spawn VMs with a public IP directly on the NIC of their VMs.
The issue is, some customers also use this network for the VIP of their LBs. While this seems to work, actually, instead of reserving a single public IP address, it reserves 3 IPs. Not a big deal for us, as they are billed internally, though it's rather bad for our customers. On top of that, whenever one of the subnet of this network becomes full, it becomes impossible to failover amphoras spawned in this subnet. Each time one runs into ERROR, it's kind of a bad situation where we have no way to fix, except contacting the customer and ask them to fix their setup.
So we advise our customers to spawn a port first, use that port for the LB, and associate a floating IP address to the original port. This is described in our doc here:
https://docs.infomaniak.cloud/network/loadbalancer_usecase_simple_http/
However, as you may know, customers never read the docs. And therefore, we'd like to forbid the use of that shared OpenStack internal network for the VIP of LBs. But as much as possible, this isn't a feature of Octavia. Is it?
I'd like to implement the feature. What's the best way to do so? Would a new "forbidden_networks" directive would do, and then have a check at the API level? Or is there a better way using the API policy?
For the moment, Octavia doesn't support such a feature, I would take a look at the neutron RBAC to check if something can be achieved there. Octavia provides a 'valid_vip_networks' setting in the config file: https://docs.openstack.org/octavia/latest/configuration/configref.html#netwo... We could have a setting (deny_vip_networks) that does the opposite validation.
Also, wouldn't it be possible that Octavia just reuse the ports that it currently hold, rather than asking new ports for the failover? If so, any clue where the code that does that could be fixed? I'd love to have this fixed AND the forbid network feature too...
During a failover, the new amphora is created but the old amphora still exists and processes incoming requests until the new one is fully functional. So we need to have 2 ports at the same time. Reusing the existing port would lead to a long interruption of the dataplane.
Cheers,
Thomas Goirand (zigo)