just adding bens back incase they are not on the list. i slectecte the wrogn reply type before On 03/09/2025 16:07, Sean Mooney wrote:
On 03/09/2025 15:10, Nathan Harper wrote:
Hi all,
We have been looking at building some routed provider networks, following this documentation: https://docs.openstack.org/neutron/latest/admin/config-routed-networks.html
In this scenario we have 4 racks, and have defined physnets for each rack and assigned SRIOV interfaces for each. We then have created a multisegment network, with a segment associated with each physnet. We get the expected resource provider in placement containing only these hypervisors.
When scheduling instances onto this network, the allocation candidates are any hypervisors in racks 1-4 (openstack filters the hypervisors using the aggregates for each segment that neutron creates). However, during instance build the pci device request sent to nova-compute always contains the physnet of the same segment.
Debugging the builds, we ended up here: https://opendev.org/openstack/nova/src/branch/master/nova/network/neutron.py..., with:
# TODO(vladikr): Additional work will be required to handle the
# case of multiple vlan segments associated with different
# physical networks.
Which originates from this commit: https://opendev.org/openstack/nova/commit/b9d9d96a407db5a2adde3aed81e61cc958...
This suggests that despite the documentation describing using multiple VLAN backed segments in this fashion, this has never worked?
That is correct. nova has never supported the multiple phsent exteion that was added to neutron in general
https://github.com/openstack/neutron-lib/blob/master/neutron_lib/api/definit...
you can have multiple network segments on the same phsenet but when routed provider networks was first designed there was an intention to have a second away to associate hosts with segments that did not depend on phsynets however that was never implemented. there was meant to be a way to associate host with segments directly via api or config that did not use phsynets to do that mapping.
Or are we missing something? Has anyone successfully used routed provider networks?
the wya that rotehed provider networks are typiclaly used is that you do not have a singel network that spans phsnets you can a 1:1 mapping between phsnet and segment and create seperate networks for each physnet
sriov has extra complications beause nova normally does tno have any awareness fo phsynets at all but for sriov you have to declare a single phsynet
for them in nova pci devspec.
a phsynet is intended to be effectively an l2 brodcast domain which is more or less what a sgment is as well.
technically the requirements for a neutron phsyent is stricter in its requirement of l2 isolation between phsnets then the isolation between segments.
--
Regards,
Nathan Harper
Principal Engineer – Cloud Development
Platform Engineering
nathanh@graphcore.ai <mailto:nathanh@graphcore.ai>
www.graphcore.ai → <http://www.graphcore.ai/>
** We have updated our privacy policy, which contains important information about how we collect and process your personal data. To read the policy, please click here <http://www.graphcore.ai/privacy> **
This email and its attachments are intended solely for the addressed recipients and may contain confidential or legally privileged information. If you are not the intended recipient you must not copy, distribute or disseminate this email in any way; to do so may be unlawful.
Any personal data/special category personal data herein are processed in accordance with UK data protection legislation. All associated feasible security measures are in place. Further details are available from the Privacy Notice on the website and/or from the Company.
Graphcore Limited (registered in England and Wales with registration number 10185006) is registered at, 1 Maple Road, Bramhall, Stockport, Cheshire, UK, SK7 2DH. This message was scanned for viruses upon transmission. However Graphcore accepts no liability for any such transmission.