Open Stack

On Fri, 2020-06-12 at 10:23 +0000, Wang, Xin-ran wrote:
> Hi all,
> 
> I prefer that physnet related stuff still managed by neutron, because it is a notion of neutron. If we let Cyborg
> update this traits to Placement, what will we do if Neutron enabled bandwidth feature and how we know whether this
> feature is enabled or not. 
> 
> Can we just let Neutron always report physnet traits. I am not very familiar with neutron, is there any gap?
> 
> Otherwise, if Cyborg do need report this to placement, my proposal is:
> 
> Neutron will provide an interface which allow Cyborg to get physnet trait/RP, if this feature is not configured, it
> will return 404, then Cyborg will know that neutron did not configured bandwidth feature, and Cyborg can report all by
> itself. If Neutron returns something meaningful, Cyborg should use the same RP and update other traits on this RP.
neutron cant provide physnet traits as it does not nessialary know the interface on the hsot exists.
physnets are specified by the operatator in the configfile for each network backend.

typically thsi is done via the bridge_mappings
https://github.com/openstack/neutron/blob/master/neutron/conf/plugins/ml2/drivers/ovs_conf.py#L51-L65
although for the sriov nic agent it uses the physical_device_mappings config option instead.
https://github.com/openstack/neutron/blob/master/neutron/conf/plugins/ml2/drivers/mech_sriov/agent_common.py#L25-L34

this infor is reported  to the neutron server via the agent report in some case but in generall this si not avaiable at
the api level.

unless the operator adds all fo the cyborg fpgas to the physical_device_mappings of the sriov nic agent on the
host by its netdev name neutron will not know what physnet any of the devices are conencted too.

realistically the cleanest way to managen this without depending on in progress featrues is to have a similar config
option in cyborg that is used by the driver to decalre the physnet the smartnic is attached too.
if we had a nameing convention for thr RPs such as <hostname>_<pci_address> e.g. host-1_0000:00:1f.6 then we could
perhaps use the new provior.yaml feature to do this instead. that would also allow neutron nova and cyborg to agree on
the RP name.

that does have some proablem however as really we woudl want cybrog device to be reported under nova created NUMA
resouce providers where as neutron would want them to be reported under the agent resouce provider and cyborg might want
a different topology. so really we need a way to the same RP to exist in multiple locations in the tree.
e.g. some form of alias or symlink capablity so that each service an look at there own view but only have once instance
of the resouces.

since we dont have that i think for step one we have to take a different approch.
if we assume that cyborg will be the only thing that creates the RPs for smartnics, and we agree on a nameing scheme
then we can use thenew  provider.yaml we are adding to nova to add the physnet traits. The provider.yaml could also be
used to create the  bandwith RPs that would normally be created by the sriov nic agent. provided the sriov nic agent
could configure the VF based soly on the pci_slot in the port binding profile and other info in the port withou needing
the device to be lisited in its own config file then you could support bandwith based shcduleing too without modifing
placement or nova.

at some point we will want to move the RP created by cyborg under the numa nodes created by nova but since that is a
work in progress we can cross that bridge at a later date. it would just require a reshape in the cyborg code to move
the RPs/Allocation and an update to how cyborg builds it provider tree. that is out of scope for now.


> 
> In this way, Cyborg and Neutron will use the same RP and keep the consistency. 
> 
> Thanks,
> Xin-Ran
> 
> -----Original Message-----
> From: Sean Mooney <smooney at redhat.com> 
> Sent: Thursday, June 11, 2020 9:44 PM
> To: Nadathur, Sundar <sundar.nadathur at intel.com>; openstack-discuss <openstack-discuss at lists.openstack.org>
> Subject: Re: [cyborg][neutron][nova] Networking support in Cyborg
> 
> On Thu, 2020-06-11 at 12:24 +0000, Nadathur, Sundar wrote:
> > Hi Sean,
> > 
> > > From: Sean Mooney <smooney at redhat.com>
> > > Sent: Thursday, June 11, 2020 4:31 AM
> > 
> >  
> > > On Thu, 2020-06-11 at 11:04 +0000, Nadathur, Sundar wrote:
> > > > [...]
> > > > * Ideally, the admin should be able to formulate the device 
> > > > profile in the same way, independent of whether it is a 
> > > > single-component or multi-component device. For that, the device 
> > > > profile must have a single resource group that includes the 
> > > > resource, traits and Cyborg
> > > 
> > > properties for both the accelerator and NIC. The device profile for 
> > > a Neutron port will presumably have only one request group. So, the 
> > > device profile would look something like this:
> > > > 
> > > >    { "name": "my-smartnic-dp",
> > > >      "groups": [{
> > > >              "resources:FPGA":  "1",
> > > >              "resources:CUSTOM_NIC_X": "1",
> > > >              "trait:CUSTOM_FPGA_REGION_ID_FOO": "required",
> > > >              "trait:CUSTOM_NIC_TRAIT_BAR": "required",
> > > >              "trait:CUSTOM_PHYSNET_VLAN3": "required",
> > > >             "accel:bitstream_id": "3AFE"
> > > >        }]
> > > >    }
> > > 
> > > having "trait:CUSTOM_PHYSNET_VLAN3": "required", in the device 
> > > profile means you have to create a seperate device profile with the 
> > > same details for each physnet and the user then need to fine the 
> > > profile that matches there neutron network's physnet which is also 
> > > problematic if they use the multiprovidernet extention.
> > > so we shoud keep the physnet seperate and have nova or neutorn 
> > > append that when we make the placment query.
> > 
> > True, we did discuss this at the PTG, and I agree. The physnet can be 
> > passed in from the command line during port creation.
> 
> that is not how that works.
> 
> when you create a neutron network with segmenation type vlan or flat it is automatically assigned a segmeantion_id and
> phsynet.
> As an admin you can chose both but as a tenant this is managed by neutron
> 
> ignoring the multiprovidernet for a second all vlan and flat network have 1 phyesnet and the port get a phsynet form
> the network it is created on.
> 
> the multiprovidernet extension allow a singlel neutron provider network to have multiple physnets but nova does not
> support that today.
> 
> so nova can get the physnet from the port/network/segment and incorporate that in the placment request but we cant
> pass it in during port creation.
> 
> in general tenants are not aware of physnets.
> 
> > 
> > > > [...]
> > > > * We discussed the physnet trait at the PTG. My suggestion is to 
> > > > keep Cyborg out of this, and out of networking in general, if possible.
> > > 
> > > well this feature is more or less the opisite of that intent but i 
> > > get that you dont want cyborg to have to confiure the networking atribute of the interface.
> > 
> > The admin could apply the trait to the right RP. Or, the OpenStack 
> > installer could automate this. That's similar in spirit to having the admin configure the physnet in PCI whitelist.
> 
> yes they could its not a partially good user experience as it quite tedious to do but yes it a viable option and
> likely sufficnet for the initial work. installer could automate it but having to do it manually would not be ideal.
> > 
> > Regards,
> > Sundar
> 
>