Open Stack

Hi Sean and Lajos,

Thank you so much for your quick response, good suggestions and feedbacks!

@Sean Mooney
> if we want to supprot cyborg/smartnic integration we should add a new
> device-profile extention that intoduces the ablity
> for a non admin user to specify a cyborg device profile name as a new
> attibute on the port.

+1,Agree. Cyborg likes this suggestion! This will be more clear that this field is for device profile usage.
The reason why we were firstly thinking of using binding:profile is that this is a way with the smallest number of changes possible in both nova and neutron.But thinking of the non-admin issue, the  violation of the one way comunicaiton of binding:profile, and the possible security risk of breaking nova(which we surely don't want to do that), we surely prefer giving up binding:profile and  finding a better place to put the new device-profile extention.

> the neutron server could then either retirve the request groups form
> cyborg and pass them as part of the port resouce
> request using the mechanium added for minium bandwidth or it can leave
> that to nova to manage.
>
> i would kind of prefer neutron to do this but both could work.


Yes, neutron server can also do that,but given the fact that we already landed the code of retriving request groups form cyborg in nova, can we reuse this process in nova and add new process in create_resource_requests to create accelerator resource request from port info? 

I would be very appreciated if this change can land in nova, as I see the advantages are:
1) This keeps accelerator request groups things handled in on place, which makes integration clear and simple, Nova controls the main manage things,the neutron handles network backends integration,and cyborg involves accelerator management.
2) Another good thing: this will dispels Lajos's concerns on port-resource-request!
3) As presented in the proposal(page 3 and 5 of the slide)[0], please don't worry! This will be a tiny change in nova. Cyborg will be very appreciated if this change can land in nova, for it saves much effort in cyborg-neutron integration.


@Lajos Katona:
> Port-resource-request (see:https://docs.openstack.org/api-ref/network/v2/index.html#port-resource-request)
> is a read-only (and admin-only) field of ports, which is filled
> based on the agent heartbeats. So now there is now polling of agents or
> similar. Adding extra "overload" to this mechanism, like polling cyborg or
> similar looks something out of the original design for me, not to speak about the performance issues to add
> 
>    - API requests towards cyborg (or anything else) to every port GET
>    operation
>    - store cyborg related information in neutron db which was fetched from
>    cyborg (periodically I suppose) to make neutron able to fill
>    port-resource-request.


As mentioned above,if request group can land in nova, we don't need to concern API request towards cyborg and cyborg info merged to port-resource-request.
Another question just for curiosity.In my understanding(Please correct me if I'm worng.), I feel that neutron doesn't need to poll cyborg periodically if neutron fill port-resource-request, just fetch it once port request happens. because neutron can expect that the cyborg device_profile(provides resource request info for nova scheduler) don't change very often,it is the flavor of accelerator, and only admin can create/delete them.

[0]pre-PTG slides update: https://docs.qq.com/slide/DVkxSUlRnVGxnUFR3

Regards,
Yumeng




On Friday, May 29, 2020, 3:21:08 PM GMT+8, Lajos Katona <katonalala at gmail.com> wrote: 





Hi,

Port-resource-request (see: https://docs.openstack.org/api-ref/network/v2/index.html#port-resource-request ) is a read-only (and admin-only) field of ports, which is filled
based on the agent heartbeats. So now there is now polling of agents or similar. Adding extra "overload" to this mechanism, like polling cyborg or similar
looks something out of the original design for me, not to speak about the performance issues to add 
    * API requests towards cyborg (or anything else) to every port GET operation
    * store cyborg related information in neutron db which was fetched from cyborg (periodically I suppose) to make neutron able to fill port-resource-request.
Regards
Lajos

Sean Mooney <smooney at redhat.com> ezt írta (időpont: 2020. máj. 28., Cs, 16:13):
> On Thu, 2020-05-28 at 20:50 +0800, yumeng bao wrote:
>> 
>> Hi all,
>> 
>> 
>> In cyborg pre-PTG meeting conducted last week[0],shaohe from Intel introduced SmartNIC support integrations,and we've
>> reached some initial agreements: 
>> 
>> The workflow for a user to create a server with network acceleartor(accelerator is managed by Cyborg) is:
>> 
>>   1. create a port with accelerator request specified into binding_profile field
>>  NOTE: Putting the accelerator request(device_profile) into binding_profile is one possible solution implemented in
>> our POC.
> the binding profile field is not really intended for this.
> 
> https://github.com/openstack/neutron-lib/blob/master/neutron_lib/api/definitions/portbindings.py#L31-L34
> its intended to pass info from nova to neutron but not the other way around.
> it was orgininally introduced so that nova could pass info to the neutron plug in
> specificly the sriov pci address. it was not intended for two way comunicaiton to present infom form neutron
> to nova.
> 
> we kindo of broke that with the trusted vf feature but since that was intended to be admin only as its a security risk
> in a mulit tenant cloud its a slightl different case. 
> i think we should avoid using the binding profile for passing info form neutron to nova and keep it for its orginal
> use of passing info from the virt dirver to the network backend.
> 
> 
>>        Another possible solution,adding a new attribute to port object for cyborg specific use instead of using
>> binding_profile, is discussed in shanghai Summit[1].
>> This needs check with neutron team, which neutron team would suggest? 
> from a nova persepctive i would prefer if this was  a new extention.
> the binding profile is admin only by default so its not realy a good way to request features be enabled.
> you can use neutron rbac policies to alther that i belive but in genral i dont think we shoudl advocate for non admins
> to be able to modify the binding profile as they can break nova. e.g. by modifying the pci addres.
> if we want to supprot cyborg/smartnic integration we should add a new device-profile extention that intoduces the ablity
> for a non admin user to specify a cyborg device profile name as a new attibute on the port.
> 
> the neutron server could then either retirve the request groups form cyborg and pass them as part of the port resouce
> request using the mechanium added for minium bandwidth or it can leave that to nova to manage.
> 
> i would kind of prefer neutron to do this but both could work.
>> 
>>   2.create a server with the port created
>> 
>> Cyborg-nova-neutron integration workflow can be found on page 3 of the slide[2] presented in pre-PTG.
>> 
>> And we also record the introduction! Please find the pre-PTG meeting vedio record in [3] and [4], they are the same,
>> just for different region access.
>> 
>> 
>> [0]http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014987.html
>> [1]https://etherpad.opendev.org/p/Shanghai-Neutron-Cyborg-xproj
>> [2]pre-PTG slides:https://docs.qq.com/slide/DVm5Jakx5ZlJXY3lw
>> [3]pre-PTG vedio records in Youtube:https://www.youtube.com/watch?v=IN4haOK7sQg&feature=youtu.be
>> [4]pre-PTG vedio records in Youku:
>> http://v.youku.com/v_show/id_XNDY5MDA4NjM2NA==.html?x&sharefrom=iphone&sharekey=51459cbd599407990dd09940061b374d4
>> 
>> Regards，
>> Yumeng
>> 
> 
> 
>