Open Stack

I know that vif_type is binding_failed on a multinode setup and I also know
why it happens.

As for interface-attach I got it work for sriov ports and even verified it
works inside the instance. The trick was to specify profile with pci_slot
and pci_vendor_info during port create. In case any one else wants to do
this.

Thank you,
Ageeleshwar K

On Thu, Feb 5, 2015 at 12:19 PM, Irena Berezovsky <irenab.dev at gmail.com>
wrote:

> Hi Akilesh,
> Please see my responses inline.
> Hope this help,
>
> BR,
> Irena
>
> On Thu, Feb 5, 2015 at 6:14 AM, Akilesh K <akilesh1597 at gmail.com> wrote:
>
>> Hi Irena,
>>
>> Issue 1 - I agree. You are correct.
>>
>> Issue 2
>> The behavior you outlined
>> 1. When port is created with vnic_type=direct, the vif_type is
>> 'unbound'. The pci_vendor_info will be available during port update when
>> 'nova boot' command is invoked and PCI device is allocated.
>> This happens when the controller and compute are on the same host. Not
>> when they are on the different host. On a multiserver setup vif_type is set
>> to binging_failed during port create.
>>
>> This is strange, since port-create operation is pure neutron API call and
> it should not differ whether you are in the multiserver or all-in-one setup.
>
> Second is i am not doing nova boot. Instead I am doing nova
>> interface-attach. In this case the pci_vendor_info is not updated by anyone
>> but me. and pci_slot is also not populated.
>>
>> interface-attach is currently not supported for SR-IOV ports. There is a
> proposed blueprint to support this:
> https://review.openstack.org/#/c/139910/.
> So for now, the only option to provide PCI passthrough vNIC is according
> to what is described in the previously referenced wiki page: create neutron
> port with vnic_type= direct and then 'nova boot' with pre-created port.
>
> Do you still think this is correct ?
>
>>
>>
>>
>> On Wed, Feb 4, 2015 at 8:08 PM, Irena Berezovsky <irenab.dev at gmail.com>
>> wrote:
>>
>>> Hi Akilesh,
>>> please see inline
>>>
>>> On Wed, Feb 4, 2015 at 11:32 AM, Akilesh K <akilesh1597 at gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>> Issue 1:
>>>> I do not understand what you mean. I did specify the physical_network.
>>>> What I am trying to say is some physical networks exists only on the
>>>> compute node and not on the network node. We are unable to create a network
>>>> on those physnets. The work around was to fake their existance on the
>>>> network node too. Which I believe is the wrong way to do.
>>>>
>>>> Every physical network  should be defined at the Controller node,
>>> including range of segmentation ids (i.e. vlan ids) available for
>>> allocation.
>>> When virtual network is created, you should verify that it has
>>> associated  network type and segmentation id (assuming you are using
>>> provider network extension).
>>>
>>>>
>>>> Issue2:
>>>> I looked directly into the code after looking at the logs.
>>>>
>>>> 1. What neutron (sriov mech driver ) is doing is loading the default
>>>> list of 'supported_pci_vendor_devs' , then it picks up the
>>>> profile->pci_vendor_info from the port defenition we sent in the port
>>>> create request and checks if it is supported. If not it says
>>>> 'binding_failed'
>>>>
>>> When port is created with vnic_type=direct, the vif_type is 'unbound'.
>>> The pci_vendor_info will be available during port update when 'nova boot'
>>> command is invoked and PCI device is allocated.
>>>
>>>
>>>>
>>>> I am fine with this
>>>>
>>>> 2. Then when I attach the created port to a host nova's vif driver
>>>> (hv_veb) is looking for profile->pci_slot in the context of the port that
>>>> was supplied and fails to attach to the instance if it is not present.
>>>>
>>>> nova vif driver receives profile->pci_slot from neutron, but it was
>>> actually filed earlier by nova during port-update.
>>>
>>>> this is what I think should be done by neutron itself. neutron's sriov
>>>> mech driver should have updated the port with the pci_slot details when the
>>>> port got created. and this does happen on a single machine install. We need
>>>> to find why it does not happen on a multi node install, possibly because
>>>> the mech driver is not running on the host with sriov devices and fix it.
>>>>
>>>> I suggest to follow
>>> https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking instructions,
>>> this should work for you.
>>>
>>> I hope you guys can understand what I mean.
>>>
>>>>
>>>> Thank you,
>>>> Ageeleshwar K
>>>>
>>>>
>>>> On Wed, Feb 4, 2015 at 2:49 PM, Itzik Brown <itzikb at redhat.com> wrote:
>>>>
>>>>>  Hi,
>>>>>
>>>>> Issue 1;
>>>>> You must specify the physical networks.
>>>>> Please look at:
>>>>> https://wiki.openstack.org/wiki/SR-IOV-Passthrough-For-Networking
>>>>>
>>>>> Issue 2:
>>>>> AFAIK the agent is supported by only one vendor.
>>>>> Can you please look for errors in Neutron's log?
>>>>>
>>>>> Thanks,
>>>>> Itzik
>>>>>
>>>>> On 02/04/2015 09:12 AM, Akilesh K wrote:
>>>>>
>>>>>     Hi,
>>>>>  I found two issues with the way neutron behaves on a multi server
>>>>> install. I got it to work but I do not this this is the right way to do it.
>>>>> It might be a bug we might want to fix and for which I could volunteer.
>>>>>
>>>>>  Setup - Multiserver juno on ubuntu.
>>>>>
>>>>>  Machine 1 - Controller
>>>>>  All api servers , l3, dhcp and ovs agent
>>>>>
>>>>>  Machine 2 - Compute
>>>>>  nova compute, neutron-ovs-agent, neutron sriov agent.
>>>>>
>>>>>
>>>>>  Issue 1:
>>>>>
>>>>>  Controller node has physnets 'External', 'Internal' configured in ml2
>>>>>
>>>>>  Compute node has physnets 'Internal', 'Physnet1', 'Physnet2'
>>>>> configured in ml2
>>>>>
>>>>>  When I do neutron net-create --provider:physicalnetwork Physnet1, It
>>>>> complains that 'Physnet1' is not available.
>>>>>
>>>>>  Offcourse its not available on the controller but is available on the
>>>>> compute node and there is no way to tell neutron to host that network on
>>>>> compute node alone
>>>>>
>>>>>  Work around
>>>>> I had to include 'Physnet1' in the controller node also to get it to
>>>>> work, except that there is not bridge mapings for this physnet.
>>>>>
>>>>>
>>>>>  Issue 2:
>>>>>
>>>>>  This is related to sriov agent. This agent is configured only on the
>>>>> compute node as that node alone has supported devices.
>>>>>
>>>>>  When I do a port create --binding:vnic_type direct --binding:host_id
>>>>> <compute node> The port is created but with binding:vif_type:
>>>>> *'binding-failed'*.   and naturally I could not attach it to any
>>>>> instance.
>>>>>
>>>>>  I looked at the code and figured out that neutron api is expecting
>>>>> binding:profile also in the format
>>>>>  {"pci_slot": "0000:03:10.1", "pci_vendor_info": "8086:10ed"}
>>>>>
>>>>>  Is this how it should be. Because on a single machine install I did
>>>>> not have to do this. However on a multiserver I had to even give the pci
>>>>> address is the exact format to get it to work.
>>>>>
>>>>>  I have a serious feeling that this could be lot simpler if neutron
>>>>> could take care of finding the details in a smart way rather than relying
>>>>> on the administrator to find which device is available and configure it.
>>>>>
>>>>>
>>>>>  Note:
>>>>>  1. If I can get some expert advice I can fix both these.
>>>>>  2. I am not sure if this question should rather be sent to
>>>>> openstack-dev group. Let me know.
>>>>>
>>>>>
>>>>>  Thank you,
>>>>>  Ageeleshwar K
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>> Post to     : openstack at lists.openstack.org
>>>>> Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150205/6ddcbea3/attachment.html>