Open Stack

Hi,

There are two possibilities for the hybrid compute nodes

-          In the first case, a compute node has two NICs,  one SRIOV NIC & the other NIC for the VirtIO

-          In the 2nd case, Compute node has only one SRIOV NIC, where VFs are used for the VMs, either macvtap or direct assignment.  And the PF is used for the uplink to the linux bridge or OVS!!

My question to the team is whether we consider both of these deployments or not?

Thx,

Nrupal

From: Irena Berezovsky [mailto:irenab at mellanox.com]
Sent: Monday, January 27, 2014 1:01 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through SRIOV

Hi Robert,
Please see inline

From: Robert Li (baoli) [mailto:baoli at cisco.com]
Sent: Monday, January 27, 2014 10:29 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through SRIOV

Hi Irena,

I agree on your first comment.

see inline as well.

thanks,
Robert

On 1/27/14 10:54 AM, "Irena Berezovsky" <irenab at mellanox.com<mailto:irenab at mellanox.com>> wrote:

Hi Robert, all,
My comments inline

Regards,
Irena
From: Robert Li (baoli) [mailto:baoli at cisco.com]
Sent: Monday, January 27, 2014 5:05 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova][neutron] PCI pass-through SRIOV

Hi Folks,

In today's meeting, we discussed a scheduler issue for SRIOV. The basic requirement is for coexistence of the following compute nodes in a cloud:
      -- SRIOV only compute nodes
      -- non-SRIOV only compute nodes
      -- Compute nodes that can support both SRIOV and non-SRIOV ports. Lack of a proper name, let's call them compute nodes with hybrid NICs support, or simply hybrid compute nodes.

I'm not sure if it's practical in having hybrid compute nodes in a real cloud. But it may be useful in the lab to bench mark the performance differences between SRIOV, non-SRIOV, and coexistence of both.
[IrenaB]
I would like to clarify a bit on the requirements you stated below.
As I see it, the hybrid compute nodes actually can be preferred in the real cloud, since one can define VM with one vNIC attached via SR-IOV virtual function while the other via some vSwitch.
But it definitely make sense to land VM with 'virtio' vNICs only on the non-SRIOV compute node.

Maybe there should be some sort of preference order of suitable nodes in scheduler choice, based on vnic types required for the VM.

In a cloud that supports SRIOV in some of the compute nodes, a request such as:

     nova boot -flavor m1.large -image <image-uuid> --nic net-id=<net-uuid> vm

doesn't require a SRIOV port. However, it's possible for the nova scheduler to place it on a compute node that supports sriov port only. Since neutron plugin runs on the controller, port-create would succeed unless neutron knows the host doesn't support non-sriov port. But connectivity on the node would not be established since no agent is running on that host to establish such connectivity.
[IrenaB] I
Having ML2 plugin as neutron backend, will fail to bind the port, in no agent is running on the Host

[ROBERT] If a host supports SRIOV only, and there is an agent running on the host to support SRIOV, would binding succeed in ML2 plugin for the above 'nova boot' request?
[IrenaB] I think by adding the vnic_typem as we plan, Mechanism Driver will bind the port only if it supports vic_type and there is live agent on this host. So it should work

On a hybrid compute node, can we run multiple neutron L2 agents on a single host? It seems possible.

Irena brought up the idea of using host aggregate. This requires creation of a non-SRIOV host aggregate, and use that in the above 'nova boot' command. It should work.

The patch I had introduced a new constraint in the existing PCI passthrough filter.

The consensus seems to be having a better solution in a later release. And for now, people can either use host aggregate or resort to their own means.

Let's keep the discussion going on this.

Thanks,
Robert





On 1/24/14 4:50 PM, "Robert Li (baoli)" <baoli at cisco.com<mailto:baoli at cisco.com>> wrote:

Hi Folks,

Based on Thursday's discussion and a chat with Irena, I took the liberty to add a summary and discussion points for SRIOV on Monday and onwards. Check it out https://wiki.openstack.org/wiki/Meetings/Passthrough. Please feel free to update it. Let's try to finalize it next week. The goal is to determine the BPs that need to get approved, and to start coding.

thanks,
Robert


On 1/22/14 8:03 AM, "Robert Li (baoli)" <baoli at cisco.com<mailto:baoli at cisco.com>> wrote:

Sounds great! Let's do it on Thursday.

--Robert

On 1/22/14 12:46 AM, "Irena Berezovsky" <irenab at mellanox.com<mailto:irenab at mellanox.com>> wrote:

Hi Robert, all,
I would suggest not to delay the SR-IOV discussion to the next week.
Let's try to cover the SRIOV side and especially the nova-neutron interaction points and interfaces this Thursday.
Once we have the interaction points well defined, we can run parallel patches to cover the full story.

Thanks a lot,
Irena

From: Robert Li (baoli) [mailto:baoli at cisco.com]
Sent: Wednesday, January 22, 2014 12:02 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [nova][neutron] PCI passthrough SRIOV

Hi Folks,

As the debate about PCI flavor versus host aggregate goes on, I'd like to move forward with the SRIOV side of things in the same time. I know that tomorrow's IRC will be focusing on the BP review, and it may well continue into Thursday. Therefore, let's start discussing SRIOV side of things on Monday.

Basically, we need to work out the details on:
        -- regardless it's PCI flavor or host aggregate or something else, how to use it to specify a SRIOV port.
        -- new parameters for -nic
        -- new parameters for neutron net-create/neutron port-create
        -- interface between nova and neutron
        -- nova side of work
        -- neutron side of work

We should start coding ASAP.

Thanks,
Robert


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140127/7ee6c352/attachment-0001.html>