Open Stack

On 19 December 2013 12:21, Ian Wells <ijw.ubuntu at cack.org.uk> wrote:
>
> John:
>>
>> At a high level:
>>
>> Neutron:
>> * user wants to connect to a particular neutron network
>> * user wants a super-fast SRIOV connection
>>
>> Administration:
>> * needs to map PCI device to what neutron network the connect to
>>
>> The big question is:
>> * is this a specific SRIOV only (provider) network
>> * OR... are other non-SRIOV connections also made to that same network
>>
>> I feel we have to go for that latter. Imagine a network on VLAN 42,
>> you might want some SRIOV into that network, and some OVS connecting
>> into the same network. The user might have VMs connected using both
>> methods, so wants the same IP address ranges and same network id
>> spanning both.
>>
>>
>> If we go for that latter new either need:
>> * some kind of nic-flavor
>> ** boot ... -nic nic-id:"public-id:,nic-flavor:"10GBpassthrough"
>> ** but neutron could store nic-flavor, and pass it through to VIF
>> driver, and user says port-id
>> * OR add NIC config into the server flavor
>> ** extra spec to say, tell VIF driver it could use on of this list of
>> PCI devices: (list pci-flavors)
>> * OR do both
>>
>> I vote for nic-flavor only, because it matches the volume-type we have
>> with cinder.
>
>
> I think the issue there is that Nova is managing the supply of PCI devices
> (which is limited and limited on a per-machine basis).  Indisputably you
> need to select the NIC you want to use as a passthrough rather than a vnic
> device, so there's something in the --nic argument, but you have to answer
> two questions:
>
> - how many devices do you need (which is now not a flavor property but in
> the --nic list, which seems to me an odd place to be defining billable
> resources)
> - what happens when someone does nova interface-attach?

Agreed.

The --nic list specifies how many NICs.

I was suggesting adding a nic-flavor on each --nic spec to say if its
PCI passthrough vs virtual NIC.

> Cinder's an indirect parallel because the resources it's adding to the
> hypervisor are virtual and unlimited, I think, or am I missing something
> here?

I was more referring more to the different "volume-types" i.e. "fast
volume" or "normal volume".
And how that is similar to "virtual" vs "fast PCI passthough" vs "slow
PCI passthrough"

Local volumes probably have the same issues as PCI passthrough with
"finite resources".
But I am not sure we have a good solution for that yet.

Mostly, it seems right that Cinder and Neutron own the configuration
about the volume and network resources.

The VIF driver and volume drivers seem to have a similar sort of
relationship with Cinder and Neutron vs Nova.

Then the issues boils down to visibility into that data so we can
schedule efficiently, which is no easy problem.

>>
>> However, it does suggest that Nova should leave all the SRIOV work to
>> the VIF driver.
>> So the VIF driver, as activate by neutron, will understand which PCI
>> devices to passthrough.
>>
>> Similar to the plan with brick, we could have an oslo lib that helps
>> you attach SRIOV devices that could be used by the neturon VIF drivers
>> and the nova PCI passthrough code.
>
> I'm not clear that this is necessary.
>
> At the moment with vNICs, you pass through devices by having a co-operation
> between Neutron (which configures a way of attaching them to put them on a
> certain network) and the hypervisor specific code (which creates them in the
> instance and attaches them as instructed by Neutron).  Why would we not
> follow the same pattern with passthrough devices?  In this instance, neutron
> would tell nova that when it's plugging this device it should be a
> passthrough device, and pass any additional parameters like the VF encap,
> and Nova would do as instructed, then Neutron would reconfigure whatever
> parts of the network need to be reconfigured in concert with the
> hypervisor's settings to make the NIC a part of the specified network.

I agree, in general terms.

Firstly, do you agree the neutron network-id can be used for
passthrough and non-passthrough VIF connections? i.e. a neturon
network-id does not imply PCI-passthrough.

Secondly, we need to agree on the information flow around defining the
"flavor" of the NIC. i.e. virtual or passthroughFast or
passthroughNormal.

My gut feeling is that neutron port description should somehow define
this via a nic-flavor that maps to a group of pci-flavors.

But from a billing point of view, I like the idea of the server flavor
saying to the VIF plug code, by the way, for this server, please
support all the nics using devices in pciflavor:fastNic should that be
possible for the users given port configuration. But this is leaking
neutron/networking information into Nova, which seems bad.

John