Open Stack

On 09/01/2017 04:42 AM, Rua, Philippe (Nokia - FI/Espoo) wrote:
> Will it be possible to include network bandwidth as a resource in Nova scheduling, for VM placement decision?

Yes.

See here for a related Neutron spec that mentions Placement: 
https://review.openstack.org/#/c/396297/7/specs/pike/strict-minimum-bandwidth-support.rst

> Context: in telecommunication applications, the network traffic is an important dimension of resource usage. For example, it is often important to distribute "bandwidth-greedy" VMs to different compute nodes. There were some earlier discussions on this topic, but I could not find a concrete outcome. [1][2][3]
> 
> After some reading, I wonder whether the Custom resource classes can provide a generic mechanism? [4][5][6]

No :) Custom resource classes are antithetical to generic/standard 
mechanisms.

We want to add two *standard* resource classes, one called 
NET_INGRESS_BYTES_SEC and another called NET_EGRESS_BYTES_SEC which 
would represent the total bandwidth in bytes per second the for 
corresponding traffic directions.

What would be the resource provider, though? There are at least two 
potential answers here:

1) A network interface controller on the compute host

In this case, the NIC on the host would be a child provider of the 
compute host resource provider. It would have an inventory record of 
resource class NET_INGRESS_BYTES_SEC with a total value representing the 
entire bandwidth of the host NIC. Instances would consume some amount of 
NET_INGRESS_BYTES_SEC corresponding to *either* the Nova flavor (if the 
resources:NET_INGRESS_BYTES_SEC extra-spec is set) *or* to the sum of 
consumed bandwidth amounts from the port profile of any ports specified 
when launching the instance (and thus would be part of the pci device 
request collection attached to the build request).

2) A "network slice" of a network interface controller on the compute host

In this case, assume that the NIC on the compute host has had its total 
bandwidth constrained via traffic control so that 50% of its available 
ingress bandwidth is allocated to network A and 50% is allocated to 
network B.

There would be multiple resources providers, each with an inventory 
record of resource class NET_INGRESS_BYTES_SEC with a total value of 1/2 
the total NIC bandwidth. Both of these resource providers would be child 
providers of the compute host resource provider. One of these child 
resource providers will be decorated with the trait "CUSTOM_NETWORK_A" 
and the other with trait "CUSTOM_NETWORK_B".

The scheduler would be able to determine which resource provider to 
consume the NET_INGRESS_BYTES_SEC resources from by looking for a 
resource provider that has both the required amount of 
NET_INGRESS_BYTES_SEC as well as the trait required by the port profile. 
If, say, the port profile specifies that the port is to go on a NIC with 
access to network "A", then the build request would contain a request to 
the scheduler for CUSTOM_NETWORK_A trait...

If you're coming to Denver, I encourage you to get with me, Sean Mooney, 
Moshe Levi and others who are interested in seeing this work move forward.

Best,
-jay

> Here is what I have in mind:
> - The VM need is specified in the flavor extra-specs, e.g. resources:CUSTOM_BANDWIDTH=123.
> - The compute node total capacity is specified in host aggregate metadata, e.g. CUSTOM_BANDWIDTH=999.
> - Nova then takes care of the rest: scheduling where the free capacity is sufficient, and performing simple resource usage accounting (updating the compute node free network bandwidth capacity as required).
> 
> Is the outline above according to current plans?
> If not, what would be possible/needed in order to achieve the same result, i.e. consider the VM network traffic need during VM placement?
> 
> BR,
> Philippe
> 
> [1] https://blueprints.launchpad.net/nova/+spec/bandwidth-as-scheduler-metric
> [2] https://wiki.openstack.org/wiki/NetworkBandwidthEntitlement
> [3] https://openstack.nimeyo.com/80515/openstack-scheduling-bandwidth-resources-nic_bw_kb-resource
> [4] https://docs.openstack.org/nova/latest/user/placement.html
> [5] http://specs.openstack.org/openstack/nova-specs/priorities/pike-priorities.html#placement
> [6] https://review.openstack.org/#/c/473627/
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>