[openstack-dev] [nova][scheduler] Proposal: FairShareScheduler.
eric.frizziero at pd.infn.it
Mon Jun 30 14:05:23 UTC 2014
we have analyzed the nova-scheduler component (FilterScheduler) in our
Openstack installation used by some scientific teams.
In our scenario, the cloud resources need to be distributed among the
teams by considering the predefined share (e.g. quota) assigned to each
team, the portion of the resources currently used and the resources they
have already consumed.
We have observed that:
1) User requests are sequentially processed (FIFO scheduling), i.e.
FilterScheduler doesn't provide any dynamic priority algorithm;
2) User requests that cannot be satisfied (e.g. if resources are not
available) fail and will be lost, i.e. on that scenario nova-scheduler
doesn't provide any queuing of the requests;
3) OpenStack simply provides a static partitioning of resources among
various projects / teams (use of quotas). If project/team 1 in a period
is systematically underutilizing its quota and the project/team 2
instead is systematically saturating its quota, the only solution to
give more resource to project/team 2 is a manual change (to be done by
the admin) to the related quotas.
The need to find a better approach to enable a more effective scheduling
in Openstack becomes more and more evident when the number of the user
requests to be handled increases significantly. This is a well known
problem which has already been solved in the past for the Batch Systems.
In order to solve those issues in our usage scenario of Openstack, we
have developed a prototype of a pluggable scheduler, named
FairShareScheduler, with the objective to extend the existing OpenStack
scheduler (FilterScheduler) by integrating a (batch like) dynamic
The architecture of the FairShareScheduler is explicitly designed to
provide a high scalability level. To all user requests will be assigned
a priority value calculated by considering the share allocated to the
user by the administrator and the evaluation of the effective resource
usage consumed in the recent past. All requests will be inserted in a
priority queue, and processed in parallel by a configurable pool of
workers without interfering with the priority order. Moreover all
significant information (e.g. priority queue) will be stored in a
persistence layer in order to provide a fault tolerance mechanism while
a proper logging system will annotate all relevant events, useful for
In more detail, some features of the FairshareScheduler are:
a) It assigns dynamically the proper priority to every new user requests;
b) The priority of the queued requests will be recalculated periodically
using the fairshare algorithm. This feature guarantees the usage of the
cloud resources is distributed among users and groups by considering the
portion of the cloud resources allocated to them (i.e. share) and the
resources already consumed;
c) all user requests will be inserted in a (persistent) priority queue
and then processed asynchronously by the dedicated process (filtering +
weighting phase) when compute resources are available;
d) From the client point of view the queued requests remain in
“Scheduling” state till the compute resources are available. No new
states added: this prevents any possible interaction issue with the
e) User requests are dequeued by a pool of WorkerThreads (configurable),
i.e. no sequential processing of the requests;
f) The failed requests at filtering + weighting phase may be inserted
again in the queue for n-times (configurable).
We have integrated the FairShareScheduler in our Openstack installation
(release "HAVANA"). We're now working to adapt the FairShareScheduler to
the new release "IceHouse".
Does anyone have experiences in those issues found in our cloud scenario?
Could the FairShareScheduler be useful for the Openstack community?
In that case, we'll be happy to share our work.
Any feedback/comment is welcome!
More information about the OpenStack-dev