Open Stack

Hi,

==
tl; dr: A decision has been made to split out the scheduler to a
separate project not on a feature parity basis with nova-scheduler, your
comments are welcome.
==

As it has been agreed now a cycle ago, the nova-scheduler will be ported
to a separate OpenStack project, called Gantt [1]. The plan is to do all
the necessary changes in Nova, and then split the code into a separate
project and provide CI against the new project [2]


During the preparation phase, it has been identified a couple of
blueprints which needed to be delivered before the split could happen :

A/
https://blueprints.launchpad.net/nova/+spec/remove-cast-to-schedule-run-instance
(merged): was about removing the possibility for the scheduler to proxy
calls to compute nodes. Now the scheduler can't call computes nodes when
booting. That said, there is still one pending action [3] about cold
migrations that needs to be tackled. Your reviews are welcome on the
spec [4] and implementation [5]


B/ A scheduler library has to be provided, so the interface would be the
same for both nova-scheduler and Gantt. The idea is to define all the
inputs/outputs of the scheduler, in particular how we update the
Scheduler internal state (here the ComputeNode table). The spec has been
approved, the implementation is waiting for reviews [6]. The main
problem is about the ComputeNode (well, compute_nodes to be precise)
table and the foreign key it has on Service, but also the foreign key
that PCITracker has on ComputeNode ID primary key, which requires the
table to be left in Nova (albeit for the solely use of the scheduler)

C/ Some of the Scheduler filters currently access other Nova objects
(aggregates and instance groups) and ServiceGroups are accessed by the
Scheduler driver to know the state of each host (is it up or not ?), so
we need to port these calls to Nova and update the scheduler state from
a distant perspective. This spec is currently under review [7] and the
changes are currently being disagreed [8].



During the last Gantt meeting held Tuesday, we discussed about the
status and the problems we have. As we are close to Juno-2, there are
some concerns about which blueprints would be implemented by Juno, so
Gantt would be updated after. Due to the problems raised in the
different blueprints (please see the links there), it has been agreed to
follow a path a bit different from the one agreed at the Summit : once
B/ is merged, Gantt will be updated and work will happen in there while
work with C/ will happen in parallel. That means we need to backport in
Gantt all changes happening to the scheduler, but (and this is the most
important point) until C/ is merged into Gantt, Gantt won't support
filters which decide on aggregates or instance groups. In other words,
until C/ happens (but also A/), Gantt won't be feature-parity with
Nova-scheduler.

That doesn't mean Gantt will move forward and leave all missing features
out of it, we will be dedicated to feature-parity as top priority but
that implies that the first releases of Gantt will be experimental and
considered for testing purposes only.


Your thoughts are welcome here whatever your opinion is.


Thanks,
-Sylvain

[1] https://etherpad.openstack.org/p/icehouse-external-scheduler
[2] https://etherpad.openstack.org/p/juno-nova-gantt-apis
[3]
https://blueprints.launchpad.net/nova/+spec/move-prep-resize-to-conductor
[4] https://review.openstack.org/94916
[5] https://review.openstack.org/103503 (WIP)
[6] https://review.openstack.org/82778
[7] https://review.openstack.org/89893
[8] https://review.openstack.org/101128 and
https://review.openstack.org/101196