Open Stack

Well, my main thought is that I would prefer to see the gantt split done sooner rather than later.  The reality is that we've been trying to split out the scheduler for months and we're still not there.  Until we bite the bullet and actually do the split I'm afraid we'll still be here discussing the `best` way to do the split at the K & L summits (there's a little bit of `the perfect is the enemy of the good' happening here).  With the creation of the client library we've created a good seam to split out the scheduler, let's do the split and fix the remaining problems (aggregates and instance group references).

To address some specific points:

1)  This is the same problem that caused the last split to fail.  Actually, I think the last split failed because we didn't get the gantt code sufficiently isolated from nova.  If we get the new split completely divorced from the nova code then we can concentrate on scheduler changes and not get bogged down constantly tracking the complete nova tree.

2)  We won't get around to creating parity between gantt and nova.  Gantt will never be the default scheduler until it has complete parity with the nova scheduler, that should give us sufficient incentive to make sure we achieve parity as soon as possible.

3)  The split should be done at the beginning of the cycle.  I don't see a need for that, we should do the split whenever we are ready.  Since gantt will be optional it shouldn't affect release issues with nova and the sooner we have a separate tree the sooner people can test and develop on the gantt tree.

4)  Start the deprecation counter at the split.  I agree the deprecation counter is something that should be started at the beginning of a cycle but that can be any time after the split and after gantt has feature parity.  It's fine to have the gantt tree up for a while before we start the deprecation counter, the two are independent of each other.

--
Don Dugger
"Censeo Toto nos in Kansa esse decisse." - D. Gale
Ph: 303/443-3786

-----Original Message-----
From: Sylvain Bauza [mailto:sbauza at redhat.com] 
Sent: Thursday, July 3, 2014 11:53 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Nova] [Gantt] Scheduler split status

Hi,

==
tl; dr: A decision has been made to split out the scheduler to a separate project not on a feature parity basis with nova-scheduler, your comments are welcome.
==

As it has been agreed now a cycle ago, the nova-scheduler will be ported to a separate OpenStack project, called Gantt [1]. The plan is to do all the necessary changes in Nova, and then split the code into a separate project and provide CI against the new project [2]


During the preparation phase, it has been identified a couple of blueprints which needed to be delivered before the split could happen :

A/
https://blueprints.launchpad.net/nova/+spec/remove-cast-to-schedule-run-instance
(merged): was about removing the possibility for the scheduler to proxy calls to compute nodes. Now the scheduler can't call computes nodes when booting. That said, there is still one pending action [3] about cold migrations that needs to be tackled. Your reviews are welcome on the spec [4] and implementation [5]


B/ A scheduler library has to be provided, so the interface would be the same for both nova-scheduler and Gantt. The idea is to define all the inputs/outputs of the scheduler, in particular how we update the Scheduler internal state (here the ComputeNode table). The spec has been approved, the implementation is waiting for reviews [6]. The main problem is about the ComputeNode (well, compute_nodes to be precise) table and the foreign key it has on Service, but also the foreign key that PCITracker has on ComputeNode ID primary key, which requires the table to be left in Nova (albeit for the solely use of the scheduler)

C/ Some of the Scheduler filters currently access other Nova objects (aggregates and instance groups) and ServiceGroups are accessed by the Scheduler driver to know the state of each host (is it up or not ?), so we need to port these calls to Nova and update the scheduler state from a distant perspective. This spec is currently under review [7] and the changes are currently being disagreed [8].



During the last Gantt meeting held Tuesday, we discussed about the status and the problems we have. As we are close to Juno-2, there are some concerns about which blueprints would be implemented by Juno, so Gantt would be updated after. Due to the problems raised in the different blueprints (please see the links there), it has been agreed to follow a path a bit different from the one agreed at the Summit : once B/ is merged, Gantt will be updated and work will happen in there while work with C/ will happen in parallel. That means we need to backport in Gantt all changes happening to the scheduler, but (and this is the most important point) until C/ is merged into Gantt, Gantt won't support filters which decide on aggregates or instance groups. In other words, until C/ happens (but also A/), Gantt won't be feature-parity with Nova-scheduler.

That doesn't mean Gantt will move forward and leave all missing features out of it, we will be dedicated to feature-parity as top priority but that implies that the first releases of Gantt will be experimental and considered for testing purposes only.


Your thoughts are welcome here whatever your opinion is.


Thanks,
-Sylvain

[1] https://etherpad.openstack.org/p/icehouse-external-scheduler
[2] https://etherpad.openstack.org/p/juno-nova-gantt-apis
[3]
https://blueprints.launchpad.net/nova/+spec/move-prep-resize-to-conductor
[4] https://review.openstack.org/94916
[5] https://review.openstack.org/103503 (WIP) [6] https://review.openstack.org/82778
[7] https://review.openstack.org/89893
[8] https://review.openstack.org/101128 and
https://review.openstack.org/101196

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev