Open Stack

Hi all,

Excellent definition of the issue at hand.
The recent blueprints of policy-based-scheduler and solver-scheduler 
indeed highlight a possible weakness in the current design, as despite 
their completely independent contributions (i.e. which filters to apply 
per request vs. how to compute a valid placement) their implementation as 
drivers makes combining them non-trivial.

As Alex Glikson hinted a couple of weekly meetings ago, our approach to 
this is to think of the driver's work as split between two entities:
-- A Placement Advisor, that constructs placement problems for scheduling 
requests (filter-scheduler and policy-based-scheduler)
-- A Placement Engine, that solves placement problems (HostManager in 
get_filtered_hosts() and solver-scheduler with its LP engine).

Such modularity should allow developing independent mechanisms that can be 
combined seamlessly through a unified & well-defined protocol based on 
constructing "placement problem" objects by the placement advisor and then 
passing them to the placement engine, which returns the solution. The 
protocol can be orchestrated by the scheduler manager. 

As can be seen at this point already, the policy-based-scheduler blueprint 
can now be positioned as an improvement of the placement advisor. 
Similarly, the solver-scheduler blueprint can be positioned as an 
improvement of the placement engine.

I'm working on a wiki page that will get into the details.
Would appreciate your initial thoughts on this approach.

Regards,
Gil



From:   Khanh-Toan Tran <khanh-toan.tran at cloudwatt.com>
To:     "OpenStack Development Mailing List \(not for usage questions\)" 
<openstack-dev at lists.openstack.org>, 
Date:   01/30/2014 01:43 PM
Subject:        Re: [openstack-dev] [Nova][Scheduler] Policy Based 
Scheduler and   Solver Scheduler



Hi Sylvain,
 
1) Some Filters such as AggregateCoreFilter, AggregateRAMFilter can change 
its parameters for aggregates. But what if admin wants to change for all 
hosts in an availability-zone? Does he have to rewrite all the parameters 
in all aggregates? Or should we create a new AvailabilityZoneCoreFilter?
 
The Policy Based Scheduler (PBS)  blueprint separates the effect (filter 
according to Core) from its target (all hosts in an aggregate, or in an 
availability-zone). It will benefit all filters, not just CoreFilter or 
RAMFilter, so that we can avoid creating for each filter XFilter the 
AggregateXFilter and AvailabilityZoneWFilter from now on. Beside, if admin 
wants to apply the a filter to some aggregates (or availability-zone) and 
not the other (don’t call filters at all, not just modify parameters), he 
can do it. It help us avoid running all filters on all hosts.
 
2) In fact, we also prepare for a separated scheduler in which PBS is a 
very first step of it, that’s why we purposely separate the Policy Based 
Scheduler from Policy Based Scheduling Module (PBSM) [1] which is the core 
of our architecture. If you look at our code, you will see that 
Policy_Based_Scheduler.py is only slightly different from Filter 
Scheduler. That is because we just want a link from Nova-scheduler to 
PBSM. We’re trying to push some more management into scheduler without 
causing too much modification, as you can see in the patch .
 
Thus I’m very happy when Gantt is proposed. As I see it, Gantt is based on 
Nova-scheduler code, with the planning on replacing nova-scheduler in J. 
The separation from Nova will be complicated, but not on scheduling part. 
Thus integrating PBS and PBSM into Gantt would not be a problem.
 
Best regards,
 
[1] 
https://docs.google.com/document/d/1gr4Pb1ErXymxN9QXR4G_jVjLqNOg2ij9oA0JrLwMVRA
 
Toan
 
De : Sylvain Bauza [mailto:sylvain.bauza at gmail.com] 
Envoyé : jeudi 30 janvier 2014 11:16
À : OpenStack Development Mailing List (not for usage questions)
Objet : Re: [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and 
Solver Scheduler
 
Hi Khanh-Toan,
 
I only have one comment on your proposal : why are you proposing something 
new for overcommitments with aggregates while the AggregateCoreFilter [1] 
and AggregateRAMFilter [2]already exist, which AIUI provide same feature ?
 
 
I'm also concerned about the scope of changes for scheduler, as Gantt is 
currently trying to replace it. Can we imagine such big changes to be 
committed on the Nova side, while it's planned to have a Scheduler service 
in the next future ?
 
-Sylvain
 
 
[1] 
https://github.com/openstack/nova/blob/master/nova/scheduler/filters/core_filter.py#L74
[2] 
https://github.com/openstack/nova/blob/master/nova/scheduler/filters/ram_filter.py#L75
 
 
 
 
2014-01-30 Khanh-Toan Tran <khanh-toan.tran at cloudwatt.com>
There is an unexpected line break in the middle of the link, so I post it
again:

https://docs.google.com/document/d/1RfP7jRsw1mXMjd7in72ARjK0fTrsQv1bqolOri
IQB2Y

> -----Message d'origine-----
> De : Khanh-Toan Tran [mailto:khanh-toan.tran at cloudwatt.com]
> Envoyé : mercredi 29 janvier 2014 13:25
> À : 'OpenStack Development Mailing List (not for usage questions)'
> Objet : [openstack-dev] [Nova][Scheduler] Policy Based Scheduler and
Solver
> Scheduler
>
> Dear all,
>
> As promised in the Scheduler/Gantt meeting, here is our analysis on the
> connection between Policy Based Scheduler and Solver Scheduler:
>
> https://docs.google.com/document/d/1RfP7jRsw1mXMjd7in72ARjK0fTrsQv1bq
> olOri
> IQB2Y
>
> This document briefs the mechanism of the two schedulers and the
possibility of
> cooperation. It is my personal point of view only.
>
> In a nutshell, Policy Based Scheduler allows admin to define policies
for different
> physical resources (an aggregate, an availability-zone, or all
> infrastructure) or different (classes of) users. Admin can modify
> (add/remove/modify) any policy in runtime, and the modification effect
is only
> in the target (e.g. the aggregate, the users) that the policy is defined
to. Solver
> Scheduler solves the placement of groups of instances simultaneously by
putting
> all the known information into a integer linear system and uses Integer
Program
> solver to solve the latter. Thus relation between VMs and between VMs-
> computes are all accounted for.
>
> If working together, Policy Based Scheduler can supply the filters and
weighers
> following the policies rules defined for different computes.
> These filters and weighers can be converted into constraints & cost
function for
> Solver Scheduler to solve. More detailed will be found in the doc.
>
> I look forward for comments and hope that we can work it out.
>
> Best regards,
>
> Khanh-Toan TRAN
>
>
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 _______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140130/0a524f3b/attachment.html>