<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
(Instead of having a copy of these emails in the gdoc, I resolved the comment in the gdoc and added a link to this email thread in the ML archives.)
<div class=""><br class="">
</div>
<div class="">Inline.<br class="">
<div class=""><br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Mar 20, 2015, at 8:34 AM, Yathiraj Udupi (yudupi) <<a href="mailto:yudupi@cisco.com" class="">yudupi@cisco.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class="">Hi Tim, </div>
<div class=""><br class="">
</div>
<div class="">I think what you are saying is a reasonable goal in terms of high-level Congress policies not having to depend on the domain-specific solver / policy engines. As long as there are Congress adapters to transform the user’s policies to something
that domain-specific solver understands it will be fine. </div>
<div class=""><br class="">
</div>
<div class="">With respect to delegation to solver scheduler, I agree that we need a combination of options (1) and (2). </div>
<div class="">For option (2), in order to leverage the pre-defined constraint classes in Solver Scheduler, it is more like the Congress adapter notifying the Solver Scheduler to include that constraint as part of the current placement decision. Also we have
to take it to account that it is not just the user-defined Congress policies that define the placement choices, the already existing infrastructure-specific constraints, in terms of Host capacities, or any other provider specific constraints will have to
be included in the placement decision calculation by the solver. The pre-configured scheduler/placement constraints will always have to be included. But some additional policies from Congress can introduce additional constraints. </div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>I’d love to hear more about this. Where do these other policies/configurations/etc. live? What do they look like? One of the things we’ve always discussed is a model where we combine policies from different places to understand the *actual* policy that
we need to act on. If you could walk us through an example that shows different kinds of constraints and where they come from, that’d be a big help!</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class=""><br class="">
</div>
<div class="">For option (1), new custom constraint class - Solver Scheduler already has an interface BaseLinearConstraint - <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stackforge_nova-2Dsolver-2Dscheduler_blob_master_nova_scheduler_solvers_linearconstraints_-5F-5Finit-5F-5F.py&d=AwMF-g&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=_MGkpdeOqH9x07MnF4pUGr69548scQd0-bA67jlWkHs&s=b0wV-mTuXGGQGYRUfvVshJDpNAT36A8bKVxcypWafgY&e=" class="">https://github.com/stackforge/nova-solver-scheduler/blob/master/nova/scheduler/solvers/linearconstraints/__init__.py</a> </div>
<div class="">with methods - <span class="pl-en" style="box-sizing: border-box; color: rgb(121, 93, 163); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; widows: 1; background-color: rgb(255, 255, 255);">get_coefficient_vectors</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; widows: 1; background-color: rgb(255, 255, 255);" class="">,
</span><span class="pl-en" style="box-sizing: border-box; color: rgb(121, 93, 163); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; widows: 1; background-color: rgb(255, 255, 255);">get_variable_vectors</span><span style="color: rgb(51, 51, 51); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; widows: 1; background-color: rgb(255, 255, 255);" class="">,
and </span><span style="background-color: rgb(255, 255, 255); color: rgb(121, 93, 163); font-family: Consolas, 'Liberation Mono', Menlo, Courier, monospace; font-size: 12px; line-height: 16.7999992370605px; white-space: pre; widows: 1;" class="">get_operations
</span>that will be invoked by the main solver class to feed in the variables, and the host metrics, along with some input parameters that used to get additional metadata that can be used to build matrices. Eventually the main solver class builds the LP program
by invoking all the classes. So for the Congress delegation scenario, it will be along these lines, where the Congress adapter will have to populate these matrices based on the Datalog policy. So as part of the solver scheduler’s workflow this special CongressConstraint
class will have to call the Congress adapter with the variables already known, and get the necessary values. </div>
<div class="">For reference, an example implementation of this constraint class is here - <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_stackforge_nova-2Dsolver-2Dscheduler_blob_master_nova_scheduler_solvers_linearconstraints_max-5Fdisk-5Fallocation-5Fconstraint.py&d=AwMF-g&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=_MGkpdeOqH9x07MnF4pUGr69548scQd0-bA67jlWkHs&s=kRmhPNGUt-v8xSY-tz1EsR_Ylcc4pr0gnuC9zRYkxno&e=" class="">https://github.com/stackforge/nova-solver-scheduler/blob/master/nova/scheduler/solvers/linearconstraints/max_disk_allocation_constraint.py</a> </div>
<div class=""><br class="">
</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>That sounds promising. I’d like to do a compare/contrast of how it would look with and without the solver-scheduler interface. I’ve never used an LP solver, so it’s hard for me to understand the benefits of the interface you have over using the raw LP
solver interface. Again, an example would help me.</div>
<div><br class="">
</div>
<div>Tim</div>
<br class="">
<blockquote type="cite" class="">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; font-size: 14px; font-family: Calibri, sans-serif;" class="">
<div class=""></div>
<div class="">Will need some more thoughts, but the approach seems reasonable. </div>
<div class=""><br class="">
</div>
<div class="">Thanks,</div>
<div class="">Yathi. </div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<span id="OLK_SRC_BODY_SECTION" class="">
<div class="">
<div class="">On 3/18/15, 8:34 AM, "Tim Hinrichs" <<a href="mailto:thinrichs@vmware.com" class="">thinrichs@vmware.com</a>> wrote:</div>
</div>
<div class=""><br class="">
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" class="" type="cite">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
I responded in the gdoc. Here’s a copy.
<div class=""><br class="">
</div>
<div class="">
<div class="">One of my goals for delegation is to avoid asking people to write policy statements specific to any particular domain-specific solver. People ought to encode policy however they like, and the system ought to figure out how best to enforce that
policy (delegation being one option).</div>
<div class=""><br class="">
</div>
<div class="">Assuming that's a reasonable goal, I see two options for delegation to solverScheduler</div>
<div class=""><br class="">
</div>
<div class="">(1) SolverScheduler exposes a custom constraint class. Congress generates the LP program from the Datalog, similar to what is described in this doc, and gives that LP program as custom constraints to the SolverScheduler. SolverScheduler is
then responsible for enforcing that policy both during provisioning of new servers and for monitoring/migrating servers once provisioning is finished.</div>
<div class=""><br class="">
</div>
<div class="">(2) The Congress adapter for SolverScheduler understands the semantics of MemoryCapacityConstraint, identifies when the user has asked for that constraint, and replaces that part of the LP program with the MemoryCapacityConstraint.</div>
<div class=""><br class="">
</div>
<div class="">We probably want a combination of (1) and (2) so that we handle any gaps in the pre-defined constraints that SolverScheduler has, while at the same time leveraging the pre-defined constraints when possible.</div>
<div class=""><br class="">
</div>
<div class="">Tim</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">
<blockquote type="cite" class="">
<div class="">On Mar 17, 2015, at 6:09 PM, Yathiraj Udupi (yudupi) <<a href="mailto:yudupi@cisco.com" class="">yudupi@cisco.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">Hi Tim, </div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><br class="">
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">I posted this comment on the doc. I am still pondering over a possibility of have a policy-driven scheduler workflow via the Solver Scheduler placement engine, which is also LP based
like you describe in your doc. </div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class="">I know in your initial meeting, you plan to go over your proposal of building a VM placement engine that subscribes to the Congress DSE, I probably will understand the Congress workflows
better and see how I could incorporate this proposal to talk to the Solver Scheduler to make the placement decisions. </div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><br class="">
</div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">The example you provide in the doc,
is a very good scenario, where a VM placement engine should continuously monitor and trigger VM migrations. </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">I am also interested in the case of a policy-driven scheduling for the initial creation of VMs. This is where
say people will call Nova APIs and create a new set of VMs. Here the scheduler workflow should address the constraints as imposed from the user's policies. </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">Say the simple policy is " Host's free RAM >= 0.25 * Memory_Capacity"</span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">I would like the scheduler to use this policy as defined from Congress, and apply it during the scheduling
as part of the Nova boot call. </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">I am really interested in and need help in coming up with a solution integrating Solver Scheduler, so say
if I have an implementation of a "MemoryCapacityConstraint", which takes a hint value "free_memory_limit" (0.25 in this example), </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">could we have a policy in Datalog </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">placement_requirement(id) :-</span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">nova:host(id), </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">solver_scheduler:applicable_constraints(id, ["MemoryCapacityConstraint", ]), </span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">applicable_metadata(id, {"free_memory_limit": 0.25, })</span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<span style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">This policy could be set and delegated by Congress to solver scheduler via the "set_policy" API. or the Solver
Scheduler can query Congress via a "get_policy" API to get this policy, and incorporate it as part of the solver scheduler workflow ?</span><br style="color: rgb(51, 51, 51); font-family: Arial, sans-serif, sans; font-size: 13px; widows: 1; background-color: rgb(255, 255, 255);" class="">
<br class="">
</div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class="">Does this sound doable ?</span></font></div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class=""><br class="">
</span></font></div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class="">Thanks,</span></font></div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class="">Yathi. </span></font></div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class=""><br class="">
</span></font></div>
<div class=""><font face="Arial" class=""><span style="font-size: 13px;" class=""><br class="">
</span></font></div>
<div style="font-family: Calibri, sans-serif; font-size: 14px;" class=""><br class="">
</div>
<span id="OLK_SRC_BODY_SECTION" style="font-family: Calibri, sans-serif; font-size: 14px;" class="">
<div class="">
<div class="">On 3/16/15, 11:05 AM, "Tim Hinrichs" <<a href="mailto:thinrichs@vmware.com" class="">thinrichs@vmware.com</a>> wrote:</div>
</div>
<div class=""><br class="">
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;" class="" type="cite">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">
Hi all,
<div class=""><br class="">
</div>
<div class="">The feedback on the POC delegation proposal has been mostly positive. Several people have asked for a meeting to discuss further. Given time zone constraints, it will likely be 8a or 9a Pacific. Let me know in the next 2 days if you want to
participate, and we will try to find a day that everyone can attend.</div>
<div class=""><br class="">
</div>
<div class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ksDilJYXV-2D5AXWON8PLMedDKr9NpS8VbT0jIy-5FMIEtI_edit&d=AwMFAg&c=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEs&r=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7s&m=uiVXAFxgoK-F8a3oNLDykcTSmPGUMW2_kFB_BnUEBFg&s=GWRQesGone_FbZRHXZmIcQd5MB20CHkriADqVNVnxiA&e=" class="">https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit</a></div>
<div class=""><br class="">
</div>
<div class="">Thanks!</div>
<div class="">Tim</div>
</div>
</div>
</blockquote>
</span></div>
__________________________________________________________________________<br class="">
OpenStack Development Mailing List (not for usage questions)<br class="">
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" class="">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</div>
</blockquote>
</span></div>
__________________________________________________________________________<br class="">
OpenStack Development Mailing List (not for usage questions)<br class="">
Unsubscribe: <a href="mailto:OpenStack-dev-request@lists.openstack.org" class="">
OpenStack-dev-request@lists.openstack.org</a>?subject:unsubscribe<br class="">
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" class="">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br class="">
</div>
</blockquote>
</div>
<br class="">
</div>
</div>
</body>
</html>