[openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage compute/storage resource

Yathiraj Udupi (yudupi) yudupi at cisco.com
Thu Feb 27 04:33:04 UTC 2014


Hi Jay, Tim, Sylvain,

It is an important topic of run time monitoring and policies for management of compute/storage resources.  I agree this is the kind of solution that we should learn from Vmware DRS.
Regarding how the Solver Scheduler fits in, we initiated this effort to address complex constraint and policy scenarios, and the kinds of policies and constraints that become relevant for Load balancing, HA, energy saving, etc are all relevant constraints that we care about addressing and solving using the Solver scheduler.   However you should consider the solver scheduler as a pure placement decision engine, whether it is used for the initial placement, or if it is used for automatically triggering any action (such as a migration etc.) at runtime based on the constraints/policy checking.

As of now, the Solver Scheduler effort has been started to address the initial Nova placement decision, as a new scheduler driver.  But eventually with the Gantt efforts, of having scheduler as a separate service, and the resulting new apis, it is heading towards being a true decision engine, that can be triggered at any point of time, to figure out the ideal placements given the current state of the resource capacities.

But the kinds of policies you have mentioned here,  appear as a true fit for being addressed as the complex constraints and cost metrics that can be used to be solved the Solver Scheduler’s intelligent constraints-based optimization solver.

We (the solver scheduler sub-team) will be very happy to work with you to address some of these scenarios, and we can have a session at the upcoming summit regarding this kind of run-time policies.

Thanks,
Yathi.

—



On 2/26/14, 6:30 PM, "Jay Lau" <jay.lau.513 at gmail.com<mailto:jay.lau.513 at gmail.com>> wrote:

Hi Tim,

I'm not sure if we can put resource monitor and adjust to solver-scheduler (Gantt), but I have proposed this to Gantt design [1], you can refer to [1] and search "jay-lau-513".

IMHO, Congress does monitoring and also take actions, but the actions seems mainly for adjusting single VM network or storage. It did not consider migrating VM according to hypervisor load.

Not sure if this topic deserved to be a design session for the coming summit, but I will try to propose.

[1] https://etherpad.openstack.org/p/icehouse-external-scheduler

Thanks,

Jay

2014-02-27 1:48 GMT+08:00 Tim Hinrichs <thinrichs at vmware.com<mailto:thinrichs at vmware.com>>:
Hi Jay and Sylvain,

The solver-scheduler sounds like a good fit to me as well.  It clearly provisions resources in accordance with policy.  Does it monitor those resources and adjust them if the system falls out of compliance with the policy?

I mentioned Congress for two reasons. (i) It does monitoring.  (ii) There was mention of compute, networking, and storage, and I couldn't tell if the idea was for policy that spans OS components or not.  Congress was designed for policies spanning OS components.

Tim

----- Original Message -----
| From: "Jay Lau" <jay.lau.513 at gmail.com<mailto:jay.lau.513 at gmail.com>>
| To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org>>
| Sent: Tuesday, February 25, 2014 10:13:14 PM
| Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal for OpenStack run time policy to manage
| compute/storage resource
|
|
|
|
|
| Thanks Sylvain and Tim for the great sharing.
|
| @Tim, I also go through with Congress and have the same feeling with
| Sylvai, it is likely that Congress is doing something simliar with
| Gantt providing a holistic way for deploying. What I want to do is
| to provide some functions which is very similar with VMWare DRS that
| can do some adaptive scheduling automatically.
|
| @Sylvain, can you please show more detail for what "Pets vs. Cattles
| analogy" means?
|
|
|
|
| 2014-02-26 9:11 GMT+08:00 Sylvain Bauza < sylvain.bauza at gmail.com<mailto:sylvain.bauza at gmail.com> > :
|
|
|
| Hi Tim,
|
|
| As per I'm reading your design document, it sounds more likely
| related to something like Solver Scheduler subteam is trying to
| focus on, ie. intelligent agnostic resources placement on an
| holistic way [1]
| IIRC, Jay is more likely talking about adaptive scheduling decisions
| based on feedback with potential counter-measures that can be done
| for decreasing load and preserving QoS of nodes.
|
|
| That said, maybe I'm wrong ?
|
|
| [1] https://blueprints.launchpad.net/nova/+spec/solver-scheduler
|
|
|
| 2014-02-26 1:09 GMT+01:00 Tim Hinrichs < thinrichs at vmware.com<mailto:thinrichs at vmware.com> > :
|
|
|
|
| Hi Jay,
|
| The Congress project aims to handle something similar to your use
| cases. I just sent a note to the ML with a Congress status update
| with the tag [Congress]. It includes links to our design docs. Let
| me know if you have trouble finding it or want to follow up.
|
| Tim
|
|
|
| ----- Original Message -----
| | From: "Sylvain Bauza" < sylvain.bauza at gmail.com<mailto:sylvain.bauza at gmail.com> >
| | To: "OpenStack Development Mailing List (not for usage questions)"
| | < openstack-dev at lists.openstack.org<mailto:openstack-dev at lists.openstack.org> >
| | Sent: Tuesday, February 25, 2014 3:58:07 PM
| | Subject: Re: [openstack-dev] [OpenStack][Runtime Policy] A proposal
| | for OpenStack run time policy to manage
| | compute/storage resource
| |
| |
| |
| | Hi Jay,
| |
| |
| | Currently, the Nova scheduler only acts upon user request (either
| | live migration or boot an instance). IMHO, that's something Gantt
| | should scope later on (or at least there could be some space within
| | the Scheduler) so that Scheduler would be responsible for managing
| | resources on a dynamic way.
| |
| |
| | I'm thinking of the Pets vs. Cattles analogy, and I definitely
| | think
| | that Compute resources could be treated like Pets, provided the
| | Scheduler does a move.
| |
| |
| | -Sylvain
| |
| |
| |
| | 2014-02-26 0:40 GMT+01:00 Jay Lau < jay.lau.513 at gmail.com<mailto:jay.lau.513 at gmail.com> > :
| |
| |
| |
| |
| | Greetings,
| |
| |
| | Here I want to bring up an old topic here and want to get some
| | input
| | from you experts.
| |
| |
| | Currently in nova and cinder, we only have some initial placement
| | polices to help customer deploy VM instance or create volume
| | storage
| | to a specified host, but after the VM or the volume was created,
| | there was no policy to monitor the hypervisors or the storage
| | servers to take some actions in the following case:
| |
| |
| | 1) Load Balance Policy: If the load of one server is too heavy,
| | then
| | probably we need to migrate some VMs from high load servers to some
| | idle servers automatically to make sure the system resource usage
| | can be balanced.
| |
| | 2) HA Policy: If one server get down for some hardware failure or
| | whatever reasons, there is no policy to make sure the VMs can be
| | evacuated or live migrated (Make sure migrate the VM before server
| | goes down) to other available servers to make sure customer
| | applications will not be affect too much.
| |
| | 3) Energy Saving Policy: If a single host load is lower than
| | configured threshold, then low down the frequency of the CPU to
| | save
| | energy; otherwise, increase the CPU frequency. If the average load
| | is lower than configured threshold, then shutdown some hypervisors
| | to save energy; otherwise, power on some hypervisors to load
| | balance. Before power off a hypervisor host, the energy policy need
| | to live migrate all VMs on the hypervisor to other available
| | hypervisors; After Power on a hypervisor host, the Load Balance
| | Policy will help live migrate some VMs to the new powered
| | hypervisor.
| |
| | 4) Customized Policy: Customer can also define some customized
| | policies based on their specified requirement.
| |
| | 5) Some run-time policies for block storage or even network.
| |
| |
| |
| | I borrow the idea from VMWare DRS (Thanks VMWare DRS), and there
| | indeed many customers want such features.
| |
| |
| |
| | I have filed a bp here [1] long ago, but after some discussion with
| | Russell, we think that this should not belong to nova but other
| | projects. Till now, I did not find a good place where we can put
| | this in, can any of you show some comments?
| |
| |
| |
| | [1]
| | https://blueprints.launchpad.net/nova/+spec/resource-optimization-service
| |
| | --
| |
| |
| | Thanks,
| |
| | Jay
| |
| | _______________________________________________
| | OpenStack-dev mailing list
| | OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
| | http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
| |
| |
| |
| | _______________________________________________
| | OpenStack-dev mailing list
| | OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
| | https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=%2FZ35AkRhp2kCW4Q3MPeE%2BxY2bqaf%2FKm29ZfiqAKXxeo%3D%0A&m=XDB3hT4WE2iDrNVK0sQ8qKooX2r1T4E%2BVHek3GREhnE%3D%0A&s=e2346cd017c9d8108c12a101892492e2ac75953e4a5ea5c17394c775cf086d7f
|
|
| |
|
| _______________________________________________
| OpenStack-dev mailing list
| OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
| http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
|
|
| _______________________________________________
| OpenStack-dev mailing list
| OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
| http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
|
|
|
|
| --
|
|
| Thanks,
|
| Jay
|
| _______________________________________________
| OpenStack-dev mailing list
| OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
| https://urldefense.proofpoint.com/v1/url?u=http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=%2FZ35AkRhp2kCW4Q3MPeE%2BxY2bqaf%2FKm29ZfiqAKXxeo%3D%0A&m=E45fhmBClHHPExheGdRk0z%2Bj72gQAP4Zc1W3XElJx60%3D%0A&s=684cee6930f5d74f56e1ab9fc42e5f3c2511f07948f357040ca2dc175c4ccee6
|

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev at lists.openstack.org<mailto:OpenStack-dev at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Thanks,

Jay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20140227/d0239116/attachment.html>


More information about the OpenStack-dev mailing list