<br><br><div class="gmail_quote">On Fri, May 25, 2012 at 4:55 AM, Szymon Grzybowski <span dir="ltr"><<a href="mailto:semyazz@gmail.com" target="_blank">semyazz@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">Hi,<br><br>We had a plan to write some "plugin" which will collect metering data from VMs and Hosts (free ram, networking, disk IO, cpu both from VMs and Hosts) via libvirt (mayby later for xen etc), but i've found ceilometer project, which is doing what I want or will be doing what I want in near future :). I was thinking about collectd/ganglia/munin as source of data and I wanted to write local agents which will poll those daemons (collectd/ganglia) and push data to Rabbit, so it's pretty close to ceilometer's guys.<br>
</div></blockquote><div><br></div><div>The idea with metering is that by standardizing on a message format you can tie in at a couple of different points. For publishing data, you could write a plugin for our agent that does the polling and lets the agent publish, or you could have your agent publish messages directly on the metering queue in the right format. However, our focus right now is on capturing information about utilization so we can document it and charge the customer. For the sort of features you are talking about, you probably want more data collected more frequently.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">My main purpose is to consume those messages and write algorithm to manage cloud from administrative perspective. I'd like to create something like DRM in VmWare - I was on presentation about this topic recently and from that brief I learnt that VmWare has something like rules to manage VMs. I think, that example (use case) from presentation would be the best way to describe what I'm trying to point out:<br>
<br>1. Administrator defined a rule: { If free ram on host is less than 15%, send notification "Alert less than 15% free ram on host"}<br>2. HostA hits less than 15% of free ram and sends notification "Alert less than 15% free ram on HostA".<br clear="all">
3. Central collector or central daemon receives Alert about free ram.<br>4. Central collector pick VM from HostA and migrate it to HostB. - How he decides which VM and to which Host migrate is part of algorithm/policy.<br>
<br>This is the basic idea, how does it work - this is what I've found out during presentation. And now my whole idea about this mechanism and openstack: there could be different policies(algorithms/plugins) how, where and when migrate VMs and this could be implemented in external plugins.<br>
<br>Someone can say that there is nova-scheduler. This scenario isn't exactly what nova-scheduler is doing, but I see, that in this solution nova-scheduler can play his role as service, which picks best host to migrate VM.<br>
</div></blockquote><div><br></div><div>It sounds like an updated scheduler class could use this new data to make its decision about which host is least busy, at least.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">I'm trying to find out what can I use from ceilometer as metering system. I've read few pages on wiki and posts on mailing list about ceilometer's architecture, messages etc.<br>Right now I know there are Agents and Collectors. Agents get data from hypervisor (metering data) and push them in queue (nova notification system). Collectors listens to queue's topic and receives those messages. I'd like to:<br>
</div></blockquote><div><br></div><div>The agent will publish to a new queue using a pattern similar to notification, but the messages won't be in quite the same format or on the same exchange.</div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote">
<ul><li>write plugin for agent to send proper alert when metering data hits proper rules (just like in above scenario with 15% free ram) </li></ul></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_quote"><ul><li>write service which will consume this alert (listen proper topic in queue) and fetch data from ceilometer collector?? (or it's backend via API) and find out which VM should be migrated and where (or delegate "Where" to nova-scheduler)<br>
</li></ul></div></blockquote><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote"><ul><li>
</li></ul>Another idea. Are there any Host states in Openstack? For example "Maintenance state"? Besides scenarios with free ram, cpu and other resources there is also typical example of how useful this mechanism could be with maintenance state.<br>
<i>Scenario:</i><br>1. Administrator changes HostA state to "maintenance".<br>2. There goes alert "HostA maintenance".<br>3. Central collector or central daemon receives Alert about maintenance.<br>4. Central collector gets list of VMs on HostA.<br>
5. Central collectors start migration all VMs from HostA to other Hosts (or invoke nova-scheduler to reassign VM from HostA to other hosts, but in this case we need to change or implement new filter in nova-scheduler)<br>
6. Administrator changes HostA state to "active".<br>7. Openstack starts new machines on this node (this is now done via nova-scheduler) or migrate VMs from other overloaded hosts to HostA. <br><br>Everything should be event-driven (alert-notification, host-state-change, administrator's action).<br>
<br>I'm also wondering what about this <a href="http://wiki.openstack.org/ResourceMonitorAlertsandNotifications" target="_blank">ResourceMonitorAlertsandNotifications</a>, because when I'm looking at schema, it has everything what I need, but blueprint link is broken. Is this idea obsolete? Or is it implemented? (this was created 2nd April 2012, so I think it's not implemented yet).<br>
<br>First question: Does it make sense? Are such mechanisms implemented in Openstack right now? Or is it worth to implement?<br></div></blockquote><div><br></div><div>The feature you are planning sounds interesting, and the implementation for the data collection will follow a pattern similar to what we are doing for metering. I don't know if it will make sense to implement the new collection as part of the metering system for two reasons. First, the frequency we plan to collect data may be too low for your purposes. That is adjustable, so maybe it can work. Second, the metering data is meant to be used for billing, so I'm not sure resource availability information like RAM on the host fits into the model. Again, that is flexible and open to re-interpretation. </div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">Second question (if first's answer isn't negative): How and when can I use ceilometer to do what I've described? From <a href="http://wiki.openstack.org/Meetings/MeteringAgenda" target="_blank">meeting's topics</a> I know that yesterday you were deciding what to use as notification bus, and in june you will be thinking about storage backend, how is it going on right now? Is such scenario possible in ceilometer (I mean plugins in agents)? Collecting data not only from guests but also from hosts.<br>
</div></blockquote><div><br></div><div>The ceilometer agent runs on the host, so it can see the information you want. However, given the limitations I described above, it may make more sense to set up a "monitoring" system that performs operations similar to ceilometer but uses its own messaging exchange so the messages can be routed to the right listeners. Or perhaps the ceilometer agent should become a more general-purpose tool that emits information for both types of consumers.</div>
<div><br></div><div>Doug</div><div><br></div></div>