Open Stack

On 07/18/2013 11:12 PM, Lu, Lianhao wrote:
> Sean Dague wrote on 2013-07-18:
>> On 07/17/2013 10:54 PM, Lu, Lianhao wrote:
>>> Hi fellows,
>>>
>>> Currently we're implementing the BP https://blueprints.launchpad.net/nova/+spec/utilization-aware-scheduling. The main idea is to have
>> an extensible plugin framework on nova-compute where every plugin can get different metrics(e.g. CPU utilization, memory cache
>> utilization, network bandwidth, etc.) to store into the DB, and the nova-scheduler will use that data from DB for scheduling decision.
>>>
>>> Currently we adds a new table to store all the metric data and have nova-scheduler join loads the new table with the compute_nodes
>> table to get all the data(https://review.openstack.org/35759). Someone is concerning about the performance penalty of the join load
>> operation when there are many metrics data stored in the DB for every single compute node. Don suggested adding a new column in the
>> current compute_nodes table in DB, and put all metric data into a dictionary key/value format and store the json encoded string of the
>> dictionary into that new column in DB.
>>>
>>> I'm just wondering which way has less performance impact, join load
>>> with a new table with quite a lot of rows, or json encode/decode a
>>> dictionary with a lot of key/value pairs?
>>>
>>> Thanks,
>>> -Lianhao
>>
>> I'm really confused. Why are we talking about collecting host metrics in
>> nova when we've got a whole project to do that in ceilometer? I think
>> utilization based scheduling would be a great thing, but it really out
>> to be interfacing with ceilometer to get that data. Storing it again in
>> nova (or even worse collecting it a second time in nova) seems like the
>> wrong direction.
>>
>> I think there was an equiv patch series at the end of Grizzly that was
>> pushed out for the same reasons.
>>
>> If there is a reason ceilometer can't be used in this case, we should
>> have that discussion here on the list. Because my initial reading of
>> this blueprint and the code patches is that it partially duplicates
>> ceilometer function, which we definitely don't want to do. Would be
>> happy to be proved wrong on that.
>>
>> 	-Sean
>>
> Using ceilometer as the source of those metrics was discussed in the
> nova-scheduler subgroup meeting. (see #topic extending data in host
> state in the following link).
> http://eavesdrop.openstack.org/meetings/scheduler/2013/scheduler.2013-04-30-15.04.log.html
> 
> In that meeting, all agreed that ceilometer would be a great source of
> metrics for scheduler, but many of them don't want to make the
> ceilometer as a mandatory dependency for nova scheduler. 

This was also discussed at the Havana summit and rejected since we
didn't want to introduce the external dependency of Ceilometer into Nova.

That said, we already have hooks at the virt layer for collecting host
metrics and we're talking about removing the pollsters from nova compute
nodes if the data can be collected from these existing hooks.

Whatever solution the scheduler group decides to use should utilize the
existing (and maintained/growing) mechanisms we have in place there.
That is, it should likely be a special notification driver that can get
the data back to the scheduler in a timely fashion. It wouldn't have to
use the rpc mechanism if it didn't want to, but it should be a plug-in
at the notification layer.

Please don't add yet another way of pulling metric data out of the hosts.

-S




> Besides, currently ceilometer doesn't have "host metrics", like the 
> cpu/network/cache utilization data of the compute node host, which
> will affect the scheduling decision. What ceilometer has currently
> is the "VM metrics", like cpu/network utilization of each VM instance.
> 
> After the nova compute node collects the "host metrics", those metrics
> could also be fed into ceilometer framework(e.g. through a ceilometer
> listener) for further processing, like alarming, etc.
> 
> -Lianhao
> 
> _______________________________________________
> OpenStack-dev mailing list
> OpenStack-dev at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>