[openstack-dev] [Nova] New DB column or new DB table?
Murray, Paul (HP Cloud Services)
pmurray at hp.com
Thu Jul 18 14:44:54 UTC 2013
I would like to chip in with something from the side here (sorry to stretch the discussion out).
I was looking for a mechanism to do something like this in the context of this blueprint on network aware scheduling: https://blueprints.launchpad.net/nova/+spec/network-bandwidth-entitlement Essentially the problem is that I need to add network bandwidth resource allocation information just like vcpu, memory and disk space already has. I could hard code this just as they are, but I can also think of a couple of others we would like to add that may be more specific to a given installation. So I could do with a generic way to feed this information back from the compute node to the scheduler just like this.
However, my use case is not the same - it is not meant to be for monitored/statistical utilization info. But I would like a similar mechanism to allow the scheduler to keep track of more general / extensible resource allocation.
Do you have any thoughts on that? Again, don't mean to deflect the discussion - just I have another use case.
>From: Sean Dague [mailto:sean at dague.net]
>Sent: 18 July 2013 12:05
>To: OpenStack Development Mailing List
>Subject: Re: [openstack-dev] [Nova] New DB column or new DB table?
>On 07/17/2013 10:54 PM, Lu, Lianhao wrote:
>> Hi fellows,
>> Currently we're implementing the BP
>The main idea is to have an extensible plugin framework on nova-compute
>where every plugin can get different metrics(e.g. CPU utilization,
>memory cache utilization, network bandwidth, etc.) to store into the
>DB, and the nova- scheduler will use that data from DB for scheduling decision.
>> Currently we adds a new table to store all the metric data and have
>scheduler join loads the new table with the compute_nodes table to get
>all the data(https://review.openstack.org/35759). Someone is concerning
>about the performance penalty of the join load operation when there are
>many metrics data stored in the DB for every single compute node. Don
>suggested adding a new column in the current compute_nodes table in DB,
>and put all metric data into a dictionary key/value format and store
>the json encoded string of the dictionary into that new column in DB.
>> I'm just wondering which way has less performance impact, join load
>> with a
>new table with quite a lot of rows, or json encode/decode a dictionary
>with a lot of key/value pairs?
>I'm really confused. Why are we talking about collecting host metrics
>in nova when we've got a whole project to do that in ceilometer? I
>think utilization based scheduling would be a great thing, but it
>really out to be interfacing with ceilometer to get that data. Storing
>it again in nova (or even worse collecting it a second time in nova) seems like the wrong direction.
>I think there was an equiv patch series at the end of Grizzly that was
>pushed out for the same reasons.
>If there is a reason ceilometer can't be used in this case, we should
>have that discussion here on the list. Because my initial reading of
>this blueprint and the code patches is that it partially duplicates
>ceilometer function, which we definitely don't want to do. Would be happy to be proved wrong on that.
>OpenStack-dev mailing list
>OpenStack-dev at lists.openstack.org
More information about the OpenStack-dev