<div dir="ltr"><div>For what it is worth...I was able to code a weigher. By dropping the code below into a .py file in the weights directory and restarting the scheduler, the scheduler will automatically consider if instances in the same group are on a node before placing another instance there. For what it is worth, both the ram weigher and the antiAffinity weigher are on the playing field and so I have set "one node in the same group= 100GB of ram".<br>
<br></div>Thanks,<br>Danny Beutler<br><br>"""<br>AntiAffinityWeigher. Weigh hosts by whether or not they have another host in the same group.<br><br>"""<br><br>from oslo.config import cfg<br>
<br>from nova import db<br>from nova.openstack.common.gettextutils import _<br>from nova.scheduler import weights<br>from nova.scheduler import filters<br>from nova.openstack.common import log as logging<br><br>LOG = logging.getLogger(__name__)<br>
<br>anti_affinity_weight_opts = [<br> cfg.FloatOpt('antiAffinityWeigher_Multiplier',<br> default=1.0,<br> help='Multiplier used for weighing hosts. Negative '<br>
'numbers mean to spread vs stack.'),<br>]<br><br>CONF = cfg.CONF<br>CONF.register_opts(anti_affinity_weight_opts)<br><br><br>class AntiAffinityWeigher(weights.BaseHostWeigher):<br> def _weight_multiplier(self):<br>
"""Override the weight multiplier."""<br> return CONF.antiAffinityWeigher_Multiplier<br><br> def _weigh_object(self, host_state, weight_properties):<br> group_hosts = weight_properties.get('group_hosts') or []<br>
LOG.debug(_("Group anti affinity weigher: check if %(host)s not "<br> "in %(configured)s"), {'host': host_state.host,<br> 'configured': group_hosts})<br>
if group_hosts:<br> group_hosts = weight_properties.get('group_hosts') or []<br> num_instances_in_group = group_hosts.count(host_state.host)<br> LOG.debug(_("Number of instances in the same group on this node %(host)s"), {'host': num_instances_in_group}) <br>
return group_hosts.count(host_state.host) * -100000<br><br> # No groups configured<br> return 0<br><br><br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Aug 4, 2014 at 12:14 PM, Dugger, Donald D <span dir="ltr"><<a href="mailto:donald.d.dugger@intel.com" target="_blank">donald.d.dugger@intel.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Danny-<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">People have been thinking about affinity/anti-affinity scheduling so this is a good area to look at but we might want to think of a general approach that addresses
this and other issues. I know that Yathi & Debo have proposed a BP:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">
<a href="https://blueprints.launchpad.net/nova/+spec/solver-scheduler" target="_blank">https://blueprints.launchpad.net/nova/+spec/solver-scheduler</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">you might want to check it out and see how it relates to your issues.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Don Dugger<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">"Censeo Toto nos in Kansa esse decisse." - D. Gale<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Ph: <a href="tel:303%2F443-3786" value="+13034433786" target="_blank">303/443-3786</a><u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif""> Danny Beutler [mailto:<a href="mailto:dannybeutler@gmail.com" target="_blank">dannybeutler@gmail.com</a>]
<br>
<b>Sent:</b> Friday, August 1, 2014 10:47 AM<br>
<b>To:</b> <a href="mailto:openstack@lists.openstack.org" target="_blank">openstack@lists.openstack.org</a><br>
<b>Subject:</b> [Openstack] Custom Nova Scheduler Weigher<u></u><u></u></span></p><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I am in the process of implementing a custom weigher class. I have created a weigher that prefers hosts which do not have other instances in the same group (think GroupAntiAffinityFilter but for weight).
<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Here is the code for the class:<br>
<br>
# Copyright (c) 2011 OpenStack Foundation<br>
# All Rights Reserved.<br>
#<br>
# Licensed under the Apache License, Version 2.0 (the "License"); you may<br>
# not use this file except in compliance with the License. You may obtain<br>
# a copy of the License at<br>
#<br>
# <a href="http://www.apache.org/licenses/LICENSE-2.0" target="_blank">http://www.apache.org/licenses/LICENSE-2.0</a><br>
#<br>
# Unless required by applicable law or agreed to in writing, software<br>
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT<br>
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the<br>
# License for the specific language governing permissions and limitations<br>
<br>
"""<br>
AntiAffinityWeigher. Weigh hosts by whether or not they have another host in the same group.<br>
<br>
"""<br>
<br>
from oslo.config import cfg<br>
<br>
from nova import db<br>
from nova.openstack.common.gettextutils import _<br>
from nova.scheduler import weights<br>
from nova.scheduler import filters<br>
from nova.openstack.common import log as logging<br>
<br>
LOG = logging.getLogger(__name__)<br>
<br>
anti_affinity_weight_opts = [<br>
cfg.FloatOpt('antiAffinityWeigher_Multiplier',<br>
default=1000.0,<br>
help='Multiplier used for weighing hosts. Negative '<br>
'numbers mean to stack vs spread.'),<br>
]<br>
<br>
CONF = cfg.CONF<br>
CONF.register_opts(anti_affinity_weight_opts)<br>
<br>
<br>
class AntiAffinityWeigher(weights.BaseHostWeigher):<br>
def _weight_multiplier(self):<br>
"""Override the weight multiplier."""<br>
return CONF.antiAffinityWeigher_Multiplier<br>
<br>
def _weigh_object(self, host_state, weight_properties):<br>
group_hosts = weight_properties.get('group_hosts') or []<br>
LOG.debug(_("Group anti affinity Weigher: check if %(host)s not "<br>
"in %(configured)s"), {'host': host_state.host,<br>
'configured': group_hosts})<br>
if group_hosts:<br>
return group_hosts.amount() * 100000<br>
<br>
# No groups configured<br>
return 0<br>
<br>
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I know the python is at least close to correct because the scheduler service wouldn't even restart until it was. After I got the bugs worked out of the module, I added modified the /etc/nova/nova.conf file to
have the custom weigher like so: <br>
scheduler_weight_classes=nova.scheduler.weights.all_weighers,nova.scheduler.AntiAffinityWeigher<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">After restarting the scheduler service I get the following error in the nova logs:<br>
<178>Aug 1 16:46:11 node-25 nova-nova CRITICAL: Class AntiAffinityWeigher cannot be found (['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 31, in import_class\n return getattr(sys.modules[mod_str],
class_str)\n', "AttributeError: 'module' object has no attribute 'AntiAffinityWeigher'\n"])<br>
Traceback (most recent call last):<br>
File "/usr/bin/nova-scheduler", line 10, in <module><br>
sys.exit(main())<br>
File "/usr/lib/python2.6/site-packages/nova/cmd/scheduler.py", line 39, in main<br>
topic=CONF.scheduler_topic)<br>
File "/usr/lib/python2.6/site-packages/nova/service.py", line 257, in create<br>
db_allowed=db_allowed)<br>
File "/usr/lib/python2.6/site-packages/nova/service.py", line 139, in __init__<br>
self.manager = manager_class(host=self.host, *args, **kwargs)<br>
File "/usr/lib/python2.6/site-packages/nova/scheduler/manager.py", line 65, in __init__<br>
self.driver = importutils.import_object(scheduler_driver)<br>
File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 40, in import_object<br>
return import_class(import_str)(*args, **kwargs)<br>
File "/usr/lib/python2.6/site-packages/nova/scheduler/filter_scheduler.py", line 59, in __init__<br>
super(FilterScheduler, self).__init__(*args, **kwargs)<br>
File "/usr/lib/python2.6/site-packages/nova/scheduler/driver.py", line 103, in __init__<br>
CONF.scheduler_host_manager)<br>
File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 40, in import_object<br>
return import_class(import_str)(*args, **kwargs)<br>
File "/usr/lib/python2.6/site-packages/nova/scheduler/host_manager.py", line 297, in __init__<br>
CONF.scheduler_weight_classes)<br>
File "/usr/lib/python2.6/site-packages/nova/loadables.py", line 105, in get_matching_classes<br>
obj = importutils.import_class(cls_name)<br>
File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 35, in import_class<br>
traceback.format_exception(*sys.exc_info())))<br>
ImportError: Class AntiAffinityWeigher cannot be found (['Traceback (most recent call last):\n', ' File "/usr/lib/python2.6/site-packages/nova/openstack/common/importutils.py", line 31, in import_class\n return getattr(sys.modules[mod_str], class_str)\n',
"AttributeError: 'module' object has no attribute 'AntiAffinityWeigher'\n"])<br>
<br>
<br>
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">I have also tried a few different naming conventions such as "AntiAffinityWeigher.AntiAffinityWeigher" and "myWeigher.AntiAffinityWeigher" to no avail.
<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Any help would be greatly appreciated.<br>
<br>
Thanks,<br>
Danny<u></u><u></u></p>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br></div>