<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
Hi all,
<div class=""><br class="">
</div>
<div class="">We’re running into an issue with deploying our infrastructure to run high throughput, low latency workloads.</div>
<div class=""><br class="">
</div>
<div class="">Background:</div>
<div class=""><br class="">
</div>
<div class="">We run Lenovo SR635 systems with an AMD Epyc 7502P processor. In the BIOS of this system, we are able to define the amount of NUMA cells per socket (called NPS). We can set 1, 2 or 4. As we run a 2x 100Gbit/s Mellanox CX5 in this system as well,
we use the preferred-io setting in the BIOS to give preferred io throughput to the Mellanox CX5.</div>
<div class="">To make sure we get as high performance as possible, we set the NPS setting to 1, resulting in a single numa cell with 64 CPU threads available.</div>
<div class=""><br class="">
</div>
<div class="">Next, in Nova (train distribution), we demand huge pages. Hugepages however, demands a NUMAtopology, but as this is one large NUMA cell, even with cpu=dedicated or requesting a single numa domain, we fail:</div>
<div class=""><br class="">
</div>
<div class=""><span style="color: rgb(55, 56, 58); font-family: Slack-Lato, appleLogo, sans-serif; font-size: 15px; font-variant-ligatures: common-ligatures; orphans: 2; widows: 2; background-color: rgb(255, 255, 255);" class="">compute03, compute03 fails NUMA
topology requirements. No host NUMA topology while the instance specified one. host_passes /usr/lib/python3/dist-packages/nova/scheduler/filters/numa_topology_filter.py:119</span></div>
<div class=""><br class="">
</div>
<div class="">
<div class="">Any idea how to counter this? Setting NPS-2 will create two NUMA domains, but also cut our performance way down.</div>
<div class=""><br class="webkit-block-placeholder">
</div>
<div class="">Thanks!</div>
<div class=""><br class="">
</div>
<div class="">
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
Regards,</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<br class="">
</div>
<div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
Eyle</div>
</div>
<br class="">
</div>
</body>
</html>