<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">When I was looking at bugs related to race conditions of scheduler [1-3], it feels like nova scheduler lacks sanity checks of schedule decisions according to different situations. We cannot even make sure that some fixes successfully mitigate
race conditions to an acceptable scale. For example, there is no easy way to test whether server-group race conditions still exists after a fix for bug[1], or to make sure that after scheduling there will be no violations of allocation ratios reported by bug[2],
or to test that the retry rate is acceptable in various corner cases proposed by bug[3]. And there will be much more in this list.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So I'm asking whether there is a plan to add those tests in the future, or is there a design exist to simplify writing and executing those kinds of tests? I'm thinking of using fake databases and fake interfaces to isolate the entire scheduler
service, so that we can easily build up a disposable environment with all kinds of fake resources and fake compute nodes to test scheduler behaviors. It is even a good way to test whether scheduler is capable to scale to 10k nodes without setting up 10k real
compute nodes.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I'm also interested in the bp[4] to reduce scheduler race conditions in green-thread level. I think it is a good start point in solving the huge racing problem of nova scheduler, and I really wish I could help on that.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] <a href="https://bugs.launchpad.net/nova/+bug/1423648">https://bugs.launchpad.net/nova/+bug/1423648</a>
<o:p></o:p></p>
<p class="MsoNormal">[2] <a href="https://bugs.launchpad.net/nova/+bug/1370207">https://bugs.launchpad.net/nova/+bug/1370207</a>
<o:p></o:p></p>
<p class="MsoNormal">[3] <a href="https://bugs.launchpad.net/nova/+bug/1341420">https://bugs.launchpad.net/nova/+bug/1341420</a>
<o:p></o:p></p>
<p class="MsoNormal">[4] <a href="https://blueprints.launchpad.net/nova/+spec/host-state-level-locking">
https://blueprints.launchpad.net/nova/+spec/host-state-level-locking</a> <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regards,<o:p></o:p></p>
<p class="MsoNormal">-Yingxin<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>