<html><head><meta http-equiv="Content-Type" content="text/html charset=iso-8859-1"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">I think Soren suggested this way back in Cactus to use MQ for compute node state rather than database and it was a good idea then. <div><div apple-content-edited="true"><span class="Apple-style-span" style="border-collapse: separate; color: rgb(0, 0, 0); font-family: Helvetica; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; line-height: normal; orphans: 2; text-align: -webkit-auto; text-indent: 0px; text-transform: none; white-space: normal; widows: 2; word-spacing: 0px; -webkit-border-horizontal-spacing: 0px; -webkit-border-vertical-spacing: 0px; -webkit-text-decorations-in-effect: none; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px; font-size: medium; "><div><div style="font-size: 12px; "><br></div></div></span></div><div><div>On Jul 19, 2013, at 10:52 AM, Boris Pavlovic <<a href="mailto:boris@pavlovic.me">boris@pavlovic.me</a>> wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div dir="ltr"><span id="docs-internal-guid-27e8af13-f765-72ec-899e-de785211dcd2"><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">Hi all, </div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><br></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><br></div><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">In Mirantis Alexey Ovtchinnikov and me are working on nova scheduler improvements.</span></div>
<br><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">As far as we can see the problem, now scheduler has two major issues:</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "><br></span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">1) Scalability. Factors that contribute to bad scalability are these:</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "> *) Each compute node every periodic task interval (60 sec by default) updates resources state in DB.</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "> *) On every boot request scheduler has to fetch information about all compute nodes from DB. </span></div>
<br><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">2) Flexibility. Flexibility perishes due to problems with:</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="background-color: transparent; font-family: Arial; font-size: 15px; white-space: pre-wrap; line-height: 1.15; "> *) Addiing new complex resources (such as big lists of complex objects e.g. required by PCI Passthrough <a href="https://review.openstack.org/#/c/34644/5/nova/db/sqlalchemy/models.py">https://review.openstack.org/#/c/34644/5/nova/db/sqlalchemy/models.py</a>)</span><br>
</div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "> *) Using different sources of data in Scheduler for example from cinder or ceilometer.</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">(as required by Volume Affinity Filter <a href="https://review.openstack.org/#/c/29343/">https://review.openstack.org/#/c/29343/</a>)</span></div>
<div><span><br></span></div><br><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">We found a simple way to mitigate this issues by avoiding of DB usage for host state storage.</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "><br></span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">A more detailed discussion of the problem state and one of a possible solution can be found here:</span></div>
<div><br></div><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "><a href="https://docs.google.com/document/d/1_DRv7it_mwalEZzLy5WO92TJcummpmWL4NWsWf0UWiQ/edit#">https://docs.google.com/document/d/1_DRv7it_mwalEZzLy5WO92TJcummpmWL4NWsWf0UWiQ/edit#</a></span></div>
<div><span><br></span></div><br><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">Best regards,</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">Boris Pavlovic</span></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; ">
<br></div><div style="line-height: 1.15; margin-top: 0pt; margin-bottom: 0pt; "><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; ">Mirantis Inc. </span></div>
<br><span style="font-size: 15px; font-family: Arial; background-color: transparent; vertical-align: baseline; white-space: pre-wrap; "></span></span></div>
_______________________________________________<br>OpenStack-dev mailing list<br><a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev<br></blockquote></div><br></div></body></html>