<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Sep 7, 2017 at 4:05 PM, Sahid Orentino Ferdjaoui <span dir="ltr"><<a href="mailto:sferdjao@redhat.com" target="_blank">sferdjao@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span class="gmail-">On Wed, Sep 06, 2017 at 11:57:25PM -0400, Jay Pipes wrote:<br>
> Sahid, Stephen, what are your thoughts on this?<br>
><br>
> On 09/06/2017 10:17 PM, Yaguang Tang wrote:<br>
> > I think the fact that RamFilter can't deal with huge pages is a bug ,<br>
> > duo to this limit, we have to set a balance between normal memory and<br>
> > huge pages to use RamFilter and NUMATopologyFilter. what do you think<br>
> > Jay?<br>
<br>
</span>Huge Pages has been built on top of the NUMA Topology<br>
implementation. You have to consider isolate all hosts which are going<br>
to handle NUMA instances to a specific host aggregate.<br>
<br>
We don't want RAMFilter to handle any NUMA related feature (in Nova<br>
world: hugepages, pinning, realtime...) but we don't want it blocking<br>
scheduling and that should not be the case.<br>
<br>
I'm surprised to see this bug I think libvirt is reporting the total<br>
amount of physical RAM available on the host and that is not depending<br>
of the size of the pages.<br>
<br>
So If that is true libvirt is reporting only the amount of small pages<br>
memory available we will probably have to fix that point instead of<br>
the RAMFilter.<br></blockquote><div> </div><div> If we fix it this way, the issue still exists, as long as user use RamFilter and <span style="color:rgb(80,0,80);font-size:12.8px">NUMATopologyFilter toghther. Libvirt reports total memory which contains huge pages </span></div><div>but normal instance can't use. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
That because we don't want to add "hack" on all filters to just pass<br>
when instances provide NUMA Topology constraints and we can't<br>
configure the scheduler to use specific filters per aggregate.<br></blockquote><div><br></div><div>How about fixing RamFilter to just pass if the instance extra specs specifies huge pages ?</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<span class="gmail-HOEnZb"><font color="#888888"><br>
s.<br>
</font></span><div class="gmail-HOEnZb"><div class="gmail-h5"><br>
> ><br>
> > On Wed, Sep 6, 2017 at 9:22 PM, Jay Pipes <<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a><br>
> > <mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>>> wrote:<br>
> ><br>
> > On 09/06/2017 01:21 AM, Weichih Lu wrote:<br>
> ><br>
> > Thanks for your response.<br>
> ><br>
> > Is this mean if I want to create an instance with flavor: 16G<br>
> > memory (hw:mem_page_size=large), I need to preserve memory more<br>
> > than 16GB ?<br>
> > This instance consume hugepages resource.<br>
> ><br>
> ><br>
> > You need to reserve fewer 1GB huge pages than 50 if you want to<br>
> > launch a 16GB instance on a host with 64GB of RAM. Try reserving 32<br>
> > 1GB huge pages.<br>
> ><br>
> > Best,<br>
> > -jay<br>
> ><br>
> > 2017-09-06 1:47 GMT+08:00 Jay Pipes <<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a><br>
> > <mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>> <mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a><br>
> > <mailto:<a href="mailto:jaypipes@gmail.com">jaypipes@gmail.com</a>>>>:<br>
> ><br>
> ><br>
> > Please remember to add a topic [nova] marker to your<br>
> > subject line.<br>
> > Answer below.<br>
> ><br>
> > On 09/05/2017 04:45 AM, Weichih Lu wrote:<br>
> ><br>
> > Dear all,<br>
> ><br>
> > I have a compute node with 64GB ram. And I set 50<br>
> > hugepages wiht<br>
> > 1GB hugepage size. I used command "free", it shows free<br>
> > memory<br>
> > is about 12GB. And free hugepages is 50.<br>
> ><br>
> ><br>
> > Correct. By assigning hugepages, you use the memory<br>
> > allocated to the<br>
> > hugepages.<br>
> ><br>
> > Then I launch an instance with 16GB memory, set flavor<br>
> > tag :<br>
> > hw:mem_page_size=large. It show Error: No valid host<br>
> > was found.<br>
> > There are not enough hosts available.<br>
> ><br>
> ><br>
> > Right, because you have only 12G of RAM available after<br>
> > creating/allocating 50G out of your 64G.<br>
> ><br>
> > Huge pages are entirely separate from the normal memory that a<br>
> > flavor consumes. The 16GB memory in your flavor is RAM<br>
> > consumed on<br>
> > the host. The huge pages are individual things that are<br>
> > consumed by<br>
> > the NUMA topology that your instance will take. RAM != huge<br>
> > pages.<br>
> > Totally different things.<br>
> ><br>
> > And I check nova-scheduler log. My<br>
> ><br>
> > compute is removed by RamFilter. I can launch an<br>
> > instance with<br>
> > 8GB memory successfully, or I can launch an instance<br>
> > with 16GB<br>
> > memory sucessfully by remove RamFilter.<br>
> ><br>
> ><br>
> > That's because RamFilter doesn't deal with huge pages.<br>
> > Because huge<br>
> > pages are a different resource than memory. The page itself<br>
> > is the<br>
> > resource.<br>
> ><br>
> > The NUMATopologyFilter is the scheduler filter that<br>
> > evaluates the<br>
> > huge page resources on a compute host and determines if the<br>
> > there<br>
> > are enough *pages* available for the instance. Note that I say<br>
> > *pages* because the unit of resource consumption for huge<br>
> > pages is<br>
> > not MB of RAM. It's a single memory page.<br>
> ><br>
> > Please read this excellent article by Steve Gordon for<br>
> > information<br>
> > on what NUMA and huge pages are and how to use them in Nova:<br>
> ><br>
> > <a href="http://redhatstackblog.redhat.com/2015/09/15/driving-in-the-fast-lane-huge-page-support-in-openstack-compute/" rel="noreferrer" target="_blank">http://redhatstackblog.redhat.<wbr>com/2015/09/15/driving-in-the-<wbr>fast-lane-huge-page-support-<wbr>in-openstack-compute/</a><br>
> > <<a href="http://redhatstackblog.redhat.com/2015/09/15/driving-in-the-fast-lane-huge-page-support-in-openstack-compute/" rel="noreferrer" target="_blank">http://redhatstackblog.<wbr>redhat.com/2015/09/15/driving-<wbr>in-the-fast-lane-huge-page-<wbr>support-in-openstack-compute/</a>><br>
> > <<a href="http://redhatstackblog.redhat.com/2015/09/15/driving-in-the-fast-lane-huge-page-support-in-openstack-compute/" rel="noreferrer" target="_blank">http://redhatstackblog.<wbr>redhat.com/2015/09/15/driving-<wbr>in-the-fast-lane-huge-page-<wbr>support-in-openstack-compute/</a><br>
> > <<a href="http://redhatstackblog.redhat.com/2015/09/15/driving-in-the-fast-lane-huge-page-support-in-openstack-compute/" rel="noreferrer" target="_blank">http://redhatstackblog.<wbr>redhat.com/2015/09/15/driving-<wbr>in-the-fast-lane-huge-page-<wbr>support-in-openstack-compute/</a>><wbr>><br>
> ><br>
> > Best,<br>
> > -jay<br>
> ><br>
> > Does RamFilter only check free memory but not free<br>
> > hugepages?<br>
> > How can I solve this problem?<br>
> ><br>
> > I use openstack mitaka version.<br>
> ><br>
> > thanks<br>
> ><br>
> > WeiChih, Lu.<br>
> ><br>
> > Best Regards.<br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > Mailing list:<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>>><br>
> > Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>>><br>
> > Unsubscribe :<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>>><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > Mailing list:<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>>><br>
> > Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>>><br>
> > Unsubscribe :<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>>><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > Mailing list:<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>><br>
> > Unsubscribe :<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > Mailing list:<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> > Post to : <a href="mailto:openstack@lists.openstack.org">openstack@lists.openstack.org</a><br>
> > <mailto:<a href="mailto:openstack@lists.openstack.org">openstack@lists.<wbr>openstack.org</a>><br>
> > Unsubscribe :<br>
> > <a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a><br>
> > <<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack" rel="noreferrer" target="_blank">http://lists.openstack.org/<wbr>cgi-bin/mailman/listinfo/<wbr>openstack</a>><br>
> ><br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > Tang Yaguang<br>
> ><br>
> ><br>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div>Tang Yaguang</div><div><br></div><br><div> </div></div></div>
</div></div>