<div dir="ltr">I am struggling with python code profiling in general. It has its own caveats like 100% plus overhead.<div>However, on a host with only nova services (DB on a different host), I see cpu utilization spike up quickly with scale. The DB server is relatively calm and never goes over 20%. On a system which relies on DB to fetch all the data, this should not happen.</div><div><br></div><div>I could not find any analysis of nova performance either. Appreciate if someone can point me to one.</div><div><br></div><div>Thanks,</div><div><br></div><div><br></div><div><br></div><div><div><div><br></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 11, 2015 at 3:57 PM, Chris Friesen <span dir="ltr"><<a href="mailto:chris.friesen@windriver.com" target="_blank">chris.friesen@windriver.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Just curious...have you measured this consuming a significant amount of CPU time? Or is it more a gut feel of "this looks like it might be expensive"?<br>
<br>
Chris<span class=""><br>
<br>
<br>
On 08/11/2015 04:51 PM, Sachin Manpathak wrote:<br>
</span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">
Here are a few --<br>
instance_get_all_by_filters joins manually with<br>
instances_fill_metadata --<br>
<a href="https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L1890" rel="noreferrer" target="_blank">https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L1890</a><br>
<a href="https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L1782" rel="noreferrer" target="_blank">https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L1782</a><br>
<br>
Almost all instance query functions manually join with instance_metadata.<br>
<br>
Another example was compute_node_get_all function which joined compute_node,<br>
services and ip tables. But it is simplified in current codebase (I am working<br>
on Juno)<br>
<br>
<br>
<br>
<br>
On Tue, Aug 11, 2015 at 3:09 PM, Clint Byrum <<a href="mailto:clint@fewbar.com" target="_blank">clint@fewbar.com</a><br></span><span class="">
<mailto:<a href="mailto:clint@fewbar.com" target="_blank">clint@fewbar.com</a>>> wrote:<br>
<br>
Excerpts from Sachin Manpathak's message of 2015-08-12 05:40:36 +0800:<br>
> Hi folks,<br>
> Nova codebase seems to follow manual joins model where all data required by<br>
> an API is fetched from multiple tables and then joined manually by using<br>
> (in most cases) python dictionary lookups.<br>
><br>
> I was wondering about the basis reasoning for doing so. I usually find<br>
> openstack services to be CPU bound in a medium sized environment and<br>
> non-trivial utilization seems to be from parts of code which do manual<br>
> joins.<br>
<br>
Could you please cite specific examples so we can follow along with your<br>
thinking without having to repeat your analysis?<br>
<br>
Thanks!<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br></span>
<<a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a>><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><span class=""><br>
<br>
<br>
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br>
</span></blockquote><div class="HOEnZb"><div class="h5">
<br>
<br>
__________________________________________________________________________<br>
OpenStack Development Mailing List (not for usage questions)<br>
Unsubscribe: <a href="http://OpenStack-dev-request@lists.openstack.org?subject:unsubscribe" rel="noreferrer" target="_blank">OpenStack-dev-request@lists.openstack.org?subject:unsubscribe</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" rel="noreferrer" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
</div></div></blockquote></div><br></div>