<div dir="ltr">I think the better way is save meters as a field in resource table. <div><br></div><div>You can look at MongoDB model and may get some ideas. </div><div><br></div><div>Beside above, sql backend can introduce Memcache to improve performance.</div>
<div>IMO, the best way may be redesign the sql model to match workload.</div><div> </div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Sat, Dec 28, 2013 at 6:51 PM, 刘胜 <span dir="ltr"><<a href="mailto:liusheng1175@126.com" target="_blank">liusheng1175@126.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="line-height:1.7;font-size:14px;font-family:arial">Hi all：<div>I have reported a bug about time consuming of “resource-list<span style="line-height:1.7">” in ceilometer CLI：</span></div>
<div><a href="https://bugs.launchpad.net/ceilometer/+bug/1264434" target="_blank">https://bugs.launchpad.net/ceilometer/+bug/1264434</a></div><div><br></div><div>In order to <span style="line-height:1.7">Identify the causes of this </span>phenomenon, <span style="line-height:1.7">I have pdb the codes in my invironment(configured mysql as </span><span style="line-height:1.7">db driver</span><span style="line-height:1.7">):</span></div>
<div><span style="line-height:1.7">the most import part of process</span> of listing resource is implemented in following codes:</div><div><br></div><div>code of get_resources() in /ceilometer/storage/impl_sqlalchemy.py：</div>
<div>…………</div><div><div style="line-height:1.7"> for meter, first_ts, last_ts in <span style="color:rgb(255,0,0)">query.all()</span>:</div><div style="line-height:1.7"> yield api_models.Resource(</div><div style="line-height:1.7">
resource_id=meter.resource_id,</div><div style="line-height:1.7"> project_id=meter.project_id,</div><div style="line-height:1.7"> first_sample_timestamp=first_ts,</div><div style="line-height:1.7">
last_sample_timestamp=last_ts,</div><div style="line-height:1.7"> source=meter.sources[0].id,</div><div style="line-height:1.7"> user_id=meter.user_id,</div><div style="line-height:1.7">
metadata=meter.resource_metadata,<span style></span></div><div style="line-height:1.7"> meter=[</div><div style="line-height:1.7"> api_models.ResourceMeter(</div><div style="line-height:1.7">
counter_name=m.counter_name,</div><div style="line-height:1.7"> counter_type=m.counter_type,</div><div style="line-height:1.7"> counter_unit=m.counter_unit,</div>
<div style="line-height:1.7"> )</div><div style="line-height:1.7"> <span style="color:rgb(255,0,0)">for m in meter.resource.meters</span></div><div style="line-height:1.7"> ],</div>
<div style="line-height:1.7"> )</div><div style="line-height:1.7">The method generate iterator of object of <span style="line-height:1.7"> </span><span style="line-height:1.7">api_models.Resource for ceilometer API to show.</span></div>
<div><span style="line-height:1.7">1.The operation “</span><span style="line-height:1.7">query.all()</span><span style="line-height:1.7">” will query the DB table “meter” with the expression </span>generated forward<span style="line-height:1.7">,</span><span style="line-height:1.7">in my </span><span style="line-height:1.7">invironment the </span><span style="line-height:1.7">DB table “meter” have more than 300000 items, so this operation may consume about 30 seconds;</span></div>
<div><span style="line-height:1.7">2.The </span><span style="line-height:1.7">operation</span><span style="line-height:1.7"> </span><span style="line-height:1.7"> </span><span style="line-height:1.7">"</span><span style="line-height:1.7">for m in meter.resource.meters</span><span style="line-height:1.7">" will </span><span style="line-height:1.7">circulate the meters of this resource . a resource of server </span><span style="line-height:1.7">may have more than 100000 meter iterms in my </span><span style="line-height:1.7">invironment. So the time of whole process is too long. I think the </span><span style="line-height:1.7">meter of </span><span style="line-height:1.7">Resource object can be reduced and I have tested this </span>modification, it is OK for listing resource,and reduce the most time consumption</div>
</div><div><br></div><div>I have noticed that there are many methods of db operation may <span style="line-height:1.7">time consumption. </span></div><div><br></div><div><span style="line-height:1.7">ps: I have configured the ceilometer pulling interval from 600s to 60s in </span>/etc/ceilometer/pipeline.yaml, but the invironment <span style="line-height:1.7">has just run 10 days!</span></div>
<div><br></div><div>I'm a beginner of ceilometer,and want to fix this bug,but I haven't found a suitable way</div><div><span style="font-family:Helvetica,'Microsoft Yahei',verdana;line-height:1.7">may be someone can help me with this?</span></div>
<div><br></div><div style="line-height:1.7"><span style="font-family:Helvetica,'Microsoft Yahei',verdana">Best Regards</span></div><div style="line-height:1.7"><span style="font-family:Helvetica,'Microsoft Yahei',verdana">liusheng</span></div>
<div><br></div></div><br>_______________________________________________<br>
OpenStack-dev mailing list<br>
<a href="mailto:OpenStack-dev@lists.openstack.org">OpenStack-dev@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev" target="_blank">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev</a><br>
<br></blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr">
<p>Best regards,</p><p>Haomai Wang, UnitedStack Inc.</p></div>
</div>