[Openstack-operators] Lets talk capacity monitoring

matt matt at nycresistor.com
Thu Jan 15 23:29:54 UTC 2015


I've found histograms to be pretty useful in figuring out patterns during
sizable time deltas... and anomaly detection there can highlight stuff you
might want to check out ( ie raise the alert condition on that device ).

example of a histogram i did many many moons ago to track disk sizes from
our nagios plugin that did dynamic disk free analytics.  I don't have any
of the animated GIFs I made that showed fluctuations over days... but that
was great from a human visual sense.

I suppose this could be further automated and refined, I've not been
focused here anymore though.

-Matt

On Thu, Jan 15, 2015 at 3:08 PM, George Shuklin <george.shuklin at gmail.com>
wrote:

> On 01/15/2015 06:43 PM, Jesse Keating wrote:
>
>> We have a need to better manage the various openstack capacities across
>> our numerous clouds. We want to be able to detect when capacity of one
>> system or another is approaching the point where it would be a good idea to
>> arrange to increase that capacity. Be it volume space, VCPU capability,
>> object storage space, etc...
>>
>> What systems are you folks using to monitor and react to such things?
>>
>>
> In our case we are using standard metrics (ganglia) and monitoring
> (shinken). I have thoughts about 'capacity planing', but the problem is
> that you cannot separate payload from wasted resources. For example, when
> snapshot is created, it eats space on compute (for some configuration)
> beyond flavor limits. If instance boots, _base is used too (and if instance
> is booting from big snapshot, it use more space in _base, than in
> /instances). CPU can be heavily used by many host-internal processes, and
> memory is shared with management software (which can be greedy too). IO can
> be overspend on snapshots/booting.
>
> So we are using cumulative graphs for free space, cpu usage, memory usage.
> It does not cover flavor/aggregate/pinning-to-host-by-metadata cases, but
> overall give some feeling about available free resources.
>
>
> _______________________________________________
> OpenStack-operators mailing list
> OpenStack-operators at lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150115/59c2092e/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: omega-disk-hist.png
Type: image/png
Size: 13933 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150115/59c2092e/attachment-0001.png>


More information about the OpenStack-operators mailing list