<div dir="ltr">I know we've been working on that on our commercial product side at big switch with an analyzer... the issue i think you are going to run into is getting insight into network upstream info from your top of racks and spine switches.<div><br></div><div>Setting up a uniform access to ovs stats in the API or in an external API ( probably preferable ) is not a bad idea.</div><div><br></div><div>Way I see it, you might want to consider an external ( not a full project in OpenStack ) api for aggregating ovs stats ( use the message bus to cull that data ). whether you want to try to make use of ceilometer or monaas is really up to you, I'd recommend only using ceilometer if you are using zones. Then making a display panel pluggable to horizon is fairly straight forward with d3.js</div><div><br></div><div>Off hand I don't know of a specific project targetting this, but it could shoehorn into projects like ceilometer or monass. Also future tap as a service work that's starting to occur in juno now may be super helpful as well. </div><div><br></div><div>This looks like it WILL exist... how it exists and what it ties into is probably waiting on a bit more definition and execution commitment from other projects. Unless you happen to be using zones and want to augment ceilometer.</div><div><br></div><div>-Matt</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jan 15, 2015 at 9:25 AM, Mathieu Gagné <span dir="ltr"><<a href="mailto:mgagne@iweb.com" target="_blank">mgagne@iweb.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 2015-01-15 11:43 AM, Jesse Keating wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
We have a need to better manage the various openstack capacities across<br>
our numerous clouds. We want to be able to detect when capacity of one<br>
system or another is approaching the point where it would be a good idea<br>
to arrange to increase that capacity. Be it volume space, VCPU<br>
capability, object storage space, etc...<br>
<br>
What systems are you folks using to monitor and react to such things?<br>
<br>
</blockquote>
<br></span>
Thanks for bringing up the subject Jesse.<br>
<br>
I believe you are not the only one facing this challenge because I am too.<br>
<br>
I added the subject to the midcycle ops meetup (Capacity planning/monitoring) which I hope to be able to attend:<br>
<a href="https://etherpad.openstack.org/p/PHL-ops-meetup" target="_blank">https://etherpad.openstack.<u></u>org/p/PHL-ops-meetup</a><br>
<br>
<br>
We are using host aggregates and have a complex combination of them. (imaging a venn diagram)<br>
<br>
What we do is retrieving all:<br>
- hypervisor stats<br>
- host aggregates<br>
<br>
>From there, we compute resource usage (vcpus, ram, disk) in any given host aggregate.<br>
<br>
This part is very challenging as we have to partially reimplement nova-scheduler logic to determine if a given hypervisor has different resource allocation ratios based on host aggregate attributes.<br>
<br>
The result in a table with resource usage percentage (and absolute numbers) for each host aggregates (and combinations).<br>
<br>
Unfortunately, I can't share yet this first tool as my coworker very tightly integrated it to our internal monitoring tool and wouldn't work outside it. No promise but I'll try to find time to extract it and share it with you guys.<br>
<br>
<br>
We also coded a very primitive tool which takes a flavor name and compute available "slots" on each hypervisors (regardless of host aggregate memberships):<br>
<br>
<a href="https://gist.github.com/mgagne/bc54c3434a119246a88d" target="_blank">https://gist.github.com/<u></u>mgagne/bc54c3434a119246a88d</a><br>
<br>
This tool is not actively used in our monitoring due to mentioned limitation as we would again have to partially reimplement nova-scheduler logic to determine if a given flavor can (or not) be spawn on a given hypervisor and filter it out from the output if it can't accept the flavor. Furthermore, it does not take into account resource allocation ratios based on host aggregates.<br>
<br>
Hopefully, other people will join in and share their tools so we can all improve our OpenStack operations experience.<span class="HOEnZb"><font color="#888888"><br>
<br>
-- <br>
Mathieu</font></span><div class="HOEnZb"><div class="h5"><br>
<br>
______________________________<u></u>_________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org" target="_blank">OpenStack-operators@lists.<u></u>openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators" target="_blank">http://lists.openstack.org/<u></u>cgi-bin/mailman/listinfo/<u></u>openstack-operators</a><br>
</div></div></blockquote></div><br></div>