Open Stack

On 4 May 2017, at 19:26, Joshua Harlow <harlowja at fastmail.com<mailto:harlowja at fastmail.com>> wrote:

So just wanted to throw this email out there to explore the area a little bit (since it's an ongoing issue that keeps on getting raised at godaddy),

The basic question is how as a cloud operator (or developer in a cloud group) do you recommend to your superiors (or people ordering hardware or network gear and so-on) that they need more cloud capacity?

It doesn't appear that nova has a 'leftover capacity for XYZ flavor' api or cinder a 'how many more volumes at X or Y or Z size' (or neutron with networks or IPs ...) so that makes me wonder how are people predicting when they need more hardware (or a more networks or more volume storage or more floating ips or ... and so on).

At godaddy we have this concept of a capacity dashboard that was doing all sorts of things to try to predict remaining capacity (if people really want this kind of code I can opensource it, though I will say it is *not* pretty).

For example (image attached of what one of these dashboards looks like):

- For a given set of flavors, how many more instances can be created with that flavor before no capacity in the cloud is left (ie giving people planning capacity a useful date when no more capacity will be left).
- Same as ^ with floating ips (static ips also), ram, vcpus, ...

Though part of what the above is missing is that same information per-tenant/project, per-user and such (ie various slicing and dicing of the same data)...

Though this dashboard and its associated code/scripts was/is basically scrapping a bunch of the various openstack APIs, sending that information into hadoop and then using hive to extract all this various information. Though technically not horrible it does seem like the various openstack project APIs should provide there own projections of this same data (without needing to scape it, send it to hadoop and then do various projections there).

So general kind of question, how are people (other cloud operators or developers) figuring out this information? Is there something that others would like to get natively integrated into projects? Is there work already ongoing to do that?


This is also an area of concern for us. Tracking the usage is feasible (home written tools also) but predicting the future with the Tetris scheduling problems (i.e. lots of cores just too distributed, one availability zone getting full but space in another). There was an interesting article today regarding similar problems in Azure so it seems a common problem (https://www.theregister.co.uk/2017/05/04/microsoft_azure_capacity_woes_hit_uk_customers/)

I don’t think there is a best practise agreement yet on how to proceed so I think external scripts will have to do. Like you, the scripting is done to solve the local problem rather than be generic but we may find enough overlap to at least gather the data consistently.

Would this fit into one of the forum sessions? Maybe the Large Deployment Team ones?

Tim

-Josh
<flavor_capacity.png>_______________________________________________
OpenStack-operators mailing list
OpenStack-operators at lists.openstack.org<mailto:OpenStack-operators at lists.openstack.org>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20170504/17088c5c/attachment.html>