Open Stack

On 10/12/15, 12:05 PM, "Monty Taylor" <mordred at inaugust.com> wrote:

>On 10/12/2015 02:45 PM, Joshua Harlow wrote:
>> Alec Hothan (ahothan) wrote:
>>>
>>>
>>>
>>>
>
>I want to do 100k hypervisors. No, that's not hyperbole.
>
>Also, I do not think that ZK/consul/etcd are very costly for small 
>deployments. Given the number of simple dev-oriented projects that start 
>with "so install ZK/consul/etcd" I think they've all proven their 
>ability to scale _down_ - and I'm also pretty sure all of them have 
>installations that clear 100k nodes.
>
>This:
>
>to produce the ubiquitous Open Source Cloud Computing platform that will 
>meet the needs of public and private clouds regardless of size, by being 
>simple to implement and massively scalable.
>
>is what we're doing.
>
>Our mission is NOT "produce a mid-range cloud that is too complex for 
>small deployments and tops out before you get to big ones"
>
>I don't think "handle massive clouds" has ever NOT been on the list of 
>stated goals. (that mission statement has not changed since we started 
>the project - although I agree with Joe, it's in need of an update- 
>there is no mention of users)

Then it'd be great that there be an official statement from the TC about the scale objectives and if possible put some numbers, "massive cloud" is ambiguous for folks who actually have to make sure they scale to specs.
So should mention "OpenStack should scale from 1 node to 100K nodes" for example. As long as everybody is fully aware about how far we are today from that lofty goal.
This clearly will have an impact on how we need to design services and how we should change the way we test for them. It will be tricky to get a 1000 node lab up and running just for openstack developers, it is just not practical at all. The only practical way will be to do proper unit testing at scale (e.g. emulate a 10K node cloud for unit testing any given service).


>
>BTW - Infra runs against currently runs against clouds rate-limited at 
>roughly 10 api calls / second. That's just one tenant - but it's a 
>perfectly managable rate. Now, if the cloud could continue to add nodes 
>and users without that rate degrading I think we'd be in really good shape.

I think that rate limit only applies to REST APIs, I don't think there is any rate limit for oslo messaging.
Even only 10 API calls per second per tenant can be a challenge with a large number of tenants. I don't think there is any provision today for example to ensure fairness across tenants.