Open Stack

Wow, I seem to have unleashed a bunch of pent-up frustration in the community! It's great to see everyone coming forward with their ideas and insights for improving the way Nova (and, by extension, all of OpenStack) can potentially scale.

I do have a few comments on the discussion:

1) This isn't a proposal to simply add some sort of DLM to Nova as a magic cure-all. The concerns about Nova's ability to scale have to do a lot more with the overall internal communication design.

2) I really liked the comment about "made-up numbers". It's so true: we are all impressed by such examples of speed that we sometimes forget whether speeding up X will improve the overall process to any significant degree. The purpose of my original email back in July, and the question I asked at the Nova midcycle, is if we could get some numbers that would be a target to shoot for with any of these experiments. Sure, I could come up with a test that shows a zillion transactions per second, but if that doesn't result in a cloud being able to schedule more efficiently, what's the point?

3) I like the idea of something like ZooKeeper, but my concern is how to efficiently query the data. If, for example, we had records for 100K compute nodes, would it be possible to do the equivalent of "SELECT * FROM resources WHERE resource_type = 'compute' AND free_ram_mb >= 2048 AND …" - well, you get the idea. Are complex data queries possible in ZK? I haven't been able to find that information anywhere.

4) It is true that even in a very large deployment, it is possible to keep all the relevant data needed for scheduling in memory. My concern is how to efficiently search that data, much like in the ZK scenario.

5) Concerns about Cassandra running with OpenJDK instead of the Oracle JVM are troubling. I sent an email about this to one of the people I know at DataStax, but so far have not received a response. And while it would be great to have people contribute to OpenJDK to make it compatible, keep in mind that that would be an ongoing commitment, not just a one-time effort.

6) I remember discussions back in the Austin-Bexar time frame about what Thierry referred to as 'flavor-based schedulers', and they were immediately discounted as not sophisticated enough to handle the sort of complex scheduling requests that were expected. I'd be interested in finding out from the big cloud providers what percentage of their requests would fall into this simple structure, and what percent are more complicated than that. Having hosts listening to queues that they know they can satisfy removes the raciness from the process, although it would require some additional handling for the situation where no host accepts the request. Still, it has the advantage of being dead simple. Unfortunately, this would probably require a bigger architectural change than integrating Cassandra into the Scheduler would.

I hope that those of us who will be at the Tokyo Summit and are interested in these ideas can get together for an informal discussion, and come up with some ideas for grand experiments and reality checks. ;-)

BTW, I started playing around with some ideas, and thought that if anyone wanted to also try Cassandra, I'd write up a quick how-to for setting up a small cluster: http://blog.leafe.com/small-scale-cassandra/. Using docker images makes it a breeze!


-- Ed Leafe





-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151015/d6d9d46c/attachment.pgp>