Open Stack

On 10 November 2015 at 19:24, Kevin Carter <kevin.carter at rackspace.com> wrote:
> Hello all,
>
> The rational behind using a solution like zookeeper makes sense however in reviewing the thread I found myself asking if there was a better way to address the problem without the addition of a Java based solution as the default. While it has been covered that the current implementation would be a reference and that "other" driver support in Tooz would allow for any backend a deployer may want, the work being proposed within devstack [0] would become the default development case thus making it the de-facto standard and I think we could do better in terms of supporting developers and delivering capability.
>
> My thoughts on using Redis+Redislock instead of Java+Zookeeper as the default option:
> * Tooz already support redislock
> * Redis has an established cluster system known for general ease of use and reliability on distributed systems.

I believe Clint already linked to
https://aphyr.com/posts/309-knossos-redis-and-linearizability or
similar - but 'known for general ease of use and reliability' is uhm,
a bold claim. Its worth comparing that (and the other redis writeups)
to this one: https://aphyr.com/posts/291-call-me-maybe-zookeeper. "Use
zookeeper, its mature".

> * Several OpenStack projects already support Redis as a backend option or have extended capabilities using a Redis.
> * Redis can be implemented in RHEL, SUSE, and DEB based systems with ease.
> * Redis is Opensource software licensed under the "three clause BSD license" and would not have any of the same questionable license implications as found when dealing with anything Java.

The openjdk is present on the same linux distributions, and has been
used in both open source and proprietary programs for decades. *what*
license implications are you speaking of?

> * The inclusion of Redis would work on a single node allowing developers to continue work using VMs running on Laptops with 4GB or ram but would also scale to support the multi-controller use case with ease. This would also give developers the ability to work on a systems that will actually resemble production.

I believe you can do this with zookeeper - both single process, or
three processes on one machine to emulate a cluster - very easily.
Quoting http://qnalist.com/questions/29943/java-heap-size-for-zookeeper
- "It's more dependent on your workload than anything. If you're
storing on order of hundreds of small znodes then 1gb is going to [be]
more then fine." Obviously we should test this and confirm it, but
developer efficiency is a key part of any decision, and AFAIK there is
absolutely nothing in the way as far as zookeeper goes. Just like
rabbitmq and openvswitch, its a mature thing, written in a language
other than Python, which needs its own care and feeding (and that
feeding is something like 90% zk specific, not 'java headaches').

> * Redislock will bring with it no additional developer facing language dependencies (Redis is written in ANSI C and works ... without external dependencies [1]) while also providing a plethora of language bindings [2].
>
>
> I apologize for questioning the proposed solution so late into the development of this thread and for not making the summit conversations to talk more with everyone whom worked on the proposal. While the ship may have sailed on this point for now I figured I'd ask why we might go down the path of Zookeeper+Java when a solution with likely little to no development effort already exists, can support just about any production/development environment, has lots of bindings, and (IMHO) would integrate with the larger community easier; many OpenStack developers and deployers already know Redis. With the inclusion of ZK+Java in DevStack and the act of making it the default it essentially creates new hard dependencies one of which is Java and I'd like to avoid that if at all possible; basically I think we can do better.


I think its fine to raise the question, but lets perhaps set some priorities.

1) The default should be mature
2) The default should be suitable for developer use on a modest laptop
3) The default should be suitable for use in the majority of clouds

I believe zk meets all those priorities, and redis does not. Its
possible that etcd and or consul do: though they are much newer and so
perhaps fail on the maturity scale. I'm certain redis does not - at
least, not unless the previously reported defects have been fixed in
the last 2 years.

-Rob



-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud