Open Stack

On 03/15/2017 01:48 PM, John Garbutt wrote:
> On 15 March 2017 at 12:33, Sean Dague <sean at dague.net> wrote:
>> On 03/15/2017 08:10 AM, John Garbutt wrote:
>>> On 15 March 2017 at 11:58, Jay Pipes <jaypipes at gmail.com> wrote:
>>>> On 03/15/2017 07:44 AM, Sean Dague wrote:
>>>>>
>>>>> On 03/14/2017 11:00 PM, Monty Taylor wrote:
>>>>> <snip>
>>>>>>
>>>>>> a) awesome. when the rest of this dips momentarily into words that might
>>>>>> sound negative, please hear it all wrapped in an "awesome" and know that
>>>>>> my personal desire is to see the thing you're working on be successful
>>>>>> without undue burden...
>>>>>>
>>>>>> b) In Tokyo, we had the big discussion about DLMs (where at least my
>>>>>> intent going in to the room was to get us to pick one and only one).
>>>>>> There were three camps in the room who were all vocal:
>>>>>>
>>>>>> 1) YES! Let's just pick one, I don't care which one
>>>>>> 2) I hate Java I don't want to run Zookeeper, so we can't pick that
>>>>>> 3) I hate go/don't trust coreos I don't want to run etcd so we can't
>>>>>> pick that
>>>>>>
>>>>>> Because of 2 and 3 the group represented by 1 lost and we ended up with:
>>>>>> "crap, we have to use an abstraction library"
>>>>>>
>>>>>> I'd argue that unless something has changed significantly, having Nova
>>>>>> grow a direct depend on etcd when the DLM discussion brought us to "the
>>>>>> operators in the room have expressed a need for a pluggable choice
>>>>>> between at least zk and etcd" should be pretty much a non-starter.
>>>>>>
>>>>>> Now, being that I was personally in group 1, I'd be THRILLED if we
>>>>>> could, as a community, decide to pick one and skip having an abstraction
>>>>>> library. I still don't care which one - and you know I love
>>>>>> gRPC/protobuf.
>>>>>>
>>>>>> But I do think that given the anti-etcd sentiment that was expressed was
>>>>>> equally as vehement as the anti-zk sentiment, that we need to circle
>>>>>> back and make a legit call on this topic.
>>>>>>
>>>>>> If we can pick one, I think having special-purpose libraries like
>>>>>> os-lively for specific purposes would be neat.
>>>>>>
>>>>>> If we still can't pick one, then I think adding the liveness check you
>>>>>> implemented for os-lively as a new feature in tooz and also implementing
>>>>>> the same thing in the zk driver would be necessary. (of course, that'll
>>>>>> probably depend on getting etcd3 support added to tooz and making sure
>>>>>> there is a good functional test for etcd3.
>>>>>
>>>>>
>>>>> We should also make it clear that:
>>>>>
>>>>> 1) Tokyo was nearly 1.5 years ago.
>>>>> 2) Many stake holders in openstack with people in that room may no
>>>>> longer be part of our community
>>>>> 3) Alignment with Kubernetes has become something important at many
>>>>> levels inside of OpenStack (which puts some real weight on the etcd front)
>>>>
>>>>
>>>> Yes, and even more so for etcd3 vs. etcd2, since a) k8s now uses etcd3 and
>>>> b) etcd2 is no longer being worked on.
>>>>
>>>>> 4) The containers ecosystem, which etcd came out of, has matured
>>>>> dramatically
>>>
>>> +1 for working towards etcd3 a "base service", based on operator acceptance.
>>> +1 for liveness checks not causing silly DB churn.
>>>
>>> While we might not need/want an abstraction layer to hide the
>>> differences between different backends, but a library (tooz and/or
>>> os-lively) so we all consistently use the tool seems to make sense.
>>>
>>> Maybe that means get tooz using etcd3 (Julian or Jay, or both maybe
>>> seemed keen?)
>>> Maybe the tooz API adds bits from the os-lively POC?
>>
>> I do have a concern where we immediately jump to a generic abstraction,
>> instead of using the underlying technology to the best of our ability.
>> It's really hard to break down and optimize the abstractions later.
>> We've got all sorts of cruft (an inefficiencies) in our DB access layer
>> because of this (UUIDs stored as UTF8 strings being a good example).
>>
>> I'd definitely be more interested in etcd3 as a defined base service,
>> people can use it directly. See what kind of patterns people come up
>> with. Abstract late once the patterns are there.
> 
> Good point.
> 
> +1 to collecting the patterns.
> Thats the bit I didn't want to throw away.

++

If the 1.5 years has changed us, I think just depending on etcd3 would
be awesome.

FWIW - when we added zk to zuul/nodepool, we chose a single system
(although the opposite system, but that's not the interesting thing
here) for all the reasons mentioned about being able to dive deep into
the actual tool. It has worked well for us. However, we did wind up
writing an in-tree API on top of kazoo in-tree almost immediately, just
because it made working with it for the logical operations we needed
more sense elsewhere in the code.

So - yeah - use the tech without a generic abstraction - but patterns
can help a bunch.