Open Stack

Yes, the backends were deployed in cluster configuration (the
configurations are available in the appendix).
I'll make a change to the doc to make sure this is reflected properly.

On Fri, Jul 22, 2016 at 11:29 AM, Kevin Benton <kevin at benton.pub> wrote:
> Were the backends (zookeeper, etcd) deployed in a cluster configuration? I
> can't quite tell from the doc.
>
> On Fri, Jul 22, 2016 at 12:58 AM, John Schwarz <jschwarz at redhat.com> wrote:
>>
>> You're right Joshua.
>>
>> Tooz HEAD points to 0f4e1198fdcbd6a29d77c67d105d201ed0fbd9e0.
>>
>> With regards to etcd and zookeeper's versions, they are:
>> zookeeper-3.4.5+28-1.cdh4.7.1.p0.13.el6.x86_64,
>> etcd-2.2.5-2.el7.0.1.x86_64.
>>
>> John.
>>
>> On Thu, Jul 21, 2016 at 8:14 PM, Joshua Harlow <harlowja at fastmail.com>
>> wrote:
>> > Hi John,
>> >
>> > Thanks for gathering this info,
>> >
>> > Do you have the versions of the backend that were used here
>> > (particularly
>> > relevant for etcd which has a new release pretty frequently).
>> >
>> > It'd be useful to capture that info also :)
>> >
>> > John Schwarz wrote:
>> >>
>> >> Hi everyone,
>> >>
>> >> Following [1], a few of us sat down during the last day of the Austin
>> >> Summit and discussed the possibility of adding formal support for
>> >> Tooz, specifically for the locking mechanism it provides. The
>> >> conclusion we reached was that benchmarks should be done to show if
>> >> and how Tooz affects the normal operation of Neutron (i.e. if locking
>> >> a resource using Zookeeper takes 3 seconds, it's not worthwhile at
>> >> all).
>> >>
>> >> We've finally finished the benchmarks and they are available at [2].
>> >> They test a specific case: when creating an HA router a lock-free
>> >> algorithm is used to assign a vrid to a router (this is later used for
>> >> keepalived), and the benchmark specifically checks the effects of
>> >> locking that function with either Zookeeper or Etcd, using the no-Tooz
>> >> case as a baseline. The locking was checked in 2 different ways - one
>> >> which presents no contention (acquire() always succeeds immediately)
>> >> and one which presents contentions (acquire() may block until a
>> >> similar process for the invoking tenant is complete).
>> >>
>> >> The benchmarks show that while using Tooz does raise the cost of an
>> >> operation, the effects are not as bad as we initially feared. In the
>> >> simple, single simultaneous request, using Zookeeper raised the
>> >> average time it took to create a router by 1.5% (from 11.811 to 11.988
>> >> seconds). On the more-realistic case of 6 simultaneous requests,
>> >> Zookeeper raised the cost by 3.74% (from 16.533 to 17.152 seconds).
>> >>
>> >> It is important to note that the setup itself was overloaded - it was
>> >> built on a single baremetal hosting 5 VMs (4 of which were
>> >> controllers) and thus we were unable to go further - for example, 10
>> >> concurrent requests overloaded the server and caused some race
>> >> conditions to appear in the L3 scheduler (bugs will be opened soon),
>> >> so for this reason we haven't tested heavier samples and limited
>> >> ourselves to 6 simultaneous requests.
>> >>
>> >> Also important to note that some kind of race condition was noticed in
>> >> tooz's etcd driver. We've discussed this with the tooz devs and
>> >> provided a patch that is supposed to fix them [3].
>> >> Lastly, races in the L3 HA Scheduler were found and we are yet to dig
>> >> into them and find out their cause - bugs will be opened for these as
>> >> well.
>> >>
>> >> I've opened the summary [2] for comments so you're welcome to open a
>> >> discussion about the results both in the ML and on the doc itself.
>> >>
>> >> (CC to all those who attended the Austin Summit meeting and other
>> >> interested parties)
>> >> Happy locking,
>> >>
>> >> [1]:
>> >>
>> >> http://lists.openstack.org/pipermail/openstack-dev/2016-April/093199.html
>> >> [2]:
>> >>
>> >> https://docs.google.com/document/d/1jdI8gkQKBE0G9koR0nLiW02d5rwyWv_-gAp7yavt4w8
>> >> [3]: https://review.openstack.org/#/c/342096/
>> >>
>> >> --
>> >> John Schwarz,
>> >> Senior Software Engineer,
>> >> Red Hat.
>> >>
>> >>
>> >> __________________________________________________________________________
>> >> OpenStack Development Mailing List (not for usage questions)
>> >> Unsubscribe:
>> >> OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>> >
>> >
>> > __________________________________________________________________________
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> --
>> John Schwarz,
>> Senior Software Engineer,
>> Red Hat.
>>
>> __________________________________________________________________________
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
John Schwarz,
Senior Software Engineer,
Red Hat.