Open Stack

As a follow up to the discussion during the IRC meeting yesterday, please vote for one of these approaches:

1)  Make the default for the rabbit_heartbeat_timeout_threshold parameter 60, which matches the default in Kilo oslo.messaging.  This will by default enable the RMQ heartbeat feature, which also matches the default in Kilo oslo.messaging.  Operators will need to set this parameter to 0 in order to disable the feature, which will be documented in the comments within the manifest.  We will reevaluate the default value for the Liberty release, since the oslo.messaging default most likely will change to 0 for that release.

2)  In addition to #1 above, also add a rabbit_enable_heartbeat parameter, which will default to false.  Setting that to true will be needed to explicitly enable the RMQ heartbeat feature, regardless of the value of rabbit_heartbeat_timeout_threshold.  By default the RMQ heartbeat feature will be disabled, which may be a marginally safer approach (due to the requirements.txt stuff, see below), but will not match the upstream Kilo oslo.messaging default.  This will also turn off the feature for people who have already been “getting it for free” if they do nothing and don’t update their composition module.

My vote is for #1.

Let’s plan to close the voting by next week’s IRC meeting, so we can come to a final conclusion at that time.

Thanks,
Mike




From: Mike Dorman
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
Date: Tuesday, June 23, 2015 at 5:07 PM
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: [openstack-dev] [puppet] oslo.messaging version and RabbitMQ heartbeat support

As a follow up to https://review.openstack.org/#/c/194399/ and the meeting discussion earlier today, I’ve determined that everybody (RDU, Ubuntu, Debian) is packaging oslo.messaging 1.8.2 or 1.8.3 with the Kilo build.  (This is also the version we get on our internal Anvil-based build.)  This is considerably lower than 1.11.0 where the default rabbit_heartbeat_timeout_threshold changes (from 60 to 0.)

If we go forward using the default rabbit_heartbeat_timeout_threshold value of 60, that will be the correct default behavior up through oslo.messaging 1.10.x.

When people upgrade to 1.11.0 or higher, we’ll no longer match the upstream default behavior.  But, it should maintain the _actual_ behavior (heartbeating enabled) for people doing an upgrade.  Once Liberty is cut, we should reevaluate to make sure we’re matching whatever the default is at that time.

However, the larger problem I see is that oslo.messaging requirements.txt in <=1.10.x does not enforce the needed versions of kombu and amqp for heartbeat to work:
https://github.com/openstack/oslo.messaging/blob/1.8.2/requirements.txt#L25-L26  This is confusing as heartbeat is enabled by default!

I am not sure what the behavior is when heartbeat is enabled with older kombu or amqp.  Does anyone know?  If it silently fails, maybe we don’t care.  But if enabling heartbeat (by default, rabbit_heartbeat_timeout_threshold=60) actively breaks, that would be bad.

I see two options here:

1)  Make default rabbit_heartbeat_timeout_threshold=60 in the Puppet modules, to strictly follow the upstream default in Kilo.  Reevaluate this default value for Liberty.  Ignore the kombu/amqp library stuff and hope “it just works itself out naturally.”

2)  Add a rabbit_enable_heartbeat parameter to explicitly enable/disable the feature, and default to disable.  This goes against the current default behavior, but will match it for oslo.messaging >=1.11.0.  I think this is the safest path, as we won’t be automatically enabling heartbeat for people who don’t have a new enough kombu or amqp.

Personally, I like #1, because I am going to enable this feature, anyway.  And I can’t really imagine why one would _not_ want to enable it.  But I am fine implementing it either way, I just want to get it done so I can get off my local forks. :)

Thoughts?

Mike

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20150701/de9504a9/attachment.html>