[octavia] victoria - loadbalancer works but its operational status is offline

Luke Camilleri luke.camilleri at zylacomputing.com
Thu May 13 19:06:50 UTC 2021

Actually I figured it out a few minutes after I had sent the message :-)

Seems like it is just a string of characters that need to be the same 
between the [health-manager] section in the amphora-agent on the amphora 
instance and the corresponding section in octavia.conf on the controller 

This should be sorted now, thanks

On 13/05/2021 18:37, Michael Johnson wrote:
> It is documented here:
> https://docs.openstack.org/octavia/latest/configuration/configref.html#health_manager.heartbeat_key
> It is also listed in the quick start guide here:
> https://docs.openstack.org/octavia/latest/contributor/guides/dev-quick-start.html#configuring-octavia
> Your deployment tool should be configuring that for you.
> Michael
> On Thu, May 13, 2021 at 6:50 AM Luke Camilleri
> <luke.camilleri at zylacomputing.com> wrote:
>> HI Gregory, I have the same issue as described here by Piotr and on checking the config I also noticed that the key is missing from the config file.
>> What is the heartbeat_key string though, following the docs to install and configure, where is this key located?
>> On 28/03/2021 20:32, apps at mossakowski.ch wrote:
>> Yes, that was it: missing [health_manager].heartbeat_key in octavia.conf
>> It is not present in openstack victoria octavia docs, I'll push it together with my installation guide for centos8.
>> Thanks for your accurate hint Gregory.
>> It is always crucial to ask the right guy:)
>> Regards,
>> Piotr Mossakowski
>> Sent from ProtonMail mobile
>> -------- Original Message --------
>> On 22 Mar 2021, 09:10, Gregory Thiemonge < gthiemonge at redhat.com> wrote:
>> Hi,
>> Most of the OFFLINE operational status issues are caused by communication problems between the amphorae and the Octavia health-manager.
>> In your case, the "Ignoring this packet. Exception: 'NoneType' object has no attribute 'encode'" log message shows that the health-manager receives the heartbeat packets from the amphorae but it is unable to decode them. Those packets are encrypted JSON messages and it seems that the key ([health_manager].heartbeat_key see https://docs.openstack.org/octavia/latest/configuration/configref.html#health-manager) used to encrypt those messages is not defined in your configuration file. So I would suggest configuring it and restarting the Octavia services, then you can re-create or failover the load balancers (you cannot change this parameter in a running load balancer).
>> Gregory
>> On Sun, Mar 21, 2021 at 6:17 PM <apps at mossakowski.ch> wrote:
>>> Hello,
>>> I have stable/victoria baremetal openstack with octavia installed on centos8 using openvswitch mechanism driver: octavia api on controller, health-manager,housekeeping,worker on 3 compute/network nodes.
>>> Official docs include only ubuntu with linuxbridge mechanism but I used https://github.com/prastamaha/openstack-octavia as a reference to get it working on centos8 with ovs.
>>> I will push those docs instructions for centos8 soon: https://github.com/openstack/octavia/tree/master/doc/source/install.
>>> I created basic http scenario using https://docs.openstack.org/octavia/victoria/user/guides/basic-cookbook.html#deploy-a-basic-http-load-balancer.
>>> Loadbalancer works but its operational status is offline (openstack_loadbalancer_outputs.txt).
>>> On all octavia workers I see the same warning message in health_manager.log:
>>> Health Manager experienced an exception processing a heartbeat message from ('', 1907). Ignoring this packet. Exception: 'NoneType' object has no attribute 'encode'
>>> I've searched for related active bug but all I found is this not related in my opinion: https://storyboard.openstack.org/#!/story/2008615
>>> I'm attaching all info I've gathered:
>>> octavia.conf and health_manager debug logs (octavia_config_and_health_manager_logs.txt)
>>> tcpdump from amphora VM (tcpdump_from_amphora_vm.txt)
>>> tcpdump from octavia worker (tcpdump_from_octavia_worker.txt)
>>> debug amphora-agent.log from amphora VM (amphora-agent.log)
>>> Can you point me to the right direction what I have missed?
>>> Thanks!
>>> Piotr Mossakowski
>>> https://github.com/moss2k13

More information about the openstack-discuss mailing list