Yes, that was it: missing \[health\_manager\].heartbeat\_key in octavia.conf It is not present in openstack victoria octavia docs, I'll push it together with my installation guide for centos8. Thanks for your accurate hint Gregory. It is always crucial to ask the right guy:) Regards, Piotr Mossakowski Sent from ProtonMail mobile \-------- Original Message -------- On 22 Mar 2021, 09:10, Gregory Thiemonge < gthiemonge@redhat.com> wrote:
Hi,
Most of the OFFLINE operational status issues are caused by communication problems between the amphorae and the Octavia health-manager.
In your case, the "Ignoring this packet. Exception: 'NoneType' object has no attribute 'encode'" log message shows that the health-manager receives the heartbeat packets from the amphorae but it is unable to decode them. Those packets are encrypted JSON messages and it seems that the key (\[health\_manager\].heartbeat\_key see [https://docs.openstack.org/octavia/latest/configuration/configref.html\#health-manager][https_docs.openstack.org_octavia_latest_configuration_configref.html_health-manager]) used to encrypt those messages is not defined in your configuration file. So I would suggest configuring it and restarting the Octavia services, then you can re-create or failover the load balancers (you cannot change this parameter in a running load balancer).
Gregory
On Sun, Mar 21, 2021 at 6:17 PM <[apps@mossakowski.ch][apps_mossakowski.ch]> wrote:
Hello,
I have stable/victoria baremetal openstack with octavia installed on centos8 using openvswitch mechanism driver: octavia api on controller, health-manager,housekeeping,worker on 3 compute/network nodes.
Official docs include only ubuntu with linuxbridge mechanism but I used https://github.com/prastamaha/openstack-octavia as a reference to get it working on centos8 with ovs.
I will push those docs instructions for centos8 soon: https://github.com/openstack/octavia/tree/master/doc/source/install.
I created basic http scenario using [https://docs.openstack.org/octavia/victoria/user/guides/basic-cookbook.html\#deploy-a-basic-http-load-balancer][https_docs.openstack.org_octavia_victoria_user_guides_basic-cookbook.html_deploy-a-basic-http-load-balancer].
Loadbalancer works but its operational status is offline (openstack\_loadbalancer\_outputs.txt).
On all octavia workers I see the same warning message in health\_manager.log:
Health Manager experienced an exception processing a heartbeat message from ('172.31.255.233', 1907). Ignoring this packet. Exception: 'NoneType' object has no attribute 'encode'
I've searched for related active bug but all I found is this not related in my opinion: [https://storyboard.openstack.org/\#!/story/2008615][https_storyboard.openstack.org_story_2008615]
I'm attaching all info I've gathered:
* octavia.conf and health\_manager debug logs (octavia\_config\_and\_health\_manager\_logs.txt)
* tcpdump from amphora VM (tcpdump\_from\_amphora\_vm.txt)
* tcpdump from octavia worker (tcpdump\_from\_octavia\_worker.txt)
* debug amphora-agent.log from amphora VM (amphora-agent.log)
Can you point me to the right direction what I have missed?
Thanks!
Piotr Mossakowski
[https_docs.openstack.org_octavia_latest_configuration_configref.html_health-manager]: https://docs.openstack.org/octavia/latest/configuration/configref.html#healt... [apps_mossakowski.ch]: mailto:apps@mossakowski.ch [https_docs.openstack.org_octavia_victoria_user_guides_basic-cookbook.html_deploy-a-basic-http-load-balancer]: https://docs.openstack.org/octavia/victoria/user/guides/basic-cookbook.html#... [https_storyboard.openstack.org_story_2008615]: https://storyboard.openstack.org/#!/story/2008615