Yes, that was it: missing [health_manager].heartbeat_key in octavia.conf<br>It is not present in openstack victoria octavia docs, I'll push it together with my installation guide for centos8.<br>Thanks for your accurate hint Gregory. <br>It is always crucial to ask the right guy:)<br><br>Regards,<br>Piotr Mossakowski<br>Sent from ProtonMail mobile<br><br><br><br>-------- Original Message --------<br>On 22 Mar 2021, 09:10, Gregory Thiemonge < gthiemonge@redhat.com> wrote:<blockquote class="protonmail_quote"><br><div dir="ltr">Hi,<br><br>Most of the OFFLINE operational status issues are caused by communication problems between the amphorae and the Octavia health-manager.<br><br>In your case, the &quot;Ignoring this packet. Exception: &#39;NoneType&#39; object has no attribute &#39;encode&#39;&quot; log message shows that the health-manager receives the heartbeat packets from the amphorae but it is unable to decode them. Those packets are encrypted JSON messages and it seems that the key ([health_manager].heartbeat_key see <a href="https://docs.openstack.org/octavia/latest/configuration/configref.html#health-manager">https://docs.openstack.org/octavia/latest/configuration/configref.html#health-manager</a>) used to encrypt those messages is not defined in your configuration file. So I would suggest configuring it and restarting the Octavia services, then you can re-create or failover the load balancers (you cannot change this parameter in a running load balancer).<br><div><br></div><div>Gregory</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Mar 21, 2021 at 6:17 PM &lt;<a href="mailto:apps@mossakowski.ch">apps@mossakowski.ch</a>&gt; wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Hello,<br></div><div>I have stable/victoria baremetal openstack with octavia installed on centos8 using openvswitch mechanism driver: octavia api on controller, health-manager,housekeeping,worker on 3 compute/network nodes.<br></div><div>Official docs include only ubuntu with linuxbridge mechanism but I used <a href="https://github.com/prastamaha/openstack-octavia" target="_blank">https://github.com/prastamaha/openstack-octavia</a> as a reference to get it working on centos8 with ovs.<br></div><div>I will push those docs instructions for centos8 soon: <a href="https://github.com/openstack/octavia/tree/master/doc/source/install" target="_blank">https://github.com/openstack/octavia/tree/master/doc/source/install</a>. <br></div><div>I created basic http scenario using <a href="https://docs.openstack.org/octavia/victoria/user/guides/basic-cookbook.html#deploy-a-basic-http-load-balancer" target="_blank">https://docs.openstack.org/octavia/victoria/user/guides/basic-cookbook.html#deploy-a-basic-http-load-balancer</a>.<br></div><div>Loadbalancer works but its operational status is offline (openstack_loadbalancer_outputs.txt).<br></div><div>On all octavia workers I see the same warning message in health_manager.log:<br></div><div>Health Manager experienced an exception processing a heartbeat message from (&#39;172.31.255.233&#39;, 1907). Ignoring this packet. Exception: &#39;NoneType&#39; object has no attribute &#39;encode&#39;<br></div><div>I&#39;ve searched for related active bug but all I found is this not related in my opinion: <a href="https://storyboard.openstack.org/#!/story/2008615" target="_blank">https://storyboard.openstack.org/#!/story/2008615</a><br></div><div>I&#39;m attaching all info I&#39;ve gathered:<br></div><ul><li>octavia.conf and health_manager debug logs (octavia_config_and_health_manager_logs.txt)<br></li><li>tcpdump from amphora VM (tcpdump_from_amphora_vm.txt)<br></li><li>tcpdump from octavia worker (tcpdump_from_octavia_worker.txt)<br></li><li>debug amphora-agent.log from amphora VM (amphora-agent.log)<br></li></ul><div>Can you point me to the right direction what I have missed?<br></div><div>Thanks!<br></div><div>Piotr Mossakowski<br></div><div><a href="https://github.com/moss2k13" target="_blank">https://github.com/moss2k13</a><br></div><div><div><br></div><div><br></div></div><div><br></div></blockquote></div>
</div>