Hi Ionut,

I didn't see this before on our production. Magnum auto healer just simply sends a POST request to Magnum api to update the health status. So I would suggest write a small script or even use curl to see if you can reproduce this firstly.


On 19/12/20 2:27 am, Ionut Biru wrote:
Hi again,

I failed to mention that is stable/victoria with couples of patches from review. Ignore the fact that in logs it  shows the 19.1.4 version in venv path.

On Fri, Dec 18, 2020 at 3:22 PM Ionut Biru <ionut@fleio.com> wrote:
Hi guys,

I have an issue with magnum api returning an error after a while:
Server-side error: "[('system library', 'fopen', 'Too many open files'), ('BIO routines', 'BIO_new_file', 'system lib'), ('x509 certificate routines', 'X509_load_cert_crl_file', 'system lib')]"

Log file: https://paste.xinu.at/6djE/

This started to appear after I enabled the template auto_healing_controller = magnum-auto-healer,  magnum_auto_healer_tag = v1.19.0.

Currently, I only have 4 clusters.

After that the API is in error state and doesn't work unless I restart it.


--
Ionut Biru - https://fleio.com


--
Ionut Biru - https://fleio.com
-- 
Cheers & Best regards,
Feilong Wang (王飞龙)
------------------------------------------------------
Senior Cloud Software Engineer
Tel: +64-48032246
Email: flwang@catalyst.net.nz
Catalyst IT Limited
Level 6, Catalyst House, 150 Willis Street, Wellington
------------------------------------------------------