[Kolla] FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left).

Bhupathi, Ramakrishna C-Ramakrishna.Bhupathi at charter.com
Wed Nov 27 13:07:05 UTC 2019


Thanks Mark. The deploy worked after a reboot  of my server. Appreciate your help.

--RamaK

-----Original Message-----
From: Mark Goddard [mailto:mark at stackhpc.com] 
Sent: Tuesday, November 26, 2019 10:05 AM
To: Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com>
Cc: openstack-discuss at lists.openstack.org
Subject: Re: [Kolla] FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left).

On Tue, 26 Nov 2019 at 14:12, Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com> wrote:
>
> Mark,
> Thanks for the tip.
>
> Now I get a failure (as below)  during deploy (I have all-in-one) config. I am on stein.
> It appears to me that this has been presumably  fixed .  But I still running into this error.
>
> I do not see anything explicit in the HAProxy/MariaDB logs.

Are both the haproxy and mariadb containers up and not restarting?
Does your api_interface have the VIP activated? Is haproxy listening on 10.1.0.250:3306 (check ss)?

>
>
>
> TASK [mariadb : wait for MariaDB to be available via HAProxy] 
> **********************************************************************
> **************************** FAILED - RETRYING: wait for MariaDB to be 
> available via HAProxy (10 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (9 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (8 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (7 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (6 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (5 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (4 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (3 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (2 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (1 retries left).
> fatal: [localhost]: FAILED! => {"attempts": 10, "changed": false, 
> "elapsed": 60, "msg": "Timeout when waiting for search string MariaDB 
> in 10.1.0.250:3306"}
>
> NO MORE HOSTS LEFT 
> **********************************************************************
> **********************************************************************
> *
>
> PLAY RECAP *****************************************************************************************************************************************************
> localhost                  : ok=76   changed=0    unreachable=0    failed=1    skipped=74   rescued=0    ignored=0
>
> Command failed ansible-playbook -i ../../all-in-one -e 
> @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e 
> CONFIG_DIR=/etc/kolla  -e kolla_action=deploy 
> /home/cloud-user/kolla-ansible/ansible/site.yml
>
> --RamaK
>
>
>
>
>
> -----Original Message-----
> From: Mark Goddard [mailto:mark at stackhpc.com]
> Sent: Monday, November 25, 2019 11:08 AM
> To: Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com>
> Cc: openstack-discuss at lists.openstack.org
> Subject: Re: [Kolla] RabbitMQ failure during deploy (Openstack with 
> Kolla)
>
> On Mon, 25 Nov 2019 at 13:54, Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com> wrote:
> >
> > Hey , I am evaluating Openstack with Kolla (have the latest ) and following the steps. I see that rabbitmq fails to start up and (keeps restarting).  Essentially the script fails in deploy. I have a single node all in one config.
> >
> >
> >
> > Can someone tell me what the cause for this failure?
> >
> >
> >
> >
> >
> > RUNNING HANDLER [rabbitmq : Waiting for rabbitmq to start on first 
> > node]
> > ********************************************************************
> > **
> > ******************************************
> >
> > fatal: [localhost]: FAILED! => {"changed": true, "cmd": "docker exec 
> > rabbitmq rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid",
> > "delta": "0:00:07.732346", "end": "2019-11-25 13:33:12.102342", "msg":
> > "non-zero return code", "rc": 137, "start": "2019-11-25 
> > 13:33:04.369996", "stderr": "", "stderr_lines": [], "stdout": 
> > "Waiting for 'rabbit at kolla-ubuntu'\npid is 6", "stdout_lines": 
> > ["Waiting for 'rabbit at kolla-ubuntu'", "pid is 6"]}
> >
> >
> >
> > RUNNING HANDLER [rabbitmq : Restart rabbitmq container (rest of 
> > nodes)]
> > ********************************************************************
> > **
> > *******************************************
> >
> >
> >
> > NO MORE HOSTS LEFT
> > ********************************************************************
> > **
> > ********************************************************************
> > **
> > **************************
> >
> >
> >
> > PLAY RECAP
> > ********************************************************************
> > **
> > ********************************************************************
> > **
> > **********************************
> >
> > localhost                  : ok=95   changed=2    unreachable=0    failed=1    skipped=78   rescued=0    ignored=0
> >
> >
> >
> > Command failed ansible-playbook -i ./all-in-one -e 
> > @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e 
> > CONFIG_DIR=/etc/kolla  -e kolla_action=deploy 
> > /home/ubuntu/venv/share/kolla-ansible/ansible/site.yml
> >
> >
> >
> >
> >
> > Accessing the container  logs for rabbitmq  .. this is what I see
> >
> >
> >
> > BOOT FAILED
> >
> > ===========
> >
> >
> >
> > Error description:
> >
> >    {could_not_start,rabbitmq_management,
> >
> >     {rabbitmq_management,
> >
> >      {bad_return,
> >
> >       {{rabbit_mgmt_app,start,[normal,[]]},
> >
> >        {'EXIT',
> >
> >         {{could_not_start_listener,
> >
> >           [{port,15672}],
> >
> >           {shutdown,
> >
> >            {failed_to_start_child,ranch_acceptors_sup,
> >
> >
> > {listen_error,rabbit_web_dispatch_sup_15672,eaddrinuse}}}},
>
> Hi RamaK, here is your issue. RabbitMQ management fails to start up due to the port already being in use (eaddrinuse). Perhaps RabbitMQ is running on the host already?
>
> >
> >          {gen_server,call,
> >
> >           [rabbit_web_dispatch_registry,
> >
> >            {add,rabbit_mgmt,
> >
> >             [{port,15672}],
> >
> >             #Fun<rabbit_web_dispatch.0.49789095>,
> >
> >
> >
> > --RamaK
> >
> > The contents of this e-mail message and any attachments are intended 
> > solely for the
> > addressee(s) and may contain confidential and/or legally privileged 
> > information. If you are not the intended recipient of this message 
> > or if this message has been addressed to you in error, please 
> > immediately alert the sender by reply e-mail and then delete this 
> > message and any attachments. If you are not the intended recipient, 
> > you are notified that any use, dissemination, distribution, copying, 
> > or storage of this message or any attachment is strictly prohibited.
> E-MAIL CONFIDENTIALITY NOTICE:
> The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
E-MAIL CONFIDENTIALITY NOTICE: 
The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.


More information about the openstack-discuss mailing list