[Kolla] FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left).
Mark Goddard
mark at stackhpc.com
Tue Nov 26 15:04:36 UTC 2019
On Tue, 26 Nov 2019 at 14:12, Bhupathi, Ramakrishna
<C-Ramakrishna.Bhupathi at charter.com> wrote:
>
> Mark,
> Thanks for the tip.
>
> Now I get a failure (as below) during deploy (I have all-in-one) config. I am on stein.
> It appears to me that this has been presumably fixed . But I still running into this error.
>
> I do not see anything explicit in the HAProxy/MariaDB logs.
Are both the haproxy and mariadb containers up and not restarting?
Does your api_interface have the VIP activated? Is haproxy listening
on 10.1.0.250:3306 (check ss)?
>
>
>
> TASK [mariadb : wait for MariaDB to be available via HAProxy] **************************************************************************************************
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (10 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (9 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (8 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (7 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (6 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (5 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (4 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (3 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (2 retries left).
> FAILED - RETRYING: wait for MariaDB to be available via HAProxy (1 retries left).
> fatal: [localhost]: FAILED! => {"attempts": 10, "changed": false, "elapsed": 60, "msg": "Timeout when waiting for search string MariaDB in 10.1.0.250:3306"}
>
> NO MORE HOSTS LEFT *********************************************************************************************************************************************
>
> PLAY RECAP *****************************************************************************************************************************************************
> localhost : ok=76 changed=0 unreachable=0 failed=1 skipped=74 rescued=0 ignored=0
>
> Command failed ansible-playbook -i ../../all-in-one -e @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e CONFIG_DIR=/etc/kolla -e kolla_action=deploy /home/cloud-user/kolla-ansible/ansible/site.yml
>
> --RamaK
>
>
>
>
>
> -----Original Message-----
> From: Mark Goddard [mailto:mark at stackhpc.com]
> Sent: Monday, November 25, 2019 11:08 AM
> To: Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com>
> Cc: openstack-discuss at lists.openstack.org
> Subject: Re: [Kolla] RabbitMQ failure during deploy (Openstack with Kolla)
>
> On Mon, 25 Nov 2019 at 13:54, Bhupathi, Ramakrishna <C-Ramakrishna.Bhupathi at charter.com> wrote:
> >
> > Hey , I am evaluating Openstack with Kolla (have the latest ) and following the steps. I see that rabbitmq fails to start up and (keeps restarting). Essentially the script fails in deploy. I have a single node all in one config.
> >
> >
> >
> > Can someone tell me what the cause for this failure?
> >
> >
> >
> >
> >
> > RUNNING HANDLER [rabbitmq : Waiting for rabbitmq to start on first
> > node]
> > **********************************************************************
> > ******************************************
> >
> > fatal: [localhost]: FAILED! => {"changed": true, "cmd": "docker exec
> > rabbitmq rabbitmqctl wait /var/lib/rabbitmq/mnesia/rabbitmq.pid",
> > "delta": "0:00:07.732346", "end": "2019-11-25 13:33:12.102342", "msg":
> > "non-zero return code", "rc": 137, "start": "2019-11-25
> > 13:33:04.369996", "stderr": "", "stderr_lines": [], "stdout": "Waiting
> > for 'rabbit at kolla-ubuntu'\npid is 6", "stdout_lines": ["Waiting for
> > 'rabbit at kolla-ubuntu'", "pid is 6"]}
> >
> >
> >
> > RUNNING HANDLER [rabbitmq : Restart rabbitmq container (rest of
> > nodes)]
> > **********************************************************************
> > *******************************************
> >
> >
> >
> > NO MORE HOSTS LEFT
> > **********************************************************************
> > **********************************************************************
> > **************************
> >
> >
> >
> > PLAY RECAP
> > **********************************************************************
> > **********************************************************************
> > **********************************
> >
> > localhost : ok=95 changed=2 unreachable=0 failed=1 skipped=78 rescued=0 ignored=0
> >
> >
> >
> > Command failed ansible-playbook -i ./all-in-one -e
> > @/etc/kolla/globals.yml -e @/etc/kolla/passwords.yml -e
> > CONFIG_DIR=/etc/kolla -e kolla_action=deploy
> > /home/ubuntu/venv/share/kolla-ansible/ansible/site.yml
> >
> >
> >
> >
> >
> > Accessing the container logs for rabbitmq .. this is what I see
> >
> >
> >
> > BOOT FAILED
> >
> > ===========
> >
> >
> >
> > Error description:
> >
> > {could_not_start,rabbitmq_management,
> >
> > {rabbitmq_management,
> >
> > {bad_return,
> >
> > {{rabbit_mgmt_app,start,[normal,[]]},
> >
> > {'EXIT',
> >
> > {{could_not_start_listener,
> >
> > [{port,15672}],
> >
> > {shutdown,
> >
> > {failed_to_start_child,ranch_acceptors_sup,
> >
> >
> > {listen_error,rabbit_web_dispatch_sup_15672,eaddrinuse}}}},
>
> Hi RamaK, here is your issue. RabbitMQ management fails to start up due to the port already being in use (eaddrinuse). Perhaps RabbitMQ is running on the host already?
>
> >
> > {gen_server,call,
> >
> > [rabbit_web_dispatch_registry,
> >
> > {add,rabbit_mgmt,
> >
> > [{port,15672}],
> >
> > #Fun<rabbit_web_dispatch.0.49789095>,
> >
> >
> >
> > --RamaK
> >
> > The contents of this e-mail message and any attachments are intended
> > solely for the
> > addressee(s) and may contain confidential and/or legally privileged
> > information. If you are not the intended recipient of this message or
> > if this message has been addressed to you in error, please immediately
> > alert the sender by reply e-mail and then delete this message and any
> > attachments. If you are not the intended recipient, you are notified
> > that any use, dissemination, distribution, copying, or storage of this
> > message or any attachment is strictly prohibited.
> E-MAIL CONFIDENTIALITY NOTICE:
> The contents of this e-mail message and any attachments are intended solely for the addressee(s) and may contain confidential and/or legally privileged information. If you are not the intended recipient of this message or if this message has been addressed to you in error, please immediately alert the sender by reply e-mail and then delete this message and any attachments. If you are not the intended recipient, you are notified that any use, dissemination, distribution, copying, or storage of this message or any attachment is strictly prohibited.
More information about the openstack-discuss
mailing list