openstack-nova-consoleauth/scheduler failed to start up
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up. Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server. Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip. -- <https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
I think you'll need to provide a bit more context so that we can assist. - What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says? On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
- What version of Openstack? - Train - - How was it deployed? - I have a set of Puppet Modules and a local yum repo containing the needed RPMs - - Besides that log, anything else? - Not much - - Can you turn up the debug level for the scheduler and see what it says? - nova-consoleauth.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.5.229' is not allowed to connect to this MariaDB server") - the ip addr "10.160.5.229' may vary - - nova-scheduler.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.6.103' is not allowed to connect to this MariaDB server") - "10.160.5.229' may vary On Thu, Jul 15, 2021 at 8:59 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I think you'll need to provide a bit more context so that we can assist.
- What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says?
On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
-- <https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
Also when I run "sudo su && mysql", I get this error: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES). We don't see this error on our controller hosts in other DCs. On Thu, Jul 15, 2021 at 10:56 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
- What version of Openstack? - Train - - How was it deployed? - I have a set of Puppet Modules and a local yum repo containing the needed RPMs - - Besides that log, anything else? - Not much - - Can you turn up the debug level for the scheduler and see what it says? - nova-consoleauth.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.5.229' is not allowed to connect to this MariaDB server") - the ip addr "10.160.5.229' may vary - - nova-scheduler.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.6.103' is not allowed to connect to this MariaDB server") - "10.160.5.229' may vary
On Thu, Jul 15, 2021 at 8:59 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I think you'll need to provide a bit more context so that we can assist.
- What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says?
On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
-- <https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
I haven't deployed Openstack with the puppet modules, but I would start looking at the mysql/galera status and if the required user/permissions/allowed connections are set. I assume that this should be done by the puppet module for mysql/galera. On Fri, Jul 16, 2021 at 2:01 AM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
Also when I run "sudo su && mysql", I get this error: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES).
We don't see this error on our controller hosts in other DCs.
On Thu, Jul 15, 2021 at 10:56 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
- What version of Openstack? - Train - - How was it deployed? - I have a set of Puppet Modules and a local yum repo containing the needed RPMs - - Besides that log, anything else? - Not much - - Can you turn up the debug level for the scheduler and see what it says? - nova-consoleauth.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.5.229' is not allowed to connect to this MariaDB server") - the ip addr "10.160.5.229' may vary - - nova-scheduler.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.6.103' is not allowed to connect to this MariaDB server") - "10.160.5.229' may vary
On Thu, Jul 15, 2021 at 8:59 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I think you'll need to provide a bit more context so that we can assist.
- What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says?
On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang < peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
The puppet modules did deploy mysql but we need to manually "yum install galera" and add nodes after that. so now mysql is up and running, galera is installed. However running 'mysql' hits this Access denied error. On Fri, Jul 16, 2021 at 1:04 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I haven't deployed Openstack with the puppet modules, but I would start looking at the mysql/galera status and if the required user/permissions/allowed connections are set.
I assume that this should be done by the puppet module for mysql/galera.
On Fri, Jul 16, 2021 at 2:01 AM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
Also when I run "sudo su && mysql", I get this error: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES).
We don't see this error on our controller hosts in other DCs.
On Thu, Jul 15, 2021 at 10:56 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
- What version of Openstack? - Train - - How was it deployed? - I have a set of Puppet Modules and a local yum repo containing the needed RPMs - - Besides that log, anything else? - Not much - - Can you turn up the debug level for the scheduler and see what it says? - nova-consoleauth.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.5.229' is not allowed to connect to this MariaDB server") - the ip addr "10.160.5.229' may vary - - nova-scheduler.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.6.103' is not allowed to connect to this MariaDB server") - "10.160.5.229' may vary
On Thu, Jul 15, 2021 at 8:59 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I think you'll need to provide a bit more context so that we can assist.
- What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says?
On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang < peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
-- <https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
On Fri, 2021-07-16 at 13:25 -0700, Pete Zhang wrote:
The puppet modules did deploy mysql but we need to manually "yum install galera" and add nodes after that. so now mysql is up and running, galera is installed. However running 'mysql' hits this Access denied error.
just an fyi the nova console auth service was deprecated in stien https://github.com/openstack/nova/commit/f18ae13e366a57ce41840ecb4e323d0e4ff... so in train you do not need to deploy it and you should not deploy it in new installs its removed ussuri upstream and in ooo/downstream we removed it in train. so at least for the console auth service should should simpley ignore the error and not deploy it. for the schduler that obviosly required. it sounds like the user permisison are not correct in mysql the host/ip/domain name you use to connect to the db is imporant you can allow root login on 127.0.0.1 for example which will only work on that adresss and will reject localhost or the host ip. im not famialr with pupet or how the moduels deploy openstack but i would suspect that the user that were created are using a different ip then the one you have put in the nova.conf
On Fri, Jul 16, 2021 at 1:04 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I haven't deployed Openstack with the puppet modules, but I would start looking at the mysql/galera status and if the required user/permissions/allowed connections are set.
I assume that this should be done by the puppet module for mysql/galera.
On Fri, Jul 16, 2021 at 2:01 AM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
Also when I run "sudo su && mysql", I get this error: ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES).
We don't see this error on our controller hosts in other DCs.
On Thu, Jul 15, 2021 at 10:56 PM Pete Zhang <peiyong.zhang@salesforce.com> wrote:
- What version of Openstack? - Train - - How was it deployed? - I have a set of Puppet Modules and a local yum repo containing the needed RPMs - - Besides that log, anything else? - Not much - - Can you turn up the debug level for the scheduler and see what it says? - nova-consoleauth.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.5.229' is not allowed to connect to this MariaDB server") - the ip addr "10.160.5.229' may vary - - nova-scheduler.log - DBError: (pymysq.err.InternalError) (1130, u"Host '10.160.6.103' is not allowed to connect to this MariaDB server") - "10.160.5.229' may vary
On Thu, Jul 15, 2021 at 8:59 PM Laurent Dumont <laurentfdumont@gmail.com> wrote:
I think you'll need to provide a bit more context so that we can assist.
- What version of Openstack? - How was it deployed? - Besides that log, anything else? - Can you turn up the debug level for the scheduler and see what it says?
On Thu, Jul 15, 2021 at 10:12 PM Pete Zhang < peiyong.zhang@salesforce.com> wrote:
We are installing openstack in a new DC and seeing openstack-nova-consoleauth/scheduler failed to start up.
Checking log and found this. Host 10.160.4.5 is not allowed to connect to this MariaDB server.
Three controller nodes are installed with MariaDB/Galera Cluster, none of them with this ip.
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
--
<https://smart.salesforce.com/sig/peiyong.zhang//us_mb/default/link.html>
participants (3)
-
Laurent Dumont
-
Pete Zhang
-
Sean Mooney