Hello, thank you for your answer. The access for user octavia on host octavia04 is denied because my database host is maria-intern.desy.de and there is no DB service on octavia04. But why is the script trying to populate the DB on localhost and not my DB host as I configured in the /etc/octavia/octavia.conf? "Access denied for user 'octavia'@' [ http://octavia04.desy.de/ | octavia04.desy.de ] ' persistence_connection = mysql+pymysql:// [ http://octavia:woxSGH45cdZL1Sa4@maria-intern.desy.de/octavia_persistence | octavia:woxSGH45cdZL1Sa4@maria-intern.desy.de/octavia_persistence ] Greets Stefan Bujack From: "Anna Taraday" <akamyshnikova@mirantis.com> To: "Stefan Bujack" <stefan.bujack@desy.de> Cc: "openstack-discuss" <openstack-discuss@lists.openstack.org> Sent: Monday, 5 October, 2020 08:58:43 Subject: Re: [Octavia] Please help with amphorav2 provider populate db command Hello, Error in your trace shows "Access denied for user ' octavia ' Please check that you followed all steps from setup guide and grant access for user. [1] - [ https://docs.openstack.org/octavia/latest/install/install-amphorav2.html#pre... | https://docs.openstack.org/octavia/latest/install/install-amphorav2.html#pre... ] On Tue, Sep 29, 2020 at 8:12 PM Bujack, Stefan < [ mailto:stefan.bujack@desy.de | stefan.bujack@desy.de ] > wrote: Hello, I think I need a little help again with the configuration of the amphora v2 provider. I get an error when I try to populate the database. It seems that the name of the localhost is used for the DB host and not what I configured in octavia.conf as DB host root@octavia04:~# octavia-db-manage --config-file /etc/octavia/octavia.conf upgrade_persistence 2020-09-29 11:45:01.911 818313 WARNING taskflow.persistence.backends.impl_sqlalchemy [-] Engine connection (validate) failed due to '(pymysql.err.OperationalError) (1045, "Access denied for user 'octavia'@' [ http://octavia04.desy.de/ | octavia04.desy.de ] ' (using password: YES)") (Background on this error at: [ http://sqlalche.me/e/e3q8 | http://sqlalche.me/e/e3q8 ] )' 2020-09-29 11:45:01.912 818313 CRITICAL octavia-db-manage [-] Unhandled error: sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'octavia'@' [ http://octavia04.desy.de/ | octavia04.desy.de ] ' (using password: YES)") (Background on this error at: [ http://sqlalche.me/e/e3q8 | http://sqlalche.me/e/e3q8 ] ) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage Traceback (most recent call last): 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2276, in _wrap_pool_connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return fn() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 303, in unique_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return _ConnectionFairy._checkout(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 760, in _checkout 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage fairy = _ConnectionRecord.checkout(pool) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 492, in checkout 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage rec = pool._do_get() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 139, in _do_get 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self._dec_overflow() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage compat.reraise(exc_type, exc_value, exc_tb) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 153, in reraise 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise value 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 136, in _do_get 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self._create_connection() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 308, in _create_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return _ConnectionRecord(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 437, in __init__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self.__connect(first_connect_check=True) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 639, in __connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage connection = pool._invoke_creator(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return dialect.connect(*cargs, **cparams) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 482, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self.dbapi.connect(*cargs, **cparams) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/__init__.py", line 94, in Connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return Connection(*args, **kwargs) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 325, in __init__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self.connect() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 599, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self._request_authentication() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 861, in _request_authentication 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage auth_packet = self._read_packet() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 684, in _read_packet 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage packet.check_error() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/protocol.py", line 220, in check_error 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage err.raise_mysql_exception(self._data) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise errorclass(errno, errval) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage pymysql.err.OperationalError: (1045, "Access denied for user 'octavia'@' [ http://octavia04.desy.de/ | octavia04.desy.de ] ' (using password: YES)") 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage The above exception was the direct cause of the following exception: 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage Traceback (most recent call last): 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/local/bin/octavia-db-manage", line 8, in <module> 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage sys.exit(main()) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/local/lib/python3.8/dist-packages/octavia/db/migration/cli.py", line 156, in main 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage CONF.command.func(config, [ http://conf.command.name/ | CONF.command.name ] ) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/local/lib/python3.8/dist-packages/octavia/db/migration/cli.py", line 98, in do_persistence_upgrade 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage persistence.initialize() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/local/lib/python3.8/dist-packages/octavia/controller/worker/v2/taskflow_jobboard_driver.py", line 50, in initialize 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage with contextlib.closing(backend.get_connection()) as connection: 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/taskflow/persistence/backends/impl_sqlalchemy.py", line 335, in get_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage conn.validate(max_retries=self._max_retries) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/taskflow/persistence/backends/impl_sqlalchemy.py", line 394, in validate 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage _try_connect(self._engine) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 311, in wrapped_f 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self.call(f, *args, **kw) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 391, in call 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage do = self.iter(retry_state=retry_state) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 338, in iter 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return fut.result() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3.8/concurrent/futures/_base.py", line 432, in result 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self.__get_result() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise self._exception 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/tenacity/__init__.py", line 394, in call 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage result = fn(*args, **kwargs) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/taskflow/persistence/backends/impl_sqlalchemy.py", line 391, in _try_connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage with contextlib.closing(engine.connect()): 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2209, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self._connection_cls(self, **kwargs) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 103, in __init__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage else engine.raw_connection() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2306, in raw_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self._wrap_pool_connect( 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2279, in _wrap_pool_connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage Connection._handle_dbapi_exception_noconnection( 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 1547, in _handle_dbapi_exception_noconnection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage util.raise_from_cause(sqlalchemy_exception, exc_info) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage reraise(type(exception), exception, tb=exc_tb, cause=cause) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 152, in reraise 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise value.with_traceback(tb) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/base.py", line 2276, in _wrap_pool_connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return fn() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 303, in unique_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return _ConnectionFairy._checkout(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 760, in _checkout 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage fairy = _ConnectionRecord.checkout(pool) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 492, in checkout 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage rec = pool._do_get() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 139, in _do_get 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self._dec_overflow() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/langhelpers.py", line 68, in __exit__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage compat.reraise(exc_type, exc_value, exc_tb) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/util/compat.py", line 153, in reraise 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise value 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/impl.py", line 136, in _do_get 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self._create_connection() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 308, in _create_connection 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return _ConnectionRecord(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 437, in __init__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self.__connect(first_connect_check=True) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/pool/base.py", line 639, in __connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage connection = pool._invoke_creator(self) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/strategies.py", line 114, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return dialect.connect(*cargs, **cparams) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/sqlalchemy/engine/default.py", line 482, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return self.dbapi.connect(*cargs, **cparams) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/__init__.py", line 94, in Connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage return Connection(*args, **kwargs) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 325, in __init__ 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self.connect() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 599, in connect 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage self._request_authentication() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 861, in _request_authentication 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage auth_packet = self._read_packet() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/connections.py", line 684, in _read_packet 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage packet.check_error() 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/protocol.py", line 220, in check_error 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage err.raise_mysql_exception(self._data) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage File "/usr/lib/python3/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage raise errorclass(errno, errval) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (1045, "Access denied for user 'octavia'@' [ http://octavia04.desy.de/ | octavia04.desy.de ] ' (using password: YES)") 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage (Background on this error at: [ http://sqlalche.me/e/e3q8 | http://sqlalche.me/e/e3q8 ] ) 2020-09-29 11:45:01.912 818313 ERROR octavia-db-manage root@octavia04:~# cat /etc/octavia/octavia.conf [DEFAULT] transport_url = rabbit:// [ mailto:openstack%3Apassword@rabbit-intern.desy.de | openstack:password@rabbit-intern.desy.de ] use_journal = True [api_settings] bind_host = 0.0.0.0 bind_port = 9876 [certificates] cert_generator = local_cert_generator ca_certificate = /etc/octavia/certs/server_ca.cert.pem ca_private_key = /etc/octavia/certs/server_ca.key.pem ca_private_key_passphrase = passphrase [controller_worker] amp_image_owner_id = f89517ee676f4618bd55849477442aca amp_image_tag = amphora amp_ssh_key_name = octaviakey amp_secgroup_list = 2236e82c-13fe-42e3-9fcf-bea43917f231 amp_boot_network_list = 9f7fefc4-f262-4d8d-9465-240f94a7e87b amp_flavor_id = 200 network_driver = allowed_address_pairs_driver compute_driver = compute_nova_driver amphora_driver = amphora_haproxy_rest_driver client_ca = /etc/octavia/certs/client_ca.cert.pem [database] connection = mysql+pymysql:// [ http://octavia:password@maria-intern.desy.de/octavia | octavia:password@maria-intern.desy.de/octavia ] [haproxy_amphora] client_cert = /etc/octavia/certs/client.cert-and-key.pem server_ca = /etc/octavia/certs/server_ca.cert.pem [health_manager] bind_port = 5555 bind_ip = 172.16.0.2 controller_ip_port_list = [ http://172.16.0.2:5555/ | 172.16.0.2:5555 ] [keystone_authtoken] www_authenticate_uri = [ https://keystone-intern.desy.de:5000/v3 | https://keystone-intern.desy.de:5000/v3 ] auth_url = [ https://keystone-intern.desy.de:5000/v3 | https://keystone-intern.desy.de:5000/v3 ] memcached_servers = [ http://nova-intern.desy.de:11211/ | nova-intern.desy.de:11211 ] auth_type = password project_domain_name = default user_domain_name = default project_name = service username = octavia password = password service_token_roles_required = True [oslo_messaging] topic = octavia_prov [service_auth] auth_url = [ https://keystone-intern.desy.de:5000/v3 | https://keystone-intern.desy.de:5000/v3 ] memcached_servers = [ http://nova-intern.desy.de:11211/ | nova-intern.desy.de:11211 ] auth_type = password project_domain_name = Default user_domain_name = Default project_name = service username = octavia password = password [task_flow] persistence_connection = mysql+pymysql:// [ http://octavia:woxSGH45cdZL1Sa4@maria-intern.desy.de/octavia_persistence | octavia:woxSGH45cdZL1Sa4@maria-intern.desy.de/octavia_persistence ] jobboard_backend_driver = 'redis_taskflow_driver' jobboard_backend_hosts = 10.254.28.113 jobboard_backend_port = 6379 jobboard_backend_password = password jobboard_backend_namespace = 'octavia_jobboard' root@octavia04:~# octavia-db-manage current 2020-09-29 12:02:23.159 819432 INFO alembic.runtime.migration [-] Context impl MySQLImpl. 2020-09-29 12:02:23.160 819432 INFO alembic.runtime.migration [-] Will assume non-transactional DDL. fbd705961c3a (head) We have an Openstack Ussuri deployment on Ubuntu 20.04. Thanks in advance, Stefan Bujack -- Regards, Ann Taraday Mirantis, Inc