Open Stack

Hi, when using Mysql behind Haproxy, i have a problem on reboot when some
nova services start after Haproxy service, but before Mysql service.
These service failed: (i re-checked for sure in /var/log/boot.log)
 * Starting Nova cert
[fail]
 * Starting Nova conductor
[fail]
 * Starting Nova scheduler
[fail]
 * Starting Cinder scheduler server
[fail]

I must login to server and re-start these services manually.

When check log of Nova-cert, I saw:

*2013-07-12 15:20:05.020 2490 CRITICAL nova [-] (OperationalError) (2013,
"Lost connection to MySQL server at 'reading initial communic
ation packet', system error: 0") None None*
2013-07-12 15:20:05.020 2490 TRACE nova Traceback (most recent call last):
2013-07-12 15:20:05.020 2490 TRACE nova   File "/usr/bin/nova-cert", line
51, in <module>
2013-07-12 15:20:05.020 2490 TRACE nova     service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 689, in wait
2013-07-12 15:20:05.020 2490 TRACE nova     _launcher.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 209, in wait
2013-07-12 15:20:05.020 2490 TRACE nova     super(ServiceLauncher,
self).wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 179, in wait
2013-07-12 15:20:05.020 2490 TRACE nova     service.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 168, in
wait
2013-07-12 15:20:05.020 2490 TRACE nova     return self._exit_event.wait()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/event.py", line 116, in wait
2013-07-12 15:20:05.020 2490 TRACE nova     return hubs.get_hub().switch()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/hubs/hub.py", line 187, in switch
2013-07-12 15:20:05.020 2490 TRACE nova     return self.greenlet.switch()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/eventlet/greenthread.py", line 194, in
main
2013-07-12 15:20:05.020 2490 TRACE nova     result = function(*args,
**kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 147, in run_server
2013-07-12 15:20:05.020 2490 TRACE nova     server.start()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/service.py", line 434, in start
2013-07-12 15:20:05.020 2490 TRACE nova     self.host, self.binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/conductor/api.py", line 261, in
service_get_by_a
rgs
2013-07-12 15:20:05.020 2490 TRACE nova     binary=binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/utils.py", line 1348, in wrapper
2013-07-12 15:20:05.020 2490 TRACE nova     return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 424, in in
ner
2013-07-12 15:20:05.020 2490 TRACE nova     return
catch_client_exception(exceptions, func, *args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/common.py",
line 407, in ca
tch_client_exception
2013-07-12 15:20:05.020 2490 TRACE nova     return func(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 325, in
service_get_
all_by
2013-07-12 15:20:05.020 2490 TRACE nova     result =
self.db.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/api.py", line 155, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova     return
IMPL.service_get_by_args(context, host, binary)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 96, in
wrapper
2013-07-12 15:20:05.020 2490 TRACE nova     return f(*args, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 409, in
service_get_by_args
2013-07-12 15:20:05.020 2490 TRACE nova     result = model_query(context,
models.Service).\
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 177, in
model_query
2013-07-12 15:20:05.020 2490 TRACE nova     session = kwargs.get('session')
or get_session()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
line 325, in get_session
2013-07-12 15:20:05.020 2490 TRACE nova     engine = get_engine()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
line 446, in get_engine
2013-07-12 15:20:05.020 2490 TRACE nova     _ENGINE =
create_engine(CONF.sql_connection)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/nova/openstack/common/db/sqlalchemy/session.py",
line 562, in create_engine
2013-07-12 15:20:05.020 2490 TRACE nova     engine.connect()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2471, in
connect
2013-07-12 15:20:05.020 2490 TRACE nova     return
self._connection_cls(self, **kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 878, in
__init__
2013-07-12 15:20:05.020 2490 TRACE nova     self.__connection = connection
or engine.raw_connection()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 2557, in
raw_connection
2013-07-12 15:20:05.020 2490 TRACE nova     return
self.pool.unique_connection()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 184, in
unique_connection
2013-07-12 15:20:05.020 2490 TRACE nova     return
_ConnectionFairy(self).checkout()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 401, in __init__
2013-07-12 15:20:05.020 2490 TRACE nova     rec = self._connection_record =
pool._do_get()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 746, in _do_get
2013-07-12 15:20:05.020 2490 TRACE nova     con = self._create_connection()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 189, in
_create_connection
2013-07-12 15:20:05.020 2490 TRACE nova     return _ConnectionRecord(self)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 282, in __init__
2013-07-12 15:20:05.020 2490 TRACE nova     self.connection =
self.__connect()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/pool.py", line 344, in
__connect
2013-07-12 15:20:05.020 2490 TRACE nova     connection =
self.__pool._creator()
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/strategies.py", line
80, in connect
2013-07-12 15:20:05.020 2490 TRACE nova     return dialect.connect(*cargs,
**cparams)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 281,
in connect
2013-07-12 15:20:05.020 2490 TRACE nova     return
self.dbapi.connect(*cargs, **cparams)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
2013-07-12 15:20:05.020 2490 TRACE nova     return Connection(*args,
**kwargs)
2013-07-12 15:20:05.020 2490 TRACE nova   File
"/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 187, in
__init__
2013-07-12 15:20:05.020 2490 TRACE nova     super(Connection,
self).__init__(*args, **kwargs2)
*2013-07-12 15:20:05.020 2490 TRACE nova OperationalError:
(OperationalError) (2013, "Lost connection to MySQL server at 'reading
initial communication packet', system error: 0") None None
*

Do you know a quick fix for this problem?
(I also send this email to report the problem)

Thanks you!

PS: I'm runing mysql, haproxy, nova-* services, cinder-* services on the
same server, using Ubuntu 12.04.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20130712/6667b737/attachment.html>