Open Stack

Hi Bertrand,

Thanks for reporting the error. I confirmed that this error was consistently reproducible. A bug ticket was created for that.

https://bugs.launchpad.net/magnum/+bug/1506226

Best regards,
Hongbin

-----Original Message-----
From: Bertrand NOEL [mailto:bertrand.noel at cern.ch] 
Sent: October-14-15 8:51 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [magnum] Creating pods results in "EOF occurred in violation of protocol" exception

Hi,
I try Magnum, following instructions on the quickstart page [1]. I successfully create the baymodel and the bay. When I run the command to create redis pods (_magnum pod-create --manifest ./redis-master.yaml --bay k8sbay_), client side, it timeouts. And server side (m-cond.log), I get the following stack trace. It also happens with other Kubernetes examples.
I try with Ubuntu 14.04, with Magnum at commit fc8f412c87ea0f9dc0fc1c24963013e6d6209f27.


2015-10-14 12:16:40.877 ERROR oslo_messaging.rpc.dispatcher
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] Exception during message handling: [Errno 8] _ssl.c:510: EOF occurred in violation of protocol
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py",
line 142, in _dispatch_and_reply
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
executor_callback))
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py",
line 186, in _dispatch
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
executor_callback)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py",
line 129, in _do_dispatch
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/conductor/handlers/k8s_conductor.py", line 89, in pod_create
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
namespace='default')
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/apis/apiv_api.py",
line 3596, in create_namespaced_pod
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
callback=params.get('callback'))
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py",
line 320, in call_api
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher response_type, auth_settings, callback)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py",
line 148, in __call_api
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher post_params=post_params, body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py",
line 350, in request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 265, in POST
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher return self.IMPL.POST(*n, **kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 187, in POST
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 133, in request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher headers=headers)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 72, in request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher **urlopen_kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 149, in request_encode_body
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher return self.urlopen(method, url, **extra_kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 161, in urlopen
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher response = conn.urlopen(method, u.request_uri, **kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 588, in urlopen
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher raise
SSLError(e)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher SSLError: 
[Errno 8] _ssl.c:510: EOF occurred in violation of protocol
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
2015-10-14 12:16:40.879 ERROR oslo_messaging._drivers.common
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] Returning exception [Errno 8] _ssl.c:510: EOF occurred in violation of protocol to caller
2015-10-14 12:16:40.879 ERROR oslo_messaging._drivers.common
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] ['Traceback (most recent call last):\n', '  File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 142, in _dispatch_and_reply\n    executor_callback))\n', ' File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 186, in _dispatch\n    executor_callback)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 129, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  
File "/opt/stack/magnum/magnum/conductor/handlers/k8s_conductor.py", 
line 89, in pod_create\n    namespace=\'default\')\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/apis/apiv_api.py",
line 3596, in create_namespaced_pod\n
callback=params.get(\'callback\'))\n', '  File "/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 320, in call_api\n    response_type, auth_settings, callback)\n', 
'  File
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 148, in __call_api\n    post_params=post_params, body=body)\n', '  
File
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 350, in request\n    body=body)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
265, in POST\n    return self.IMPL.POST(*n, **kw)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
187, in POST\n    body=body)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
133, in request\n    headers=headers)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 72, in 
request\n    **urlopen_kw)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 149, 
in request_encode_body\n    return self.urlopen(method, url, 
**extra_kw)\n', '  File
"/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 
161, in urlopen\n    response = conn.urlopen(method, u.request_uri, 
**kw)\n', '  File
"/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 
588, in urlopen\n    raise SSLError(e)\n', 'SSLError: [Errno 8] 
_ssl.c:510: EOF occurred in violation of protocol\n']
2015-10-14 12:16:40.880 ERROR oslo_messaging._utils
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] The dispatcher method must catches all exceptions
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils Traceback (most recent call last):
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_utils.py", line 81, in run
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils self._executor_callback)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py",
line 152, in _dispatch_and_reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils
incoming.reply(failure=exc_info)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py",
line 101, in reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils self._send_reply(conn, reply, failure, log_failure=log_failure)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py",
line 59, in _send_reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     log_failure)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/common.py",
line 199, in serialize_remote_exception
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     json_data = 
jsonutils.dumps(data)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py", line 185, in dumps
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     return 
json.dumps(obj, default=default, **kwargs)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/__init__.py", line 250, in dumps
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils sort_keys=sort_keys,
**kw).encode(obj)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/encoder.py", line 207, in encode
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     chunks = 
self.iterencode(o, _one_shot=True)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     return 
_iterencode(o, 0)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils ValueError: Circular reference detected
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils



[1] http://docs.openstack.org/developer/magnum/dev/dev-quickstart.html

__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev