[openstack-dev] [magnum] Creating pods results in "EOF occurred in violation of protocol" exception

Bertrand NOEL bertrand.noel at cern.ch
Wed Oct 14 12:51:23 UTC 2015


Hi,
I try Magnum, following instructions on the quickstart page [1]. I 
successfully create the baymodel and the bay. When I run the command to 
create redis pods (_magnum pod-create --manifest ./redis-master.yaml 
--bay k8sbay_), client side, it timeouts. And server side (m-cond.log), 
I get the following stack trace. It also happens with other Kubernetes 
examples.
I try with Ubuntu 14.04, with Magnum at commit 
fc8f412c87ea0f9dc0fc1c24963013e6d6209f27.


2015-10-14 12:16:40.877 ERROR oslo_messaging.rpc.dispatcher 
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] Exception during 
message handling: [Errno 8] _ssl.c:510: EOF occurred in violation of 
protocol
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher Traceback 
(most recent call last):
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 142, in _dispatch_and_reply
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
executor_callback))
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 186, in _dispatch
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
executor_callback)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 129, in _do_dispatch
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher result = 
func(ctxt, **new_args)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/conductor/handlers/k8s_conductor.py", line 89, 
in pod_create
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
namespace='default')
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/apis/apiv_api.py", 
line 3596, in create_namespaced_pod
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
callback=params.get('callback'))
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 320, in call_api
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
response_type, auth_settings, callback)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 148, in __call_api
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher 
post_params=post_params, body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 350, in request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
265, in POST
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher return 
self.IMPL.POST(*n, **kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
187, in POST
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher body=body)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
133, in request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher headers=headers)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 72, in 
request
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher **urlopen_kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 149, 
in request_encode_body
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher return 
self.urlopen(method, url, **extra_kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 
161, in urlopen
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher response = 
conn.urlopen(method, u.request_uri, **kw)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher   File 
"/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 
588, in urlopen
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher raise 
SSLError(e)
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher SSLError: 
[Errno 8] _ssl.c:510: EOF occurred in violation of protocol
2015-10-14 12:16:40.877 TRACE oslo_messaging.rpc.dispatcher
2015-10-14 12:16:40.879 ERROR oslo_messaging._drivers.common 
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] Returning 
exception [Errno 8] _ssl.c:510: EOF occurred in violation of protocol to 
caller
2015-10-14 12:16:40.879 ERROR oslo_messaging._drivers.common 
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] ['Traceback (most 
recent call last):\n', '  File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 142, in _dispatch_and_reply\n    executor_callback))\n', ' File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 186, in _dispatch\n    executor_callback)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 129, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  
File "/opt/stack/magnum/magnum/conductor/handlers/k8s_conductor.py", 
line 89, in pod_create\n    namespace=\'default\')\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/apis/apiv_api.py", 
line 3596, in create_namespaced_pod\n 
callback=params.get(\'callback\'))\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 320, in call_api\n    response_type, auth_settings, callback)\n', 
'  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 148, in __call_api\n    post_params=post_params, body=body)\n', '  
File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/api_client.py", 
line 350, in request\n    body=body)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
265, in POST\n    return self.IMPL.POST(*n, **kw)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
187, in POST\n    body=body)\n', '  File 
"/opt/stack/magnum/magnum/common/pythonk8sclient/swagger_client/rest.py", line 
133, in request\n    headers=headers)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 72, in 
request\n    **urlopen_kw)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 149, 
in request_encode_body\n    return self.urlopen(method, url, 
**extra_kw)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 
161, in urlopen\n    response = conn.urlopen(method, u.request_uri, 
**kw)\n', '  File 
"/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 
588, in urlopen\n    raise SSLError(e)\n', 'SSLError: [Errno 8] 
_ssl.c:510: EOF occurred in violation of protocol\n']
2015-10-14 12:16:40.880 ERROR oslo_messaging._utils 
[req-960570cf-17b2-489f-9376-81890e2bf2d8 admin admin] The dispatcher 
method must catches all exceptions
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils Traceback (most 
recent call last):
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_utils.py", line 
81, in run
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils self._executor_callback)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", 
line 152, in _dispatch_and_reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils 
incoming.reply(failure=exc_info)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", 
line 101, in reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils 
self._send_reply(conn, reply, failure, log_failure=log_failure)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", 
line 59, in _send_reply
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     log_failure)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/common.py", 
line 199, in serialize_remote_exception
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     json_data = 
jsonutils.dumps(data)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/local/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py", line 
185, in dumps
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     return 
json.dumps(obj, default=default, **kwargs)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/__init__.py", line 250, in dumps
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils sort_keys=sort_keys, 
**kw).encode(obj)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/encoder.py", line 207, in encode
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     chunks = 
self.iterencode(o, _one_shot=True)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils   File 
"/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils     return 
_iterencode(o, 0)
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils ValueError: Circular 
reference detected
2015-10-14 12:16:40.880 TRACE oslo_messaging._utils



[1] http://docs.openstack.org/developer/magnum/dev/dev-quickstart.html



More information about the OpenStack-dev mailing list