<div dir="ltr">Hi,<br><br>I was trying to create cells under openstack using devstack. My setup contains 3 machines. One toplevel and 2 compute cells.<br>I'm following this documentation, <a href="http://docs.openstack.org/trunk/config-reference/content/section_compute-cells.html">http://docs.openstack.org/trunk/config-reference/content/section_compute-cells.html</a><span class=""></span><span class=""></span>.<br><br>Both these cells instantiation are generating errors.<br>1.
 first one doesn't generate any error logs unless I issue a command at 
the parent 'nova cell-show cell2'. At this point the toplevel cell 
throws the following error, <br><br><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">2014-10-17 12:03:34.888 ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: Circular reference detected<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", 
line 134, in _dispatch_and_reply<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     incoming.message))<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
 line 72, in reply<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     self._send_reply(conn, reply, failure, log_failure=log_failure)<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
 line 62, in _send_reply<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", 
line 302, in serialize_msg<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     _MESSAGE_KEY: jsonutils.dumps(raw_msg)}<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py",
 line 172, in dumps<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     return json.dumps(value, default=default, **kwargs)<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     sort_keys=sort_keys, **kw).encode(obj)<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib64/python2.7/json/encoder.py", line 207, in encode<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     chunks = self.iterencode(o, _one_shot=True)<br>2014-10-17
 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File 
"/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     return _iterencode(o, 0)<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher ValueError: Circular reference detected<br>2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher <br>2014-10-17 12:03:34.890 ERROR oslo.messaging._drivers.common [-] Returning exception Circular reference detected to caller<br>2014-10-17
 12:03:34.890 ERROR oslo.messaging._drivers.common [-] ['Traceback (most
 recent call last):\n', '  File 
"/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", 
line 134, in _dispatch_and_reply\n    incoming.message))\n', '  File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
 line 72, in reply\n    self._send_reply(conn, reply, failure, 
log_failure=log_failure)\n', '  File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
 line 62, in _send_reply\n    conn.direct_send(self.reply_q, 
rpc_common.serialize_msg(msg))\n', '  File 
"/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", 
line 302, in serialize_msg\n    _MESSAGE_KEY: 
jsonutils.dumps(raw_msg)}\n', '  File 
"/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py",
 line 172, in dumps\n    return json.dumps(value, default=default, 
**kwargs)\n', '  File "/usr/lib64/python2.7/json/__init__.py", line 250,
 in dumps\n    sort_keys=sort_keys, **kw).encode(obj)\n', '  File 
"/usr/lib64/python2.7/json/encoder.py", line 207, in encode\n    chunks =
 self.iterencode(o, _one_shot=True)\n', '  File 
"/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode\n    
return _iterencode(o, 0)\n', 'ValueError: Circular reference 
detected\n']<br><br></blockquote><div>This one seems to be similar to the bug reported here, <a href="https://bugs.launchpad.net/nova/+bug/1312002">https://bugs.launchpad.net/nova/+bug/1312002</a> <br></div><div><br></div><div> 2.
 In the second child cell initialization, the error crops up as soon as I
 add the toplevel cell in the child cell using 'nova-manage' command.<br></div><br><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">2014-10-17
 12:05:29.500 ERROR nova.cells.messaging 
[req-f74d05cf-061a-4488-bfcb-0cb1edec44e2 None None] Error locating next
 hop for message: Inconsistency in cell routing: destination is 
cell1!toplevel but routing_path is cell1!cell1<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging Traceback (most recent call last):<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 406, in process<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging     next_hop = self._get_next_hop()<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging   File "/opt/stack/nova/nova/cells/messaging.py", line 375, in _get_next_hop<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging     raise exception.CellRoutingInconsistency(reason=reason)<br>2014-10-17
 12:05:29.500 TRACE nova.cells.messaging CellRoutingInconsistency: 
Inconsistency in cell routing: destination is cell1!toplevel but 
routing_path is cell1!cell1<br>2014-10-17 12:05:29.500 TRACE nova.cells.messaging <br><br></blockquote><div><br><br></div><div>Any ideas?<br></div><br>Regards,<br><br>Vineet Menon</div>