Open Stack

Hi,

I was trying to create cells under openstack using devstack. My setup
contains 3 machines. One toplevel and 2 compute cells.
I'm following this documentation,
http://docs.openstack.org/trunk/config-reference/content/section_compute-cells.html
.

Both these cells instantiation are generating errors.
1. first one doesn't generate any error logs unless I issue a command at
the parent 'nova cell-show cell2'. At this point the toplevel cell throws
the following error,

2014-10-17 12:03:34.888 ERROR oslo.messaging.rpc.dispatcher [-] Exception
> during message handling: Circular reference detected
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher Traceback
> (most recent call last):
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line
> 134, in _dispatch_and_reply
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> incoming.message))
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
> line 72, in reply
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> self._send_reply(conn, reply, failure, log_failure=log_failure)
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
> line 62, in _send_reply
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> conn.direct_send(self.reply_q, rpc_common.serialize_msg(msg))
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", line
> 302, in serialize_msg
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> _MESSAGE_KEY: jsonutils.dumps(raw_msg)}
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py",
> line 172, in dumps
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     return
> json.dumps(value, default=default, **kwargs)
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib64/python2.7/json/__init__.py", line 250, in dumps
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> sort_keys=sort_keys, **kw).encode(obj)
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib64/python2.7/json/encoder.py", line 207, in encode
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     chunks =
> self.iterencode(o, _one_shot=True)
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher   File
> "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher     return
> _iterencode(o, 0)
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher ValueError:
> Circular reference detected
> 2014-10-17 12:03:34.888 TRACE oslo.messaging.rpc.dispatcher
> 2014-10-17 12:03:34.890 ERROR oslo.messaging._drivers.common [-] Returning
> exception Circular reference detected to caller
> 2014-10-17 12:03:34.890 ERROR oslo.messaging._drivers.common [-]
> ['Traceback (most recent call last):\n', '  File
> "/usr/lib/python2.7/site-packages/oslo/messaging/rpc/dispatcher.py", line
> 134, in _dispatch_and_reply\n    incoming.message))\n', '  File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
> line 72, in reply\n    self._send_reply(conn, reply, failure,
> log_failure=log_failure)\n', '  File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/amqpdriver.py",
> line 62, in _send_reply\n    conn.direct_send(self.reply_q,
> rpc_common.serialize_msg(msg))\n', '  File
> "/usr/lib/python2.7/site-packages/oslo/messaging/_drivers/common.py", line
> 302, in serialize_msg\n    _MESSAGE_KEY: jsonutils.dumps(raw_msg)}\n', '
> File
> "/usr/lib/python2.7/site-packages/oslo/messaging/openstack/common/jsonutils.py",
> line 172, in dumps\n    return json.dumps(value, default=default,
> **kwargs)\n', '  File "/usr/lib64/python2.7/json/__init__.py", line 250, in
> dumps\n    sort_keys=sort_keys, **kw).encode(obj)\n', '  File
> "/usr/lib64/python2.7/json/encoder.py", line 207, in encode\n    chunks =
> self.iterencode(o, _one_shot=True)\n', '  File
> "/usr/lib64/python2.7/json/encoder.py", line 270, in iterencode\n    return
> _iterencode(o, 0)\n', 'ValueError: Circular reference detected\n']
>
> This one seems to be similar to the bug reported here,
https://bugs.launchpad.net/nova/+bug/1312002

 2. In the second child cell initialization, the error crops up as soon as
I add the toplevel cell in the child cell using 'nova-manage' command.

2014-10-17 12:05:29.500 ERROR nova.cells.messaging
> [req-f74d05cf-061a-4488-bfcb-0cb1edec44e2 None None] Error locating next
> hop for message: Inconsistency in cell routing: destination is
> cell1!toplevel but routing_path is cell1!cell1
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging Traceback (most recent
> call last):
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging   File
> "/opt/stack/nova/nova/cells/messaging.py", line 406, in process
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging     next_hop =
> self._get_next_hop()
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging   File
> "/opt/stack/nova/nova/cells/messaging.py", line 375, in _get_next_hop
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging     raise
> exception.CellRoutingInconsistency(reason=reason)
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging
> CellRoutingInconsistency: Inconsistency in cell routing: destination is
> cell1!toplevel but routing_path is cell1!cell1
> 2014-10-17 12:05:29.500 TRACE nova.cells.messaging
>
>

Any ideas?

Regards,

Vineet Menon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20141017/2c62e906/attachment.html>