[Openstack] Can't connect to RabbitMQ

Dheerendra dheerendra.madhusudhana at gmail.com
Wed Oct 30 17:05:34 UTC 2013


Good. Can you share the configuration file-nova.conf on compute ? It will
help everybody.

-Dheerendra


On Wed, Oct 30, 2013 at 8:25 PM, Florian Lindner <mailinglists at xgm.de>wrote:

> Am Mittwoch, 30. Oktober 2013, 19:02:33 schrieb Dheerendra:
> > also look at the logs /var/log/nova/nova-network.log on compute node. It
> > should tell whether it is able to connect to rabbit on controller.
>
> I got now! The problem was a missing database entry on the compute node.
> Sorry
> guys for this stupid error, though the docs don't mention creating the
> entry.
>
> Thanks for your help!
> Florian
>
> >
> > -Dheerendra
> >
> >
> > On Wed, Oct 30, 2013 at 6:41 PM, Dheerendra <
> >
> > dheerendra.madhusudhana at gmail.com> wrote:
> > > Hi Florian
> > >
> > > I observed this kind of issue when there is race condition.
> > >
> > > 1. RabbitMQ is NOT running on controller-node
> > > 2. You issue nova-network from compute-node. Now this step hangs.
> > > 3. Just see what is happening in nova-api.log on controller(tail -f
> > > /var/log/nova/nova-api.log)
> > > 4. Now you start the RabbitMQ on the controller node.
> > > 5. nova-network command compute returns the error. Also you will see
> > > 'timeout' error in nova-api.log.
> > >
> > > From your log I suspect this because initially RabitMQ was not up.
> Looks
> > > like you started it afterwards.
> > > Did you try to run the same 'nova-network' command for the second time
> ?
> > > i.e after restarting the rabbitMQ ?
> > >
> > > -Dheerendra
> > >
> > > On Wed, Oct 30, 2013 at 3:47 PM, Florian Lindner
> <mailinglists at xgm.de>wrote:
> > >> Am Dienstag, 29. Oktober 2013, 14:32:00 schrieb Razique Mahroua:
> > >> > You are using Neutron?
> > >> > Looks like the Neutron server is unable to perform the request
> actually
> > >>
> > >> Actually no, I don't think so, I think I use the nova-network.
> > >>
> > >> The documentation
> > >> http://docs.openstack.org/havana/install-guide/install/apt/content/says:
> > >>
> > >> Configuring Networking can be one of the most bewildering experiences
> you
> > >> will
> > >> encounter when working with OpenStack. To assist in this we have
> chosen
> > >> the
> > >> simplest production-ready configuration for this guide: the legacy
> > >> networking
> > >> in OpenStack Compute, with a flat network, that takes care of DHCP.
> > >>
> > >> This setup uses "multi-host" functionality: the networking is
> configured
> > >> to be
> > >> highly available by splitting networking functionality across multiple
> > >> hosts.
> > >> As a result, there is no single network controller that acts as a
> single
> > >> point
> > >> of failure. Because each compute node is configured for networking in
> > >> this
> > >> setup, no additional networking configuration is required on the
> > >> controller.
> > >>
> > >> (Chapter 5: Enabling Networking)
> > >>
> > >> and I have installed the nova-network package.
> > >>
> > >> I've downloaded my entire /etc to http://xgm.de/upload/etc/ if you or
> > >> anyone
> > >> else wants to have a look! Yes, these "123" are the actual passwords,
> > >> it's
> > >> just from a VM on my local computer.  ;-)
> > >>
> > >> Thanks a lot!
> > >>
> > >> Florian
> > >>
> > >> > On October 29, 2013 at 12:00:18, Florian Lindner (
> mailinglists at xgm.de)
> > >> > wrote:
> > >> >
> > >> > Am Montag, 28. Oktober 2013, 13:44:00 schrieb Craig E. Ward:
> > >> > > On the RabbitMQ server, are vhost and user id setup correctly?
> > >> >
> > >> > Ok, part of the problem seemed to be that the documentation assumed
> I
> > >>
> > >> set
> > >>
> > >> > the password guest for the rabbit guest account. Did that now and
> the
> > >> > problem changed:
> > >> >
> > >> > root at controller:~# nova network-create vmnet --fixed-range-v4=
> > >>
> > >> 10.0.0.0/24 --
> > >>
> > >> > bridge-interface=br100 --multi-host=T
> > >> > ERROR: The server has either erred or is incapable of performing the
> > >> > requested operation. (HTTP 500) (Request-ID:
> > >> > req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3)
> > >> >
> > >> > nova-api.log shows:
> > >> >
> > >> > http://pastebin.com/raw.php?i=13U8a4Na
> > >> >
> > >> >
> > >> > 2013-10-29 19:14:18.587 1272 INFO nova.osapi_compute.wsgi.server [-]
> > >>
> > >> (1272)
> > >>
> > >> > accepted ('192.168.122.10', 53116)
> > >> > 2013-10-29 19:14:19.207 1272 INFO nova.openstack.common.rpc.common
> > >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> > >>
> > >> a1dd41c156c64810b41d46d4062a2b17
> > >>
> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Connected to AMQP server on
> > >> > controller:5672 2013-10-29 19:14:19.218 1272 INFO
> > >> > nova.openstack.common.rpc.common
> > >>
> > >> [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> > >>
> > >> > a1dd41c156c64810b41d46d4062a2b17 68d6cb4e2ceb4700931b6489ad25fb6f]
> > >> > Connected to AMQP server on controller:5672 2013-10-29 19:15:19.238
> > >> > 1272
> > >> > ERROR nova.api.openstack [req-900a80bd- dfc1-49fc-9599-239fd5b9b7e3
> > >> > a1dd41c156c64810b41d46d4062a2b17
> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] Caught error: Timeout while
> waiting
> > >>
> > >> on RPC
> > >>
> > >> > response - topic: "network", RPC method: "create_networks" info:
> > >> > "<unknown>" 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack
> > >>
> > >> Traceback
> > >>
> > >> > (most recent call last):
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py",
> line
> > >>
> > >> 119,
> > >>
> > >> > in __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > req.get_response(self.application)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in
> send
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack application,
> > >> > catch_exc_info=False)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in
> > >> > call_application
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack app_iter =
> > >> > application(self.environ, start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in
> __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >>
> > >> resp(environ,
> > >>
> > >> > start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >>
> > >>
> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py
> > >> ",
> > >>
> > >> > line 539, in __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >>
> > >> self.app(env,
> > >>
> > >> > start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in
> __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >>
> > >> resp(environ,
> > >>
> > >> > start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in
> __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >>
> > >> resp(environ,
> > >>
> > >> > start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131,
> in
> > >> > __call__ 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack
> response
> > >>
> > >> =
> > >>
> > >> > self.app(environ, start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in
> __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >>
> > >> resp(environ,
> > >>
> > >> > start_response)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in
> __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack resp =
> > >> > self.call_func(req, *args, **self.kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in
> call_func
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > self.func(req, *args, **kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line
> > >>
> > >> 917, in
> > >>
> > >> > __call__
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack content_type,
> > >>
> > >> body,
> > >>
> > >> > accept)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line
> > >>
> > >> 976, in
> > >>
> > >> > _process_stack
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack action_result
> =
> > >> > self.dispatch(meth, request, action_args)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line
> > >>
> > >> 1057, in
> > >>
> > >> > dispatch
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > method(req=request, **action_args)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-
> > >> > packages/nova/api/openstack/compute/contrib/os_networks.py", line
> 126,
> > >>
> > >> in
> > >>
> > >> > create
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack network =
> > >> > self.network_api.create(context, **params)[0]
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 93, in
> > >>
> > >> wrapped
> > >>
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> func(self,
> > >> > context, *args, **kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 142, in
> > >>
> > >> create
> > >>
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > self.network_rpcapi.create_networks(context, **kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line
> 102, in
> > >> > create_networks
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > self.client.call(ctxt, 'create_networks', **kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 85, in
> call
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > self._invoke(self.proxy.call, ctxt, method, **kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 63, in
> > >>
> > >> _invoke
> > >>
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > >> > cast_or_call(ctxt, msg, **self.kwargs)
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > >> >
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py",
> > >>
> > >> line
> > >>
> > >> > 130, in call
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack exc.info,
> > >> > real_topic, msg.get('method'))
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack Timeout:
> Timeout
> > >>
> > >> while
> > >>
> > >> > waiting on RPC response - topic: "network", RPC method:
> > >> "create_networks"
> > >>
> > >> > info: "<unknown>"
> > >> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack
> > >> > 2013-10-29 19:15:19.249 1272 INFO nova.api.openstack [req-900a80bd-
> > >> > dfc1-49fc-9599-239fd5b9b7e3 a1dd41c156c64810b41d46d4062a2b17
> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f]
> > >> >
> http://controller:8774/v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks
> > >> > returned with HTTP 500
> > >> > 2013-10-29 19:15:19.255 1272 INFO nova.osapi_compute.wsgi.server
> > >> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> > >>
> > >> a1dd41c156c64810b41d46d4062a2b17
> > >>
> > >> > 68d6cb4e2ceb4700931b6489ad25fb6f] 192.168.122.10 "POST
> > >> > /v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks HTTP/1.1" status:
> 500
> > >>
> > >> len:
> > >> > 335 time: 60.6673729
> > >> >
> > >> > Sorry, I got not idea...
> > >> >
> > >> > Thanks!
> > >> > Florian
> > >> >
> > >> > > % rabbitmqctl list_users
> > >> > >
> > >> > > % rabbitmqctl list_vhosts
> > >> > >
> > >> > > The user and vhost should match what is in the OpenStack
> > >>
> > >> configurations.
> > >>
> > >> > > On 10/28/13 1:14 PM, Florian Lindner wrote:
> > >> > > > Am Montag, 28. Oktober 2013, 12:04:02 schrieb Razique Mahroua:
> > >> > > >> hey
> > >> > > >> is «controller» a hostame all ur servers can resolve?
> > >> > > >
> > >> > > > Yes, set in /etc/hosts.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Florian
> > >> > > >
> > >> > > >> - Razique
> > >> > > >>
> > >> > > >> On Oct 28, 2013, at 11:40, Florian Lindner <
> mailinglists at xgm.de>
> > >>
> > >> wrote:
> > >> > > >>> Hello,
> > >> > > >>>
> > >> > > >>> I've installed OpenStack on a Ubuntu 12.04 VM according to the
> > >>
> > >> install
> > >>
> > >> > > >>> guide. It worked fine until I created the compute node on a
> > >>
> > >> second VM
> > >>
> > >> > > >>> and
> > >>
> > >> > > >>> tried to set up nova networking:
> > >>
> http://docs.openstack.org/havana/install-guide/install/apt/content/nov
> > >>
> > >> > > >>> a-> >>> ne twork.html
> > >> > > >>>
> > >> > > >>> # nova network-create vmnet --fixed-range-v4=
> 10.0.0.0/24--bridge-
> > >> > > >>> interface=br100 --multi-host=T
> > >> > > >>>
> > >> > > >>> just does nothing until canceled. I tried on the controller
> and
> > >> > > >>> compute1
> > >> > > >>> node.
> > >> > > >>>
> > >> > > >>> nova-api.log shows:
> > >> > > >>>
> > >> > > >>> ERROR nova.openstack.common.rpc.common AMQP server on
> > >>
> > >> controller:5672
> > >>
> > >> > > >>> is
> > >> > > >>> unreachable: Socket closed. Trying again in 1 seconds.
> > >> > > >>> INFO nova.openstack.common.rpc.common Reconnecting to AMQP
> server
> > >>
> > >> on
> > >>
> > >> > > >>> controller:5672
> > >> > > >>>
> > >> > > >>> controller:~# netstat -lpn | grep beam
> > >> > > >>> tcp 0 0 0.0.0.0:36595 0.0.0.0:*
> > >> > > >>> LISTEN
> > >> > > >>> 1045/beam
> > >> > > >>> tcp6 0 0 :::5672 :::*
> > >> > > >>> LISTEN
> > >> > > >>> 1045/beam
> > >> > > >>>
> > >> > > >>> It listens on this port, but IPv6. IMO on linux listing on an
> > >> > > >>> IPv6
> > >> > > >>> port
> > >> > > >>> also means listening on the same v4 port.
> > >> > > >>>
> > >> > > >>> What's wrong with the installation?
> > >>
> > >> _______________________________________________
> > >> Mailing list:
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> > >> Post to     : openstack at lists.openstack.org
> > >> Unsubscribe :
> > >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131030/83f4b754/attachment.html>


More information about the Openstack mailing list