Open Stack

Hi Florian

I observed this kind of issue when there is race condition.

1. RabbitMQ is NOT running on controller-node
2. You issue nova-network from compute-node. Now this step hangs.
3. Just see what is happening in nova-api.log on controller(tail -f
/var/log/nova/nova-api.log)
4. Now you start the RabbitMQ on the controller node.
5. nova-network command compute returns the error. Also you will see
'timeout' error in nova-api.log.

>From your log I suspect this because initially RabitMQ was not up. Looks
like you started it afterwards.
Did you try to run the same 'nova-network' command for the second time ?
i.e after restarting the rabbitMQ ?

-Dheerendra



On Wed, Oct 30, 2013 at 3:47 PM, Florian Lindner <mailinglists at xgm.de>wrote:

> Am Dienstag, 29. Oktober 2013, 14:32:00 schrieb Razique Mahroua:
> > You are using Neutron?
> > Looks like the Neutron server is unable to perform the request actually
>
> Actually no, I don't think so, I think I use the nova-network.
>
> The documentation
> http://docs.openstack.org/havana/install-guide/install/apt/content/ says:
>
> Configuring Networking can be one of the most bewildering experiences you
> will
> encounter when working with OpenStack. To assist in this we have chosen the
> simplest production-ready configuration for this guide: the legacy
> networking
> in OpenStack Compute, with a flat network, that takes care of DHCP.
>
> This setup uses "multi-host" functionality: the networking is configured
> to be
> highly available by splitting networking functionality across multiple
> hosts.
> As a result, there is no single network controller that acts as a single
> point
> of failure. Because each compute node is configured for networking in this
> setup, no additional networking configuration is required on the
> controller.
>
> (Chapter 5: Enabling Networking)
>
> and I have installed the nova-network package.
>
> I've downloaded my entire /etc to http://xgm.de/upload/etc/ if you or
> anyone
> else wants to have a look! Yes, these "123" are the actual passwords, it's
> just from a VM on my local computer.  ;-)
>
> Thanks a lot!
>
> Florian
>
> >
> >
> > On October 29, 2013 at 12:00:18, Florian Lindner (mailinglists at xgm.de)
> > wrote:
> > Am Montag, 28. Oktober 2013, 13:44:00 schrieb Craig E. Ward:
> > > On the RabbitMQ server, are vhost and user id setup correctly?
> >
> > Ok, part of the problem seemed to be that the documentation assumed I set
> > the password guest for the rabbit guest account. Did that now and the
> > problem changed:
> >
> > root at controller:~# nova network-create vmnet --fixed-range-v4=
> 10.0.0.0/24 --
> > bridge-interface=br100 --multi-host=T
> > ERROR: The server has either erred or is incapable of performing the
> > requested operation. (HTTP 500) (Request-ID:
> > req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3)
> >
> > nova-api.log shows:
> >
> > http://pastebin.com/raw.php?i=13U8a4Na
> >
> >
> > 2013-10-29 19:14:18.587 1272 INFO nova.osapi_compute.wsgi.server [-]
> (1272)
> > accepted ('192.168.122.10', 53116)
> > 2013-10-29 19:14:19.207 1272 INFO nova.openstack.common.rpc.common
> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> a1dd41c156c64810b41d46d4062a2b17
> > 68d6cb4e2ceb4700931b6489ad25fb6f] Connected to AMQP server on
> > controller:5672 2013-10-29 19:14:19.218 1272 INFO
> > nova.openstack.common.rpc.common
> [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> > a1dd41c156c64810b41d46d4062a2b17 68d6cb4e2ceb4700931b6489ad25fb6f]
> > Connected to AMQP server on controller:5672 2013-10-29 19:15:19.238 1272
> > ERROR nova.api.openstack [req-900a80bd- dfc1-49fc-9599-239fd5b9b7e3
> > a1dd41c156c64810b41d46d4062a2b17
> > 68d6cb4e2ceb4700931b6489ad25fb6f] Caught error: Timeout while waiting on
> RPC
> > response - topic: "network", RPC method: "create_networks" info:
> > "<unknown>" 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack
> Traceback
> > (most recent call last):
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/__init__.py", line
> 119,
> > in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > req.get_response(self.application)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1296, in send
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack application,
> > catch_exc_info=False)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/request.py", line 1260, in
> > call_application
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack app_iter =
> > application(self.environ, start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> resp(environ,
> > start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> >
> "/usr/lib/python2.7/dist-packages/keystoneclient/middleware/auth_token.py",
> > line 539, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> self.app(env,
> > start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> resp(environ,
> > start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> resp(environ,
> > start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/routes/middleware.py", line 131, in
> > __call__ 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack response =
> > self.app(environ, start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 144, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> resp(environ,
> > start_response)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 130, in __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack resp =
> > self.call_func(req, *args, **self.kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/webob/dec.py", line 195, in call_func
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > self.func(req, *args, **kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 917,
> in
> > __call__
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack content_type, body,
> > accept)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line 976,
> in
> > _process_stack
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack action_result =
> > self.dispatch(meth, request, action_args)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/api/openstack/wsgi.py", line
> 1057, in
> > dispatch
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > method(req=request, **action_args)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-
> > packages/nova/api/openstack/compute/contrib/os_networks.py", line 126, in
> > create
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack network =
> > self.network_api.create(context, **params)[0]
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 93, in
> wrapped
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return func(self,
> > context, *args, **kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/network/api.py", line 142, in
> create
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > self.network_rpcapi.create_networks(context, **kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/network/rpcapi.py", line 102, in
> > create_networks
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > self.client.call(ctxt, 'create_networks', **kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 85, in call
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > self._invoke(self.proxy.call, ctxt, method, **kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/rpcclient.py", line 63, in _invoke
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack return
> > cast_or_call(ctxt, msg, **self.kwargs)
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack File
> > "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/proxy.py",
> line
> > 130, in call
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack exc.info,
> > real_topic, msg.get('method'))
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack Timeout: Timeout
> while
> > waiting on RPC response - topic: "network", RPC method: "create_networks"
> > info: "<unknown>"
> > 2013-10-29 19:15:19.238 1272 TRACE nova.api.openstack
> > 2013-10-29 19:15:19.249 1272 INFO nova.api.openstack [req-900a80bd-
> > dfc1-49fc-9599-239fd5b9b7e3 a1dd41c156c64810b41d46d4062a2b17
> > 68d6cb4e2ceb4700931b6489ad25fb6f]
> > http://controller:8774/v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks
> > returned with HTTP 500
> > 2013-10-29 19:15:19.255 1272 INFO nova.osapi_compute.wsgi.server
> > [req-900a80bd-dfc1-49fc-9599-239fd5b9b7e3
> a1dd41c156c64810b41d46d4062a2b17
> > 68d6cb4e2ceb4700931b6489ad25fb6f] 192.168.122.10 "POST
> > /v2/68d6cb4e2ceb4700931b6489ad25fb6f/os-networks HTTP/1.1" status: 500
> len:
> > 335 time: 60.6673729
> >
> > Sorry, I got not idea...
> >
> > Thanks!
> > Florian
> >
> > > % rabbitmqctl list_users
> > >
> > > % rabbitmqctl list_vhosts
> > >
> > > The user and vhost should match what is in the OpenStack
> configurations.
> > >
> > > On 10/28/13 1:14 PM, Florian Lindner wrote:
> > > > Am Montag, 28. Oktober 2013, 12:04:02 schrieb Razique Mahroua:
> > > >> hey
> > > >> is «controller» a hostame all ur servers can resolve?
> > > >
> > > > Yes, set in /etc/hosts.
> > > >
> > > > Regards,
> > > > Florian
> > > >
> > > >> - Razique
> > > >>
> > > >> On Oct 28, 2013, at 11:40, Florian Lindner <mailinglists at xgm.de>
> wrote:
> > > >>> Hello,
> > > >>>
> > > >>> I've installed OpenStack on a Ubuntu 12.04 VM according to the
> install
> > > >>> guide. It worked fine until I created the compute node on a second
> VM
> > > >>> and
> > > >>> tried to set up nova networking:
> > > >>>
> > > >>>
> http://docs.openstack.org/havana/install-guide/install/apt/content/nov
> > > >>> a-> >>> ne twork.html
> > > >>>
> > > >>> # nova network-create vmnet --fixed-range-v4=10.0.0.0/24 --bridge-
> > > >>> interface=br100 --multi-host=T
> > > >>>
> > > >>> just does nothing until canceled. I tried on the controller and
> > > >>> compute1
> > > >>> node.
> > > >>>
> > > >>> nova-api.log shows:
> > > >>>
> > > >>> ERROR nova.openstack.common.rpc.common AMQP server on
> controller:5672
> > > >>> is
> > > >>> unreachable: Socket closed. Trying again in 1 seconds.
> > > >>> INFO nova.openstack.common.rpc.common Reconnecting to AMQP server
> on
> > > >>> controller:5672
> > > >>>
> > > >>> controller:~# netstat -lpn | grep beam
> > > >>> tcp 0 0 0.0.0.0:36595 0.0.0.0:*
> > > >>> LISTEN
> > > >>> 1045/beam
> > > >>> tcp6 0 0 :::5672 :::*
> > > >>> LISTEN
> > > >>> 1045/beam
> > > >>>
> > > >>> It listens on this port, but IPv6. IMO on linux listing on an IPv6
> > > >>> port
> > > >>> also means listening on the same v4 port.
> > > >>>
> > > >>> What's wrong with the installation?
>
>
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20131030/5d8103e0/attachment.html>