[Openstack] Help with Openstack Liberty: Failed to allocate the network

The O.G. theoriginalguru at gmail.com
Wed Apr 27 22:23:51 UTC 2016 

Hi Remo,

On Thu, Apr 21, 2016 at 7:00 AM, Remo Mattei <remo at italy1.com> wrote:

> has this been a new installation or an upgrade?
>

This is a brand new installation of Openstack/Liberty on CentOS7


> Not sure what you mean with option 2  since the install guide does not say
> anything about options.
>

 If you see here:

http://docs.openstack.org/liberty/install-guide-rdo/neutron-controller-install.html

Under "Configure networking options", there's an option 1 (Provider
network) and option 2 (self-service network). All I'm saying is that we
have been using "option 2" while following the guide.

did you check  if rabbit is up?
>

Yes. We repeated all the "verify operations" steps in the guide to be sure
that the services are working. All verification steps passed. I assume this
means rabbit is up or it would not have passed. The rabbit process is
running also. Is there some other step you recommend to verify the proper
operation of rabbitmq?

did you check and see if the services are all good
>

Well, something is *not* good, hence we have problems. But, if you mean by
"systemctl status <service_name>", then yes, all services appear to be
good. Is there certain commands you would recommend other than systemctl
status to check on services?


> did you check and see if the agents are happy?
>

 I don't know what "happy" means, but this is the output of neutron
agent-list:

# neutron agent-list
+--------------------------------------+--------------------+-------------------------------+-------+----------------+---------------------------+
| id                                   | agent_type         |
host                          | alive | admin_state_up |
binary                    |
+--------------------------------------+--------------------+-------------------------------+-------+----------------+---------------------------+
| 321c6188-0a8b-435e-98c7-42fcd2b9f9de | Linux bridge agent | controller |
:-)   | True           | neutron-linuxbridge-agent |
| 3cc20152-3db7-4a63-8b9e-530cbf272ab3 | Linux bridge agent | compute   |
:-)   | True           | neutron-linuxbridge-agent |
| 60351fc0-873d-4fa3-b295-2146174e567c | Metadata agent     | controller |
:-)   | True           | neutron-metadata-agent    |
| a2dcef04-8ccf-4034-a7f9-4558cc8798f9 | L3 agent           | controller |
:-)   | True           | neutron-l3-agent          |
| d8aed254-edd0-4f6a-a264-4953a1a2c7e8 | DHCP agent         | controller |
:-)   | True           | neutron-dhcp-agent        |
+--------------------------------------+--------------------+-------------------------------+-------+----------------+---------------------------+

If the smily face :-) means happy, then yes I think they are happy. Is
there something else I should check?


> just a few things to check let us know!
>
>
Thanks for your reply. Do you have any hints or insight on the "BadRequest"
message in the logs I posted ?

Bond

Remo
>
> On Apr 20, 2016, at 19:05, The O.G. <theoriginalguru at gmail.com> wrote:
>
> We are following:
>
> http://docs.openstack.org/liberty/install-guide-rdo/index.html
>
> Using CentOS7. We've chosen the option 2 for networking, where we have
> both a private and public network. We've pretty much following the guide
> step by step up to and finishing "Add the dashboard" step. All the
> verification steps passed.
>
> Now, we are trying to launch an instance on the public and private
> networks as outlined here:
>
>
> http://docs.openstack.org/liberty/install-guide-rdo/launch-instance-public.html
>
>
> http://docs.openstack.org/liberty/install-guide-rdo/launch-instance-private.html
>
> In both attempts to launch an instance, the instance gets aborted and
> shows status = Error. With this message:
>
> Build of instance 93bf7968-09b2-4890-9c3a-0d909ce57017 aborted: Failed to
> allocate the network(s), not rescheduling.
>
> On the compute node, in /var/log/nova/nova-compute.log, we get these
> messages (I'm filtering on ERROR|WARNING only):
>
> 2016-04-20 18:35:31.484 860 WARNING nova.virt.libvirt.driver
> [req-b5ec5b9d-2299-411f-b259-7a48b2b47d26 e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Timeout waiting for vif plugging
> callback for instance 93bf7968-09b2-4890-9c3a-0d909ce57017
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager
> [req-b5ec5b9d-2299-411f-b259-7a48b2b47d26 e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Instance failed to spawn
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Traceback (most recent call last):
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in
> _build_resources
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]     yield resources
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in
> _build_and_run_instance
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> block_device_info=block_device_info)
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2515,
> in spawn
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> block_device_info=block_device_info)
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4612,
> in _create_domain_and_network
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]     raise
> exception.VirtualInterfaceCreateException()
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] VirtualInterfaceCreateException:
> Virtual Interface creation failed
> 2016-04-20 18:35:32.007 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager
> [req-b5ec5b9d-2299-411f-b259-7a48b2b47d26 e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Failed to allocate network(s)
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Traceback (most recent call last):
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in
> _build_and_run_instance
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> block_device_info=block_device_info)
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2515,
> in spawn
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> block_device_info=block_device_info)
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4612,
> in _create_domain_and_network
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]     raise
> exception.VirtualInterfaceCreateException()
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] VirtualInterfaceCreateException:
> Virtual Interface creation failed
> 2016-04-20 18:35:32.587 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager
> [req-b5ec5b9d-2299-411f-b259-7a48b2b47d26 e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Build of instance
> 93bf7968-09b2-4890-9c3a-0d909ce57017 aborted: Failed to allocate the
> network(s), not rescheduling.
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] Traceback (most recent call last):
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 1905, in
> _do_build_and_run_instance
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]     filter_properties)
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2044, in
> _build_and_run_instance
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]     reason=msg)
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017] BuildAbortException: Build of
> instance 93bf7968-09b2-4890-9c3a-0d909ce57017 aborted: Failed to allocate
> the network(s), not rescheduling.
> 2016-04-20 18:35:32.590 860 ERROR nova.compute.manager [instance:
> 93bf7968-09b2-4890-9c3a-0d909ce57017]
> 2016-04-20 18:42:55.909 860 WARNING nova.virt.libvirt.driver
> [req-9a205dde-ed7f-43aa-96d4-f6776347359e e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c] Timeout waiting for vif plugging
> callback for instance ae94da8c-3fe6-44d6-93e5-b84a606c725c
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager
> [req-9a205dde-ed7f-43aa-96d4-f6776347359e e11c48fa3827406c9e6304f88332e059
> a8f41d79dc7a490c91013616f66c0134 - - -] [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c] Instance failed to spawn
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c] Traceback (most recent call last):
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2155, in
> _build_resources
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]     yield resources
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]   File
> "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2009, in
> _build_and_run_instance
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]
> block_device_info=block_device_info)
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2515,
> in spawn
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]
> block_device_info=block_device_info)
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]   File
> "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4612,
> in _create_domain_and_network
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]     raise
> exception.VirtualInterfaceCreateException()
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c] VirtualInterfaceCreateException:
> Virtual Interface creation failed
> 2016-04-20 18:42:56.358 860 ERROR nova.compute.manager [instance:
> ae94da8c-3fe6-44d6-93e5-b84a606c725c]
>
>
> On the controller node, in /var/log/neutron/server.log:
>
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova [-] Failed to
> notify nova on events: [{'tag': u'4c0a7262-ac40-4556-bb11-5b20afceec73',
> 'name': 'network-vif-deleted', 'server_uuid':
> u'93bf7968-09b2-4890-9c3a-0d909ce57017'}]
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova Traceback (most
> recent call last):
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py", line 247, in
> send_events
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova
> batched_events)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/v2/contrib/server_external_events.py",
> line 39, in create
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova
> return_raw=True)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/base.py", line 169, in _create
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     _resp, body
> = self.api.client.post(url, body=body)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 176, in
> post
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/client.py", line 91, in request
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 206, in
> request
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     resp =
> super(LegacyJsonAdapter, self).request(*args, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 95, in
> request
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> self.session.request(url, method, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 304, in
> request
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     auth_headers
> = self.get_auth_headers(auth)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 617, in
> get_auth_headers
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> auth.get_headers(self, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/base.py", line 142,
> in get_headers
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     token =
> self.get_token(session)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 200, in get_token
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> self.get_access(session).auth_token
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 240, in get_access
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova
> self.auth_ref = self.get_auth_ref(session)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/generic/base.py",
> line 186, in get_auth_ref
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> self._plugin.get_auth_ref(session, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py",
> line 190, in get_auth_ref
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova
> authenticated=False, log=False, **rkwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 501, in
> post
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 401, in
> request
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova     raise
> exceptions.from_response(resp, method, url)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova BadRequest:
> Expecting to find id or name in user - the server could not comply with the
> request since it is either malformed or otherwise incorrect. The client is
> assumed to be in error. (HTTP 400) (Request-ID:
> req-0a9e36df-420c-46f2-a523-c168f0c295c6)
> 2016-04-20 18:35:35.023 2699 ERROR neutron.notifiers.nova
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova [-] Failed to
> notify nova on events: [{'status': 'completed', 'tag':
> u'67291fab-d2d6-42ec-ae4e-6e5ed60fa424', 'name': 'network-vif-plugged',
> 'server_uuid': u'ae94da8c-3fe6-44d6-93e5-b84a606c725c'}]
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova Traceback (most
> recent call last):
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py", line 247, in
> send_events
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova
> batched_events)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/v2/contrib/server_external_events.py",
> line 39, in create
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova
> return_raw=True)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/base.py", line 169, in _create
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     _resp, body
> = self.api.client.post(url, body=body)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 176, in
> post
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/client.py", line 91, in request
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 206, in
> request
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     resp =
> super(LegacyJsonAdapter, self).request(*args, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 95, in
> request
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> self.session.request(url, method, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 304, in
> request
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     auth_headers
> = self.get_auth_headers(auth)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 617, in
> get_auth_headers
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> auth.get_headers(self, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/base.py", line 142,
> in get_headers
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     token =
> self.get_token(session)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 200, in get_token
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> self.get_access(session).auth_token
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 240, in get_access
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova
> self.auth_ref = self.get_auth_ref(session)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/generic/base.py",
> line 186, in get_auth_ref
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> self._plugin.get_auth_ref(session, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py",
> line 190, in get_auth_ref
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova
> authenticated=False, log=False, **rkwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 501, in
> post
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 401, in
> request
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova     raise
> exceptions.from_response(resp, method, url)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova BadRequest:
> Expecting to find id or name in user - the server could not comply with the
> request since it is either malformed or otherwise incorrect. The client is
> assumed to be in error. (HTTP 400) (Request-ID:
> req-7e1674f8-70f3-4c45-b5e1-3c6739bc745f)
> 2016-04-20 18:38:00.063 2701 ERROR neutron.notifiers.nova
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova [-] Failed to
> notify nova on events: [{'tag': u'67291fab-d2d6-42ec-ae4e-6e5ed60fa424',
> 'name': 'network-vif-deleted', 'server_uuid':
> u'ae94da8c-3fe6-44d6-93e5-b84a606c725c'}]
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova Traceback (most
> recent call last):
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/neutron/notifiers/nova.py", line 247, in
> send_events
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova
> batched_events)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/v2/contrib/server_external_events.py",
> line 39, in create
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova
> return_raw=True)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/base.py", line 169, in _create
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     _resp, body
> = self.api.client.post(url, body=body)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 176, in
> post
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/novaclient/client.py", line 91, in request
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 206, in
> request
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     resp =
> super(LegacyJsonAdapter, self).request(*args, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/adapter.py", line 95, in
> request
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> self.session.request(url, method, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 304, in
> request
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     auth_headers
> = self.get_auth_headers(auth)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 617, in
> get_auth_headers
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> auth.get_headers(self, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/base.py", line 142,
> in get_headers
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     token =
> self.get_token(session)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 200, in get_token
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> self.get_access(session).auth_token
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/base.py",
> line 240, in get_access
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova
> self.auth_ref = self.get_auth_ref(session)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/generic/base.py",
> line 186, in get_auth_ref
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> self._plugin.get_auth_ref(session, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/auth/identity/v3/base.py",
> line 190, in get_auth_ref
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova
> authenticated=False, log=False, **rkwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 501, in
> post
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> self.request(url, 'POST', **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/utils.py", line 337, in
> inner
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     return
> func(*args, **kwargs)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova   File
> "/usr/lib/python2.7/site-packages/keystoneclient/session.py", line 401, in
> request
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova     raise
> exceptions.from_response(resp, method, url)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova BadRequest:
> Expecting to find id or name in user - the server could not comply with the
> request since it is either malformed or otherwise incorrect. The client is
> assumed to be in error. (HTTP 400) (Request-ID:
> req-022851be-63e5-4f22-851d-60536516b686)
> 2016-04-20 18:42:59.448 2699 ERROR neutron.notifiers.nova
>
>
> It's not obvious to me how to fix this problem. Can someone please provide
> some guidance? What else should I look at? What service is nova calling
> when it times out? What is the "BadRequest" about? (where can I find the
> actual request? and what should it look like?)
>
> Any help would be appreciated...
> Bond
> !DSPAM:1,571839d9243111631916134!
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to     : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
> !DSPAM:1,571839d9243111631916134!
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20160427/daec6a7b/attachment.html>

More information about the Openstack mailing list