[Openstack] Devstack - : Failed to allocate the network(s), not rescheduling

Ramu, MohanX mohanx.ramu at intel.com
Fri Dec 15 13:04:08 UTC 2017 

Hi Trinath,

We don't have /etc/neutron/plugins/ml2/openvswitch_agent.ini file. We use basic devstack, we don't have any addition into it. Please suggest us what else we should do configuration when we use  Devstack set up. We thought that Devstack itself it has all the required configurations. No much idea about Devstack.

But we had another set ups' OpenStack PIKE, OCATA and MITAKA(without devstack), there we have used LinuxBridge agent. It is working perfect.

Thanks & Regards,
Mohan Ramu

From: Trinath Somanchi [mailto:trinath.somanchi at nxp.com]
Sent: Friday, December 15, 2017 5:29 PM
To: Ramu, MohanX <mohanx.ramu at intel.com>; Bernd Bausch <berndbausch at gmail.com>; openstack at lists.openstack.org
Cc: Sivapuram, Naga SeshendraX <naga.seshendrax.sivapuram at intel.com>
Subject: RE: [Openstack] Devstack - : Failed to allocate the network(s), not rescheduling

What is the of_listen_port configured in [ovs] at /etc/neutron/plugins/ml2/openvswitch_agent.ini (or agent config file).

The error, 2017-12-14 21:29:36.326 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [-] Switch connection timeout

Is caused to bridge configuration issues.

Try, of_listen_port = 7733 and spawn the VMs.

Hope this helps,

/
  Trinath Somanchi,
   NXP SDN/NFV Team.

From: Ramu, MohanX [mailto:mohanx.ramu at intel.com]
Sent: Friday, December 15, 2017 4:58 PM
To: Bernd Bausch <berndbausch at gmail.com<mailto:berndbausch at gmail.com>>; openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Cc: Sivapuram, Naga SeshendraX <naga.seshendrax.sivapuram at intel.com<mailto:naga.seshendrax.sivapuram at intel.com>>
Subject: Re: [Openstack] Devstack - : Failed to allocate the network(s), not rescheduling

Hi Bernd,

PFA the local.conf files.

Below are the details used for our devstack environment.

OpenStack version : 15.0.9
Network Used : public

We are not able to launch instances every time. Sometimes instances are launching without any issue. Below are the screenshots for launching multiple instances same time.

Launched 5 instances same time, only 2 instances are created.


[cid:image001.jpg at 01D375D2.6687E3C0]

Tried 10 instances same time 5 instances are created.


[cid:image002.jpg at 01D375D2.6687E3C0]


Please find logs both in controller and compute side.


q-agt.log (Controller Side) :-

2017-12-14 22:39:32.880 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:37.884 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:37.963 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:42.861 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:47.865 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:47.965 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:48.074 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [-] Switch connection timeout
2017-12-14 22:39:48.075 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 4 from (pid=6178) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 22:39:48.076 DEBUG neutron.agent.ovsdb.impl_idl [-] Running txn command(idx=0): DbGetCommand(column=datapath_id, table=Bridge, record=br-int) from (pid=6178) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:100
2017-12-14 22:39:48.077 DEBUG neutron.agent.ovsdb.impl_idl [-] Transaction caused no change from (pid=6178) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:124
2017-12-14 22:39:48.078 INFO neutron.plugins.ml2.drive

q-agt.log (Compute Side) :-

2017-12-14 21:29:38.212 DEBUG neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [req-dadb9cbf-741e-4378-bfc3-9dabf5e78a7d None None] ofctl request version=0x4,msg_type=0x12,msg_len=0x38,xid=0x86ee6f60,OFPFlowStatsRequest(cookie=0,cookie_mask=0,flags=0,match=OFPMatch(oxm_fields={}),out_group=4294967295,out_port=4294967295,table_id=23,type=1) result [OFPFlowStatsReply(body=[OFPFlowStats(byte_count=0,cookie=9413309945793027316L,duration_nsec=913000000,duration_sec=162395,flags=0,hard_timeout=0,idle_timeout=0,instructions=[],length=56,match=OFPMatch(oxm_fields={}),packet_count=0,priority=0,table_id=23)],flags=0,type=1)] from (pid=517) _send_msg /opt/stack/neutron/neutron/plugins/ml2/drivers/openvswitch/agent/openflow/native/o2017-12-14 21:29:31.874 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=7063) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:36.090 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=7063) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:36.875 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=7063) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:31.874 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=3621) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:36.090 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=3621) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:36.326 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [-] Switch connection timeout
2017-12-14 21:29:36.326 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 4 from (pid=3621) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:36.327 DEBUG neutron.agent.ovsdb.impl_idl [-] Running txn command(idx=0): DbGetCommand(column=datapath_id, table=Bridge, record=br-int) from (pid=3621) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:100
2017-12-14 21:29:36.327 DEBUG neutron.agent.ovsdb.impl_idl [-] Transaction caused no change from (pid=3621) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:127
2017-12-14 21:29:36.327 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-int changed its datapath-ID from de78d015444e to 0000de78d015444e
2017-12-14 21:29:36.875 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=3621) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:31.873 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=815) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:31.875 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=815) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:34.346 ERROR neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ofswitch [-] Switch connection timeout
2017-12-14 21:29:34.346 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 4 from (pid=815) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202
2017-12-14 21:29:34.346 DEBUG neutron.agent.ovsdb.impl_idl [-] Running txn command(idx=0): DbGetCommand(column=datapath_id, table=Bridge, record=br-int) from (pid=815) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:100
2017-12-14 21:29:34.346 DEBUG neutron.agent.ovsdb.impl_idl [-] Transaction caused no change from (pid=815) do_commit /opt/stack/neutron/neutron/agent/ovsdb/impl_idl.py:127
2017-12-14 21:29:34.347 INFO neutron.plugins.ml2.drivers.openvswitch.agent.openflow.native.ovs_bridge [-] Bridge br-int changed its datapath-ID from de78d015444e to 0000de78d015444e
2017-12-14 21:29:36.875 DEBUG neutron.agent.ovsdb.native.vlog [-] [POLLIN] on fd 3 from (pid=815) __log_wakeup /usr/local/lib/python2.7/dist-packages/ovs/poller.py:202

n-cpu.log (Compute Side) :-

2017-12-14 21:24:03.442 DEBUG keystoneauth1.identity.base [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mself.auth_ref ---><keystoneauth1.access.access.AccessInfoV3 object at 0x7ff9d42a2ed0>^[[00m ^[[00;33mfrom (pid=7061) get_access /usr/local/lib/python2.7/dist-packages/keystoneauth1/identity/base.py:138^[[00m^M
2017-12-14 21:24:03.442 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mquery_params  --> {}^[[00m^M
2017-12-14 21:24:03.443 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35murl -----::-> http://10.105.166.213/placement/resource_providers/dcf28911-6125-4bc4-87b6-bc71b913d8b2/inventories^[[00m^M<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F10.105.166.213%2Fplacement%2Fresource_providers%2Fdcf28911-6125-4bc4-87b6-bc71b913d8b2%2Finventories%255e%255b%255b00m%255eM&data=02%7C01%7Ctrinath.somanchi%40nxp.com%7C4cb98e3055544767b83a08d543b021c7%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636489346125753805&sdata=DmzSzlumR5kY0JmoLYS2ghpPSauqJFmVJ7OhXIFfkvU%3D&reserved=0>
2017-12-14 21:24:03.443 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mBefore connection_params^[[00m^M
2017-12-14 21:24:03.443 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mAfter connection_params --> {}^[[00m^M
2017-12-14 21:24:03.443 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mBefore send(**kwargs) ^[[00m^M
2017-12-14 21:24:03.443 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m_send_request call^[[00m^M
2017-12-14 21:24:03.520 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m_send_request ---> <Response [200]>^[[00m^M
2017-12-14 21:24:03.520 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mresp.status_code:::  --> (u'http://10.105.166.213/placement/resource_providers/dcf28911-6125-4bc4-87b6-bc71b913d8b2/inventories', 200)^[[00m^M
2017-12-14 21:24:03.520 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mENd of method^[[00m^M
2017-12-14 21:24:03.521 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mresp   ---> <Response [200]>^[[00m^M
2017-12-14 21:24:03.521 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mresp.status_code  -->(200, u'http://10.105.166.213/placement/resource_providers/dcf28911-6125-4bc4-87b6-bc71b913d8b2/inventories') ^[[00m^M
2017-12-14 21:24:03.521 DEBUG nova.scheduler.client.report [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mUpdating our resource provider generation from 31 to 32^[[00m ^[[00;33mfrom (pid=7061) _get_inventory_and_update_provider_generation /opt/stack/nova/nova/scheduler/client/report.py:449^[[00m^M
2017-12-14 21:24:03.521 DEBUG oslo_concurrency.lockutils [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mLock "compute_resources" released by "nova.compute.resource_tracker.abort_instance_claim" :: held 0.567s^[[00m ^[[00;33mfrom (pid=7061) inner /usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py:282^[[00m^M
2017-12-14 21:24:03.521 ERROR nova.compute.manager [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] Failed to allocate network(s)^[[00m^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00mTraceback (most recent call last):^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 1938, in _build_and_run_instance^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m    with self._build_resources(context, instance,^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 2756, in spawn^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m    post_xml_callback=gen_confdrive,^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m  File "/opt/stack/nova/nova/virt/libvirt/driver.py", line 5195, in _create_domain_and_network^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m    instance=instance)^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00mVirtualInterfaceCreateException: Virtual Interface creation failed^M
2017-12-14 21:24:03.521 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m^M
2017-12-14 21:24:03.522 DEBUG nova.compute.utils [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] Virtual Interface creation failed^[[00m ^[[00;33mfrom (pid=7061) notify_about_instance_usage /opt/stack/nova/nova/compute/utils.py:317^[[00m^M
2017-12-14 21:24:03.522 DEBUG oslo_messaging._drivers.amqpdriver [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mCAST unique_id: 9887810c2974467badeac9be672b15d2 NOTIFY exchange 'nova' topic 'notifications.error'^[[00m ^[[00;33mfrom (pid=7061) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:546^[[00m^M
2017-12-14 21:24:03.525 DEBUG oslo_messaging._drivers.amqpdriver [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mCAST unique_id: a59e16bd5ee3492c9db22dd99bd067a3 NOTIFY exchange 'nova' topic 'versioned_notifications.error'^[[00m ^[[00;33mfrom (pid=7061) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:546^[[00m^M
2017-12-14 21:24:03.526 ERROR nova.compute.manager [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] Build of instance 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd aborted: Failed to allocate the network(s), not rescheduling.^[[00m^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00mTraceback (most recent call last):^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 1788, in _do_build_and_run_instance^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m    return build_results.FAILED^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 1998, in _build_and_run_instance^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m    msg = _('Failed to allocate the network(s) with error %s, '^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00mBuildAbortException: Build of instance 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd aborted: Failed to allocate the network(s), not rescheduling.^M
2017-12-14 21:24:03.526 TRACE nova.compute.manager ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] ^[[00m^M
2017-12-14 21:24:03.527 DEBUG nova.compute.manager [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] Deallocating network for instance^[[00m ^[[00;33mfrom (pid=7061) _deallocate_network /opt/stack/nova/nova/compute/manager.py:1669^[[00m^M
2017-12-14 21:24:03.527 DEBUG nova.network.neutronv2.api [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35m[instance: 2398fce6-6f98-406d-9b1a-8cf9a1fbfccd] deallocate_for_instance()^[[00m ^[[00;33mfrom (pid=7061) deallocate_for_instance /opt/stack/nova/nova/network/neutronv2/api.py:1173^[[00m^M
2017-12-14 21:24:03.527 DEBUG keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mauth_headers ----> {'X-Auth-Token': u'gAAAAABaM1TN1hhnQrnfQVASXfFXzyOOq1D8zHS9e1v54TjmkZQiXOSKBcS6kYWTVpDeRKlMHh2GZgOFQ9wJ_gSqvOQ371bTfCk61XV7PR5YeOF9pUP6VQI6x0EojazONSNJJxydImbv1Qf6FISU4HP527itUt_3cpnRnn8-1XE9M6rq_47tIN7fTwwWtkEj6C2x6Djki3Ng'}^[[00m ^[[00;33mfrom (pid=7061) request /usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py:492^[[00m^M
2017-12-14 21:24:03.528 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mquery_params  --> {}^[[00m^M
2017-12-14 21:24:03.528 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35murl -----::-> http://10.105.166.213:9696/v2.0/ports.json?device_id=2398fce6-6f98-406d-9b1a-8cf9a1fbfccd^[[00m^M<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F10.105.166.213%3A9696%2Fv2.0%2Fports.json%3Fdevice_id%3D2398fce6-6f98-406d-9b1a-8cf9a1fbfccd%255e%255b%255b00m%255eM&data=02%7C01%7Ctrinath.somanchi%40nxp.com%7C4cb98e3055544767b83a08d543b021c7%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636489346125753805&sdata=5UJFiEfEtS3pHl3qotjriw2rLPhqTVaQxsswIasW5Jw%3D&reserved=0>
2017-12-14 21:24:03.528 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mBefore connection_params^[[00m^M
2017-12-14 21:24:03.528 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mAfter connection_params --> {}^[[00m^M
2017-12-14 21:24:03.528 WARNING keystoneauth1.session [^[[01;36mreq-2e8bd076-f1a4-45ff-a7e7-f54b8bee9c96 ^[[00;36madmin admin] ^[[01;35mBefore send(**kwargs) ^[[00m^M


From: Bernd Bausch [mailto:berndbausch at gmail.com]
Sent: Friday, December 15, 2017 6:21 AM
To: Ramu, MohanX <mohanx.ramu at intel.com<mailto:mohanx.ramu at intel.com>>; openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>
Cc: 'Jay Pipes' <jaypipes at gmail.com<mailto:jaypipes at gmail.com>>
Subject: RE: Devstack - : Failed to allocate the network(s), not rescheduling

This just says that Nova was unable to set up the network plumbing on the compute node.

Depending on the OpenStack version you deployed, you should have other messages in the same log file, for example if you are using Pike, "No more network or fixed IP to be allocated" or "Failed to allocate network(s)" (without the article). It would be helpful to know if there are such messages.

Most probably, there is more information in the Neutron logs. Since you use DevStack, you probably have an all-in-one deployment, but in case you have separate compute nodes, also check the Neutron agent log there.

You may also want to provide more information such as: What OpenStack version you have, which log are you sharing, when does this happen, does it always happen or only under certain circumstances, have you ever been able to launch an instance, what networks do you have, what network do you connect the instance to, what's your local.conf etc.

From: Ramu, MohanX [mailto:mohanx.ramu at intel.com]
Sent: Thursday, December 14, 2017 10:54 PM
To: 'openstack at lists.openstack.org' <openstack at lists.openstack.org<mailto:openstack at lists.openstack.org>>
Cc: 'Bernd Bausch' <berndbausch at gmail.com<mailto:berndbausch at gmail.com>>; Jay Pipes <jaypipes at gmail.com<mailto:jaypipes at gmail.com>>
Subject: Devstack - : Failed to allocate the network(s), not rescheduling


Hi All,

2017-12-14 02:56:06.230 DEBUG oslo_messaging._drivers.amqpdriver [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mCAST unique_id: fa9e604328a445f39702e0089de0248a NOTIFY exchange 'nova' topic 'notifications.error'^[[00m ^[[00;33mfrom (pid=7061) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:546^[[00m^M
2017-12-14 02:56:06.236 DEBUG oslo_messaging._drivers.amqpdriver [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mCAST unique_id: 999c7e8ac35f4b9db809312ddccc8b7f NOTIFY exchange 'nova' topic 'versioned_notifications.error'^[[00m ^[[00;33mfrom (pid=7061) _send /usr/local/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py:546^[[00m^M
2017-12-14 02:56:06.238 ERROR nova.compute.manager [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] Build of instance 8ed6476c-8c0c-4baf-b24f-8530feb836e7 aborted: Failed to allocate the network(s), not rescheduling.^[[00m^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00mTraceback (most recent call last):^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 1788, in _do_build_and_run_instance^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00m    return build_results.FAILED^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00m  File "/opt/stack/nova/nova/compute/manager.py", line 1998, in _build_and_run_instance^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00m    msg = _('Failed to allocate the network(s) with error %s, '^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00mBuildAbortException: Build of instance 8ed6476c-8c0c-4baf-b24f-8530feb836e7 aborted: Failed to allocate the network(s), not rescheduling.^M
2017-12-14 02:56:06.238 TRACE nova.compute.manager ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] ^[[00m^M
2017-12-14 02:56:06.239 DEBUG nova.compute.manager [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] Deallocating network for instance^[[00m ^[[00;33mfrom (pid=7061) _deallocate_network /opt/stack/nova/nova/compute/manager.py:1669^[[00m^M
2017-12-14 02:56:06.239 DEBUG nova.network.neutronv2.api [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35m[instance: 8ed6476c-8c0c-4baf-b24f-8530feb836e7] deallocate_for_instance()^[[00m ^[[00;33mfrom (pid=7061) deallocate_for_instance /opt/stack/nova/nova/network/neutronv2/api.py:1173^[[00m^M
2017-12-14 02:56:06.240 DEBUG keystoneauth1.session [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mauth_headers ----> {'X-Auth-Token': u'gAAAAABaMlbpaJ9OocE4phmMPWC6eSmajqtx1id547QxiGEhxMlSddZRexD8TPPH4Il0n3mAemxAWZVXXJeMDAb47ZIlf4YNogzUitsDgkR3-VRSNbB0MdpBFv3aO_aUv5ZBOe1__Ya_bJtW33-QglQbeBqkUoUaQli8aMldZ_BhLBOsqFCYSIWXnd15z73p1j9hmcbxU3Mb'}^[[00m ^[[00;33mfrom (pid=7061) request /usr/local/lib/python2.7/dist-packages/keystoneauth1/session.py:492^[[00m^M
2017-12-14 02:56:06.240 WARNING keystoneauth1.session [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mquery_params  --> {}^[[00m^M
2017-12-14 02:56:06.240 WARNING keystoneauth1.session [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35murl -----::-> http://10.105.166.213:9696/v2.0/ports.json?device_id=8ed6476c-8c0c-4baf-b24f-8530feb836e7^[[00m^M<https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2F10.105.166.213%3A9696%2Fv2.0%2Fports.json%3Fdevice_id%3D8ed6476c-8c0c-4baf-b24f-8530feb836e7%255e%255b%255b00m%255eM&data=02%7C01%7Ctrinath.somanchi%40nxp.com%7C4cb98e3055544767b83a08d543b021c7%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C636489346125753805&sdata=Ab4DFk8Tfv2p2vN8p0Cm3gaJtydUCNrt8w08Y20HZZQ%3D&reserved=0>
2017-12-14 02:56:06.241 WARNING keystoneauth1.session [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mBefore connection_params^[[00m^M
2017-12-14 02:56:06.241 WARNING keystoneauth1.session [^[[01;36mreq-0949894a-1c03-4956-8582-49637a29e3b0 ^[[00;36madmin admin] ^[[01;35mAfter connection_params --> {}^[[00m^M


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20171215/ae0e29a3/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 30036 bytes
Desc: image001.jpg
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20171215/ae0e29a3/attachment.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 63974 bytes
Desc: image002.jpg
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20171215/ae0e29a3/attachment-0001.jpg>

More information about the Openstack mailing list