[Openstack-operators] RE : Can't launch docker instance, Unexpected vif_type=binding_failed.
Kevin Benton
blak111 at gmail.com
Tue Jul 28 21:54:04 UTC 2015
We also need to see the /etc/neutron/plugins/ml2/ml2_conf.ini from both
because that has the physical network mappings.
Also, I don't see anything obvious in the agent logs there. Can you include
the portion surrounding the "Failed to bind port" message from the Neutron
server log?
On Tue, Jul 28, 2015 at 4:13 AM, Asmaa Chebba <achebba at cdta.dz> wrote:
> openvswitch agent is running and the logs in compute2 are as follow:
>
> 1. OVS-cleanup.log
> 2015-06-20 12:52:19.976 1529 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-06-23 15:48:43.401 1332 INFO neutron.common.config [-] Logging
> enabled!
> 2015-06-23 15:48:43.893 1332 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-06-23 15:48:44.520 1332 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-06-24 11:49:21.423 1770 INFO neutron.common.config [-] Logging
> enabled!
> 2015-06-24 11:49:22.123 1770 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-06-24 11:49:22.628 1770 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-06-25 00:21:55.634 1337 INFO neutron.common.config [-] Logging
> enabled!
> 2015-06-25 00:21:56.858 1337 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-06-25 00:21:57.900 1337 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-07-07 16:43:42.608 1457 INFO neutron.common.config [-] Logging
> enabled!
> 2015-07-07 16:43:43.399 1457 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-07-07 16:43:43.792 1457 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-07-08 15:04:31.954 1351 INFO neutron.common.config [-] Logging
> enabled!
> 2015-07-08 15:04:32.888 1351 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-07-08 15:04:33.235 1351 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed successfully
> 2015-07-20 13:25:20.300 1550 INFO neutron.common.config [-] Logging
> enabled!
> 2015-07-20 13:25:22.665 1550 INFO neutron.agent.ovs_cleanup_util [-]
> Cleaning br-int
> 2015-07-20 13:25:22.770 1550 INFO neutron.agent.ovs_cleanup_util [-] OVS
> cleanup completed succe
>
> 2. Openvswitch-agent.log
> 2015-07-28 13:23:29.151 4615 ERROR neutron.agent.linux.ovsdb_monitor [-]
> Error received from ovsdb monitor:
> 2015-07-28T11:23:29Z|00001|fatal_signal|WARN|terminating with signal 15
> (Terminated)
> 2015-07-28 13:23:29.190 4615 ERROR neutron.agent.linux.utils [-]
> Command: ['ps', '--ppid', '4764', '-o', 'pid=']
> Exit code: 1
> Stdout: ''
> Stderr: ''
> 2015-07-28 13:23:29.835 4615 CRITICAL neutron
> [req-dbf6bc78-c2df-4454-9e19-5f09bf688ee9 None] AssertionError: Trying to
> re-send() an already-triggered event.
> 2015-07-28 13:23:29.835 4615 TRACE neutron Traceback (most recent call
> last):
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/bin/neutron-openvswitch-agent", line 10, in <module>
> 2015-07-28 13:23:29.835 4615 TRACE neutron sys.exit(main())
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
> line 1565, in main
> 2015-07-28 13:23:29.835 4615 TRACE neutron agent.daemon_loop()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/plugins/openvswitch/agent/ovs_neutron_agent.py",
> line 1485, in daemon_loop
> 2015-07-28 13:23:29.835 4615 TRACE neutron
> self.rpc_loop(polling_manager=pm)
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
> 2015-07-28 13:23:29.835 4615 TRACE neutron self.gen.next()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/agent/linux/polling.py", line 39,
> in get_polling_manager
> 2015-07-28 13:23:29.835 4615 TRACE neutron pm.stop()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/agent/linux/polling.py", line
> 106, in stop
> 2015-07-28 13:23:29.835 4615 TRACE neutron self._monitor.stop()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/agent/linux/async_process.py",
> line 89, in stop
> 2015-07-28 13:23:29.835 4615 TRACE neutron self._kill()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/agent/linux/ovsdb_monitor.py",
> line 99, in _kill
> 2015-07-28 13:23:29.835 4615 TRACE neutron
> super(SimpleInterfaceMonitor, self)._kill(*args, **kwargs)
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/neutron/agent/linux/async_process.py",
> line 116, in _kill
> 2015-07-28 13:23:29.835 4615 TRACE neutron self._kill_event.send()
> 2015-07-28 13:23:29.835 4615 TRACE neutron File
> "/usr/lib/python2.7/dist-packages/eventlet/event.py", line 150, in send
> 2015-07-28 13:23:29.835 4615 TRACE neutron assert self._result is
> NOT_USED, 'Trying to re-send() an already-triggered event.'
> 2015-07-28 13:23:29.835 4615 TRACE neutron AssertionError: Trying to
> re-send() an already-triggered event.
> 2015-07-28 13:23:29.835 4615 TRACE neutron
> 2015-07-28 13:23:32.197 6195 INFO neutron.common.config [-] Logging
> enabled!
> 2015-07-28 13:23:33.005 6195 INFO oslo.messaging._drivers.impl_rabbit [-]
> Connecting to AMQP server on controller:5672
> 2015-07-28 13:23:33.120 6195 INFO oslo.messaging._drivers.impl_rabbit [-]
> Connected to AMQP server on controller:5672
> 2015-07-28 13:23:33.126 6195 INFO oslo.messaging._drivers.impl_rabbit [-]
> Connecting to AMQP server on controller:5672
> 2015-07-28 13:23:33.143 6195 INFO oslo.messaging._drivers.impl_rabbit [-]
> Connected to AMQP server on controller:5672
> 2015-07-28 13:23:34.287 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connecting to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.304 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connected to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.315 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connecting to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.321 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connected to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.327 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connecting to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.333 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connected to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.338 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connecting to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.344 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connected to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.349 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connecting to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.356 6195 INFO oslo.messaging._drivers.impl_rabbit
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 ] Connected to AMQP server on
> controller:5672
> 2015-07-28 13:23:34.360 6195 INFO
> neutron.plugins.openvswitch.agent.ovs_neutron_agent
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 None] Agent initialized
> successfully, now running...
> 2015-07-28 13:23:34.367 6195 INFO
> neutron.plugins.openvswitch.agent.ovs_neutron_agent
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 None] Agent out of sync with
> plugin!
> 2015-07-28 13:23:34.447 6195 INFO
> neutron.plugins.openvswitch.agent.ovs_neutron_agent
> [req-15e7e089-2323-4101-811e-ff2537fe3c27 None] Agent tunnel out of sync
> with plugin!
>
>
> 3. Openvswitch-agent.log.1 ==>
> 2015-07-26 12:15:51.431 4615 INFO neutron.agent.securitygroups_rpc
> [req-8d59b5d6-4b0f-4ced-a336-5a94159ac601 None] Security group member
> updated [u'b6692f4f-f96e-48eb-a796-8df11fb6c16b']
>
>
> - And the result of ovs-vsctl show:
>
> 0a578213-5de5-48fd-923e-14b7b201c6c8
> Bridge br-int
> fail_mode: secure
> Port br-int
> Interface br-int
> type: internal
> Port patch-tun
> Interface patch-tun
> type: patch
> options: {peer=patch-int}
> Bridge br-tun
> fail_mode: secure
> Port "gre-c0a80204"
> Interface "gre-c0a80204"
> type: gre
> options: {df_default="true", in_key=flow,
> local_ip="192.168.2.5", out_key=flow, remote_ip="192.168.2.4"}
> Port br-tun
> Interface br-tun
> type: internal
> Port patch-int
> Interface patch-int
> type: patch
> options: {peer=patch-tun}
> Port "gre-c0a80203"
> Interface "gre-c0a80203"
> type: gre
> options: {df_default="true", in_key=flow,
> local_ip="192.168.2.5", out_key=flow, remote_ip="192.168.2.3"}
> Port "gre-c0a80202"
> Interface "gre-c0a80202"
> type: gre
> options: {df_default="true", in_key=flow,
> local_ip="192.168.2.5", out_key=flow, remote_ip="192.168.2.2"}
> ovs_version: "2.0.2"
>
> As for the configuration, here is the files of neutron.conf:
>
> 1. ON compute
> [DEFAULT]
> verbose = True
> lock_path = $state_path/lock
> core_plugin = ml2
> service_plugins = router
> auth_strategy = keystone
> allow_overlapping_ips = True
> rabbit_host=controller
>
> [agent]
> root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
>
> [keystone_authtoken]
> auth_uri = http://controller:5000/v2.0
> identity_uri = http://controller:35357
> admin_tenant_name = service
> admin_user = neutron
> admin_password = PASSWORD
>
> [service_providers]
>
> service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default
>
> service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
>
> 2. ON controller
> [DEFAULT]
> verbose = True
> lock_path = $state_path/lock
> core_plugin = ml2
> service_plugins = router
> auth_strategy = keystone
> allow_overlapping_ips = True
> agent_down_time = 1000
> notify_nova_on_port_status_changes = True
> notify_nova_on_port_data_changes = True
> nova_url = http://controller:8774/v2
> nova_region_name = regionOne
> nova_admin_username = nova
> nova_admin_tenant_id =f41a56a0ca3d4ef982fadb9b2363c09f
> nova_admin_password = PASSWORD
> nova_admin_auth_url = http://controller:35357/v2.0
> rabbit_host=controller
> rabbit_password=PASSWORD
> rpc_backend=rabbit
>
> [agent]
> root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
> report_interval = 100
>
> [keystone_authtoken]
> auth_uri = http://controller:5000/v2.0
> identity_uri = http://controller:35357
> admin_tenant_name = service
> admin_user = neutron
> admin_password = PASSWORD
>
> [database]
> connection = mysql://neutron:PASSWORD@controller/neutron
>
> [service_providers]
>
> service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default
>
> service_provider=VPN:openswan:neutron.services.vpn.service_drivers.ipsec.IPsecVPNDriver:default
>
> Thanks.
> ________________________________________
> De : Assaf Muller [amuller at redhat.com]
> Date d'envoi : lundi 27 juillet 2015 15:13
> À : Kevin Benton
> Cc: Asmaa Chebba; OpenStack Operators
> Objet : Re: [Openstack-operators] Can't launch docker instance, Unexpected
> vif_type=binding_failed.
>
> Also can you paste the configuration for both the OVS agent and your
> neutron server? Binding failure is almost always a configuration mismatch.
>
> ----- Original Message -----
> >
> >
> > Is the neutron openvswitch agent running on host compute2? What do the
> logs
> > say for the agent there?
> > On Jul 22, 2015 07:22, "Asmaa Chebba" < achebba at cdta.dz > wrote:
> >
> >
> >
> > Hi,
> > I installed Docker with juno release on Ubuntu
> > all compute/networking services are up and enabled, and I can add docker
> > images with glance however, I can't launch an instance (stopped at
> spawning
> > step)
> > in the nova-compute log, I found :
> > Instance failed to spawn
> > InstanceDeployFailure: Cannot setup network: Unexpected
> > vif_type=binding_failed
> > and when verifying the neutron-server log:
> > Failed to bind port 5d299cc9-e3f3-48a0-a80f-f204910a47e7 on host compute2
> >
> > Any idea on how to solve this?
> > I appriciate your help.
> > Tahnks.
> >
> >
> >
> >
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
> >
> > _______________________________________________
> > OpenStack-operators mailing list
> > OpenStack-operators at lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators
> >
>
--
Kevin Benton
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-operators/attachments/20150728/4c46570a/attachment.html>
More information about the OpenStack-operators
mailing list