[Openstack] [OpenStack]Failed to configure neutron with gre
Jason Bishop
jason.bishop at gmail.com
Mon May 19 05:51:05 UTC 2014
When I got this error last I had a typo in my neutron.conf file on the
compute node. this was causing the br-tun not to be created. I fixed my
typo and then the br-tun showed up like normal and nova-compute ran fine.
I am sure there are many other scenarios. turn on debugging for all of
the neutron services on the compute node and check that they are all
running fine.
good luck
Bridge br-tun
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Port br-tun
Interface br-tun
type: internal
Port "gre-c0a80a0a"
Interface "gre-c0a80a0a"
type: gre
options: {in_key=flow, local_ip="192.168.10.20",
out_key=flow, remote_ip="192.168.10.10"}
Port "gre-c0a80a15"
Interface "gre-c0a80a15"
type: gre
options: {in_key=flow, local_ip="192.168.10.20",
out_key=flow, remote_ip="192.168.10.21"}
ovs_version: "2.0.1"
On Tue, Apr 29, 2014 at 9:16 PM, Super Wang <wangxuepo at gmail.com> wrote:
> Hi folk,
>
> I'm working on setup a 5 nodes environment of openstack icehouse on ubuntu
> 14.04 by following this guide:
> http://docs.openstack.org/trunk/install-guide/install/apt/content/index.html
>
> All my nodes are deployed on VMWare workstation 10.0.1, here's the network
> topology:
>
> Controller:
> eth0 - static NAT, for package installation only
> eth1 - host only - management network - 10.1.0.10
>
> Network01:
> eth0 - static NAT - for package installation only - 192.168.1.20
> eth1 - host only - management network - 10.1.0.20
> eth2 - host only - private network - 10.2.0.20
> eth3 - static NAT - external network, same as eth0, but ip range is from
> 192.168.1.101 to 192.168.1.120
>
> Compute01:
> eth0 - static NAT - for package installation only - 192.168.1.31
> eth1 - host only - management network - 10.1.0.31
> eth2 - host only - private network - 10.2.0.31
>
> Compute03:
> eth0 - static NAT - for package installation only - 192.168.1.32
> eth1 - host only - management network - 10.1.0.32
> eth2 - host only - private network - 10.2.0.32
>
> Storage01:
> eth0 - static NAT - for package installation only - 192.168.1.41
> eth1 - host only - management network - 10.1.0.41
> eth2 - host only - private network - 10.2.0.41
>
> And I failed to deploy new instance with this error: Unexpected
> vif_type=binding_failed, I'm still very new to openstack, tried google, not
> much helpful, it will be great if you can help me out of this, thanks for
> you time.
>
> Here are the configuraion of neutron on network node:
>
> neutron.conf
>
> [DEFAULT]
> verbose = True
> state_path = /var/lib/neutron
> lock_path = $state_path/lock
> core_plugin = ml2
> service_plugins = router
> auth_strategy = keystone
> allow_overlapping_ips = True
> rpc_backend = neutron.openstack.common.rpc.impl_kombu
> rabbit_host = openstack-controller
> rabbit_password = password
> rabbit_port = 5672
> rabbit_hosts = $rabbit_host:$rabbit_port
> rabbit_userid = guest
> notification_driver = neutron.openstack.common.notifier.rpc_notifier
> [quotas]
> [agent]
> root_helper = sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf
> [keystone_authtoken]
> auth_uri = http://openstack-controller:5000
> auth_host = openstack-controller
> auth_port = 35357
> auth_protocol = http
> admin_tenant_name = service
> admin_user = neutron
> admin_password = neutron
> [database]
> connection = mysql://neutron:password@openstack-controller/neutron
> [service_providers]
>
> l3_agent.ini
>
> [DEFAULT]
> verbose = True
> interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
> use_namespaces = True
>
> dhcp_agent.ini
>
> [DEFAULT]
> verbose = True
> interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
> dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
> use_namespaces = True
>
> metadata_agent.ini
>
> [DEFAULT]
> verbose = True
> auth_url = http://openstack-controller:5000/v2.0
> auth_region = RegionOne
> admin_tenant_name = service
> admin_user = neutron
> admin_password = neutron
> nova_metadata_ip = openstack-controller
> metadata_proxy_shared_secret = secret
>
> ml2_config.ini
>
> [ml2]
> type_drivers = gre
> tenant_network_types = gre
> mechanism_drivers = openvswitch
> [ml2_type_flat]
> [ml2_type_vlan]
> [ml2_type_gre]
> tunnel_id_ranges = 1:1000
> [ml2_type_vxlan]
> [securitygroup]
> firewall_driver =
> neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
> enable_security_group = True
> [ovs]
> local_ip = 10.2.0.20
> tunnel_type = gre
> enable_tunneling = True
>
> Sorry for the format bellow, it works very well on notepad but not browser
>
> cloud at openstack-controller:~$ neutron net-list
>
> +--------------------------------------+----------+-----------------------------------------------------+
> | id | name | subnets
> |
>
> +--------------------------------------+----------+-----------------------------------------------------+
> | 27b53894-1b1b-474c-9543-c1995798fd0c | ext-net |
> e13a05d5-511d-4e39-bc6a-298f8f718f06 192.168.1.0/24 |
> | 42d0b68f-0f28-4786-8316-0c1d8beeed95 | demo-net |
> eb70275d-e0dc-4296-ac46-5073c1170bf8 172.168.2.0/24 |
>
> +--------------------------------------+----------+-----------------------------------------------------+
> cloud at openstack-controller:~$ neutron net-show ext-net
> +---------------------------+--------------------------------------+
> | Field | Value |
> +---------------------------+--------------------------------------+
> | admin_state_up | True |
> | id | 27b53894-1b1b-474c-9543-c1995798fd0c |
> | name | ext-net |
> | provider:network_type | gre |
> | provider:physical_network | |
> | provider:segmentation_id | 1 |
> | router:external | True |
> | shared | True |
> | status | ACTIVE |
> | subnets | e13a05d5-511d-4e39-bc6a-298f8f718f06 |
> | tenant_id | 42fc8cffaa88402796daa222765f9f69 |
> +---------------------------+--------------------------------------+
> cloud at openstack-controller:~$ neutron net-show demo-net
> +---------------------------+--------------------------------------+
> | Field | Value |
> +---------------------------+--------------------------------------+
> | admin_state_up | True |
> | id | 42d0b68f-0f28-4786-8316-0c1d8beeed95 |
> | name | demo-net |
> | provider:network_type | gre |
> | provider:physical_network | |
> | provider:segmentation_id | 2 |
> | router:external | False |
> | shared | False |
> | status | ACTIVE |
> | subnets | eb70275d-e0dc-4296-ac46-5073c1170bf8 |
> | tenant_id | b96454f1425b4a159da8daad18a438de |
> +---------------------------+--------------------------------------+
> cloud at openstack-controller:~$ neutron subnet-list
>
> +--------------------------------------+-------------+----------------+----------------------------------------------------+
> | id | name | cidr |
> allocation_pools |
>
> +--------------------------------------+-------------+----------------+----------------------------------------------------+
> | e13a05d5-511d-4e39-bc6a-298f8f718f06 | ext-subnet | 192.168.1.0/24 |
> {"start": "192.168.1.101", "end": "192.168.1.120"} |
> | eb70275d-e0dc-4296-ac46-5073c1170bf8 | demo-subnet | 172.168.2.0/24 |
> {"start": "172.168.2.100", "end": "172.168.2.199"} |
>
> +--------------------------------------+-------------+----------------+----------------------------------------------------+
> cloud at openstack-controller:~$ neutron subnet-show ext-subnet
> +------------------+----------------------------------------------------+
> | Field | Value |
> +------------------+----------------------------------------------------+
> | allocation_pools | {"start": "192.168.1.101", "end": "192.168.1.120"} |
> | cidr | 192.168.1.0/24 |
> | dns_nameservers | |
> | enable_dhcp | False |
> | gateway_ip | 192.168.1.2 |
> | host_routes | |
> | id | e13a05d5-511d-4e39-bc6a-298f8f718f06 |
> | ip_version | 4 |
> | name | ext-subnet |
> | network_id | 27b53894-1b1b-474c-9543-c1995798fd0c |
> | tenant_id | 42fc8cffaa88402796daa222765f9f69 |
> +------------------+----------------------------------------------------+
> cloud at openstack-controller:~$ neutron subnet-show demo-subnet
> +------------------+----------------------------------------------------+
> | Field | Value |
> +------------------+----------------------------------------------------+
> | allocation_pools | {"start": "172.168.2.100", "end": "172.168.2.199"} |
> | cidr | 172.168.2.0/24 |
> | dns_nameservers | |
> | enable_dhcp | True |
> | gateway_ip | 172.168.2.1 |
> | host_routes | |
> | id | eb70275d-e0dc-4296-ac46-5073c1170bf8 |
> | ip_version | 4 |
> | name | demo-subnet |
> | network_id | 42d0b68f-0f28-4786-8316-0c1d8beeed95 |
> | tenant_id | b96454f1425b4a159da8daad18a438de |
> +------------------+----------------------------------------------------+
> cloud at openstack-controller:~$ neutron router-list
>
> +--------------------------------------+-------------+-----------------------------------------------------------------------------+
> | id | name |
> external_gateway_info
> |
>
> +--------------------------------------+-------------+-----------------------------------------------------------------------------+
> | 6725cec0-4d2b-47eb-9e99-3c606883df1e | demo-router | {"network_id":
> "27b53894-1b1b-474c-9543-c1995798fd0c", "enable_snat": true} |
>
> +--------------------------------------+-------------+-----------------------------------------------------------------------------+
> cloud at openstack-controller:~$ neutron router-show demo-router
>
> +-----------------------+-----------------------------------------------------------------------------+
> | Field | Value
> |
>
> +-----------------------+-----------------------------------------------------------------------------+
> | admin_state_up | True
> |
> | external_gateway_info | {"network_id":
> "27b53894-1b1b-474c-9543-c1995798fd0c", "enable_snat": true} |
> | id | 6725cec0-4d2b-47eb-9e99-3c606883df1e
> |
> | name | demo-router
> |
> | routes |
> |
> | status | ACTIVE
> |
> | tenant_id | b96454f1425b4a159da8daad18a438de
> |
>
> +-----------------------+-----------------------------------------------------------------------------+
> cloud at openstack-controller:~$ neutron router-port-list demo-router
>
> +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
> | id | name | mac_address |
> fixed_ips
> |
>
> +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
> | d1d30c05-4bc2-48d8-b53f-ccbbe6b5743d | | fa:16:3e:a7:36:69 |
> {"subnet_id": "eb70275d-e0dc-4296-ac46-5073c1170bf8", "ip_address":
> "172.168.2.1"} |
> | f21256b2-7a23-448d-9664-724f488ea026 | | fa:16:3e:cd:1f:9b |
> {"subnet_id": "e13a05d5-511d-4e39-bc6a-298f8f718f06", "ip_address":
> "192.168.1.103"} |
>
> +--------------------------------------+------+-------------------+--------------------------------------------------------------------------------------+
> cloud at openstack-controller:~$ neutron agent-list
>
> +--------------------------------------+--------------------+---------------------+-------+----------------+
> | id | agent_type | host
> | alive | admin_state_up |
>
> +--------------------------------------+--------------------+---------------------+-------+----------------+
> | 3f89235f-8dfd-4583-b08a-970a066fdb83 | L3 agent |
> openstack-network | :-) | True |
> | 537c3551-5bb1-4018-9480-807599fdeafc | Open vSwitch agent |
> openstack-compute02 | :-) | True |
> | 54455397-6cef-439d-b7e7-b243a1ffc30b | Open vSwitch agent |
> openstack-network | :-) | True |
> | 55645859-8d8e-48df-909b-94c7cf2e20de | Open vSwitch agent |
> openstack-compute01 | :-) | True |
> | a7724dca-1c66-4893-af41-25d350209da5 | Metadata agent |
> openstack-network | :-) | True |
> | aef4d469-fed0-4457-b2fd-aea9e9699e0d | DHCP agent |
> openstack-network | :-) | True |
>
> +--------------------------------------+--------------------+---------------------+-------+----------------+
> cloud at openstack-network:~$ sudo ovs-vsctl show
> 0217dac2-b129-4a14-b019-ce3415364a9f
> Bridge br-int
> Port "tapbf4260b0-c8"
> tag: 4095
> Interface "tapbf4260b0-c8"
> type: internal
> Port patch-tun
> Interface patch-tun
> type: patch
> options: {peer=patch-int}
> Port br-int
> Interface br-int
> type: internal
> Port "qr-d1d30c05-4b"
> tag: 4095
> Interface "qr-d1d30c05-4b"
> type: internal
> Bridge br-tun
> Port patch-int
> Interface patch-int
> type: patch
> options: {peer=patch-tun}
> Port br-tun
> Interface br-tun
> type: internal
> Port "gre-0a020020"
> Interface "gre-0a020020"
> type: gre
> options: {in_key=flow, local_ip="10.2.0.20", out_key=flow,
> remote_ip="10.2.0.32"}
> Port "gre-0a02001f"
> Interface "gre-0a02001f"
> type: gre
> options: {in_key=flow, local_ip="10.2.0.20", out_key=flow,
> remote_ip="10.2.0.31"}
> Bridge br-ex
> Port br-ex
> Interface br-ex
> type: internal
> Port "eth3"
> Interface "eth3"
> Port "qg-f21256b2-7a"
> Interface "qg-f21256b2-7a"
> type: internal
> ovs_version: "2.0.1"
> cloud at openstack-network:~$ sudo cat /etc/network/interfaces
> # This file describes the network interfaces available on your system
> # and how to activate them. For more information, see interfaces(5).
> # The loopback network interface
> auto lo
> iface lo inet loopback
> # The primary network interface
> auto eth0
> iface eth0 inet static
> address 192.168.1.20
> netmask 255.255.255.0
> gateway 192.168.1.2
> dns-nameservers 192.168.1.2
> auto eth1
> iface eth1 inet static
> address 10.1.0.20
> netmask 255.255.0.0
> auto eth2
> iface eth2 inet static
> address 10.2.0.20
> netmask 255.255.0.0
> auto eth3
> iface eth3 inet manual
> up ip link set dev eth3 up
> down ip link set dev eth3 down
>
>
> I'm not quit sure about if I can use the same NAT network on eth3(as
> external network) with eth0, when I tried to ping the gateway of
> demote-subnet 172.168.2.1 it says:
>
> cloud at openstack-network:~$ ping 172.168.2.1
> PING 172.168.2.1 (172.168.2.1) 56(84) bytes of data.
> From 192.168.1.20 icmp_seq=10 Destination Host Unreachable
> From 192.168.1.20 icmp_seq=11 Destination Host Unreachable
> From 192.168.1.20 icmp_seq=12 Destination Host Unreachable
> From 192.168.1.20 icmp_seq=13 Destination Host Unreachable
> From 192.168.1.20 icmp_seq=14 Destination Host Unreachable
>
> But I can ping 192.168.1.20 successfully
>
> cloud at openstack-network:~$ ping 192.168.1.20
> PING 192.168.1.20 (192.168.1.20) 56(84) bytes of data.
> 64 bytes from 192.168.1.20: icmp_seq=1 ttl=64 time=0.127 ms
> 64 bytes from 192.168.1.20: icmp_seq=2 ttl=64 time=0.202 ms
> 64 bytes from 192.168.1.20: icmp_seq=4 ttl=64 time=0.098 ms
>
>
>
> Bellow is the detailed error from /var/log/nova/nova-compute.log on
> openstack-compute02
>
> 2014-04-29 17:42:04.482 3462 INFO nova.compute.resource_tracker [-]
> Compute_service record updated for openstack-compute02:openstack-compute02
> 2014-04-29 17:43:04.524 3462 AUDIT nova.compute.resource_tracker [-]
> Auditing locally available compute resources
> 2014-04-29 17:43:04.666 3462 AUDIT nova.compute.resource_tracker [-] Free
> ram (MB): 474
> 2014-04-29 17:43:04.667 3462 AUDIT nova.compute.resource_tracker [-] Free
> disk (GB): 6
> 2014-04-29 17:43:04.667 3462 AUDIT nova.compute.resource_tracker [-] Free
> VCPUS: 1
> 2014-04-29 17:43:04.687 3462 INFO nova.compute.resource_tracker [-]
> Compute_service record updated for openstack-compute02:openstack-compute02
> 2014-04-29 17:43:38.085 3462 AUDIT nova.compute.manager
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Starting instance...
> 2014-04-29 17:43:38.210 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Attempting claim: memory 512 MB, disk
> 1 GB, VCPUs 1
> 2014-04-29 17:43:38.210 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Total memory: 986 MB, used: 512.00 MB
> 2014-04-29 17:43:38.211 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] memory limit: 1479.00 MB, free:
> 967.00 MB
> 2014-04-29 17:43:38.211 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Total disk: 6 GB, used: 0.00 GB
> 2014-04-29 17:43:38.212 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] disk limit not specified, defaulting
> to unlimited
> 2014-04-29 17:43:38.213 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Total CPUs: 1 VCPUs, used: 0.00 VCPUs
> 2014-04-29 17:43:38.213 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] CPUs limit not specified, defaulting
> to unlimited
> 2014-04-29 17:43:38.214 3462 AUDIT nova.compute.claims
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Claim successful
> 2014-04-29 17:43:38.986 3462 INFO nova.virt.libvirt.driver
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Creating image
> 2014-04-29 17:43:39.239 3462 WARNING nova.virt.disk.vfs.guestfs
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] Failed to close augeas aug_close: call
> launch before using this function(in guestfish, don't forget to use the
> 'run' command)
> 2014-04-29 17:43:39.547 3462 ERROR nova.compute.manager
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Instance failed to spawn
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Traceback (most recent call last):
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1720, in
> _spawn
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] block_device_info)
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2250,
> in spawn
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] write_to_disk=True)
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3431,
> in to_xml
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] disk_info, rescue,
> block_device_info)
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3247,
> in get_guest_config
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] flavor)
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 384, in
> get_config
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] _("Unexpected vif_type=%s") %
> vif_type)
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] NovaException: Unexpected
> vif_type=binding_failed
> 2014-04-29 17:43:39.547 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1]
> 2014-04-29 17:43:39.587 3462 AUDIT nova.compute.manager
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Terminating instance
> 2014-04-29 17:43:39.881 3462 ERROR nova.virt.libvirt.driver [-] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] During wait destroy, instance
> disappeared.
> 2014-04-29 17:43:40.009 3462 ERROR nova.compute.manager
> [req-346e5f84-3c34-43a2-a445-0710b4106246 6404404b3fb34591bd72be8788bb1c7e
> b96454f1425b4a159da8daad18a438de] [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Error: Unexpected
> vif_type=binding_failed
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] Traceback (most recent call last):
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1311, in
> _build_instance
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] set_access_ip=set_access_ip)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 399, in
> decorated_function
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] return function(self, context,
> *args, **kwargs)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1723, in
> _spawn
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] LOG.exception(_('Instance failed
> to spawn'), instance=instance)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] six.reraise(self.type_,
> self.value, self.tb)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1720, in
> _spawn
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] block_device_info)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 2250,
> in spawn
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] write_to_disk=True)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3431,
> in to_xml
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] disk_info, rescue,
> block_device_info)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3247,
> in get_guest_config
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] flavor)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 384, in
> get_config
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] _("Unexpected vif_type=%s") %
> vif_type)
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1] NovaException: Unexpected
> vif_type=binding_failed
> 2014-04-29 17:43:40.009 3462 TRACE nova.compute.manager [instance:
> f8b24eb2-7b04-43c8-8515-fdf907df59e1]
> 2014-04-29 17:43:40.310 3462 ERROR oslo.messaging.rpc.dispatcher [-]
> Exception during message handling: Unexpected vif_type=binding_failed
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> Traceback (most recent call last):
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 133, in _dispatch_and_reply
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> incoming.message))
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 176, in _dispatch
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return self._do_dispatch(endpoint, method, ctxt, args)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 122, in _do_dispatch
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> result = getattr(endpoint, method)(ctxt, **new_args)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139,
> ininner
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return func(*args, **kwargs)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> payload)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return f(self, context, *args, **kw)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in
> decorated_function
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher pass
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in
> decorated_function
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return function(self, context, *args, **kwargs)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
> decorated_function
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> function(self, context, *args, **kwargs)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in
> decorated_function
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher e,
> sys.exc_info())
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in
> decorated_function
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return function(self, context, *args, **kwargs)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2075, in
> run_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> do_run_instance()
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line
> 249, in inner
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> return f(*args, **kwargs)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2074, in
> do_run_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> legacy_bdm_in_spec)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1207, in
> _run_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> notify("error", fault=e) # notify that build failed
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1191, in
> _run_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> instance, image_meta, legacy_bdm_in_spec)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1355, in
> _build_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> filter_properties, bdms, legacy_bdm_in_spec)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1401, in
> _reschedule_or_error
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> self._log_original_error(exc_info, instance_uuid)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1396, in
> _reschedule_or_error
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher bdms,
> requested_networks)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in
> _shutdown_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> requested_networks)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2115, in
> _shutdown_instance
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> block_device_info)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 953,
> in destroy
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> destroy_disks)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 989,
> in cleanup
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> self.unplug_vifs(instance, network_info)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 860,
> in unplug_vifs
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> self.vif_driver.unplug(instance, vif)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 798, in
> unplug
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> _("Unexpected vif_type=%s") % vif_type)
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> NovaException: Unexpected vif_type=binding_failed
> 2014-04-29 17:43:40.310 3462 TRACE oslo.messaging.rpc.dispatcher
> 2014-04-29 17:43:40.317 3462 ERROR oslo.messaging._drivers.common [-]
> Returning exception Unexpected vif_type=binding_failed to caller
> 2014-04-29 17:43:40.317 3462 ERROR oslo.messaging._drivers.common [-]
> ['Traceback (most recent call last):\n', ' File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 133, in _dispatch_and_reply\n incoming.message))\n', ' File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 176, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt,
> args)\n', ' File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 122, in _do_dispatch\n result = getattr(endpoint, method)(ctxt,
> **new_args)\n', ' File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/server.py", line 139,
> in inner\n return func(*args, **kwargs)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped\n
> payload)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in
> wrapped\n return f(self, context, *args, **kw)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in
> decorated_function\n pass\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266,
> in decorated_function\n return function(self, context, *args,
> **kwargs)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in
> decorated_function\n function(self, context, *args, **kwargs)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309,
> in decorated_function\n e, sys.exc_info())\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296,
> in decorated_function\n return function(self, context, *args,
> **kwargs)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2075, in
> run_instance\n do_run_instance()\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/lockutils.py", line
> 249, in inner\n return f(*args, **kwargs)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2074, in
> do_run_instance\n legacy_bdm_in_spec)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1207, in
> _run_instance\n notify("error", fault=e) # notify that build failed\n',
> ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 1191, in _run_instance\n instance, image_meta, legacy_bdm_in_spec)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 1355, in _build_instance\n filter_properties, bdms,
> legacy_bdm_in_spec)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 1401, in
> _reschedule_or_error\n self._log_original_error(exc_info,
> instance_uuid)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 1396, in _reschedule_or_error\n bdms, requested_networks)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2125, in
> _shutdown_instance\n requested_networks)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 68, in __exit__\n six.reraise(self.type_, self.value, self.tb)\n', '
> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line
> 2115, in _shutdown_instance\n block_device_info)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 953,
> in destroy\n destroy_disks)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 989,
> in cleanup\n self.unplug_vifs(instance,network_info)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 860,
> in unplug_vifs\n self.vif_driver.unplug(instance, vif)\n', ' File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/vif.py", line 798, in
> unplug\n _("Unexpected vif_type=%s") % vif_type)\n','NovaException:
> Unexpected vif_type=binding_failed\n']
> 2014-04-29 17:44:04.737 3462 AUDIT nova.compute.resource_tracker [-]
> Auditing locally available compute resources
> 2014-04-29 17:44:04.874 3462 AUDIT nova.compute.resource_tracker [-] Free
> ram (MB): -38
> 2014-04-29 17:44:04.874 3462 AUDIT nova.compute.resource_tracker [-] Free
> disk (GB): 5
> 2014-04-29 17:44:04.875 3462 AUDIT nova.compute.resource_tracker [-] Free
> VCPUS: 0
> 2014-04-29 17:44:04.895 3462 INFO nova.compute.resource_tracker [-]
> Compute_service record updated for openstack-compute02:openstack-compute02
> 2014-04-29 17:45:05.270 3462 AUDIT nova.compute.resource_tracker [-]
> Auditing locally available compute resources
> 2014-04-29 17:45:05.412 3462 AUDIT nova.compute.resource_tracker [-] Free
> ram (MB): -38
> 2014-04-29 17:45:05.412 3462 AUDIT nova.compute.resource_tracker [-] Free
> disk (GB): 5
> 2014-04-29 17:45:05.413 3462 AUDIT nova.compute.resource_tracker [-] Free
> VCPUS: 0
>
>
> Super Wang
>
> _______________________________________________
> Mailing list:
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
> Post to : openstack at lists.openstack.org
> Unsubscribe :
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20140518/aaf8ca93/attachment.html>
More information about the Openstack
mailing list