[ISSUE]Openstack create VNF instance error

Jhen-Hao Yu tcr1br24 at gmail.com
Wed Mar 10 08:11:06 UTC 2021


Dear Sir,

This our testbed:
Openstack (stein) + Opendaylight (neon) + ovs (v2.11.1)

We have trouble when creating vnf on compute node using OpenStack CLI:
#openstack vnf create vnfd1 vnf1

Here are the log message and ovs information.*COMPUTE NODE1:*
nova-compute.log
=============================

Attempting claim on node cmp001: memory 2048 MB, disk 15 GB, vcpus 1 CPU
Total memory: 7976 MB, used: 6656.00 MB
memory limit not specified, defaulting to unlimited
Total disk: 96 GB, used: 45.00 GB
disk limit not specified, defaulting to unlimited
Total vcpu: 4 VCPU, used: 3.00 VCPU
vcpu limit not specified, defaulting to unlimited
Claim successful on node cmp001
Creating image
ERROR vif_plug_ovs.ovsdb.impl_vsctl
[req-a514ba2a-7ada-4a35-944e-0c861055112e
eacae20dfeb9442eb8643bb8784b03d1 caee3c14634f42438ec8479eb49ba388 -
default default] Unable to execute ['ovs-vsctl', '--timeout=120',
'--oneline', '--format=json', '--db=tcp:127.0.0.1:6640', '--',
'--may-exist', 'add-br', 'br-int', '--', 'set', 'Bridge', 'br-int',
'datapath_type=netdev']. Exception: Unexpected error while running
command.
Command: ovs-vsctl --timeout=120 --oneline --format=json
--db=tcp:127.0.0.1:6640 -- --may-exist add-br br-int -- set Bridge
br-int datapath_type=netdev
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed
(Connection refused)\n':
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error
while running command.
ERROR vif_plug_ovs.ovsdb.impl_vsctl  Unable to execute ['ovs-vsctl',
'--timeout=120', '--oneline', '--format=json',
'--db=tcp:127.0.0.1:6640', '--', '--may-exist', 'add-port', 'br-int',
'vhu5e8b5246-cd', '--', 'set', 'Interface', 'vhu5e8b5246-cd',
'external_ids:iface-id=5e8b5246-cd05-4735-9692-29d51db6e72d',
'external_ids:iface-status=active',
'external_ids:attached-mac=fa:16:3e:8f:b1:a1',
'external_ids:vm-uuid=30675d24-bd74-409c-ba36-4c59a72283ec',
'type=dpdkvhostuser']. Exception: Unexpected error while running
command.
Command: ovs-vsctl --timeout=120 --oneline --format=json
--db=tcp:127.0.0.1:6640 -- --may-exist add-port br-int vhu5e8b5246-cd
-- set Interface vhu5e8b5246-cd
external_ids:iface-id=5e8b5246-cd05-4735-9692-29d51db6e72d
external_ids:iface-status=active
external_ids:attached-mac=fa:16:3e:8f:b1:a1
external_ids:vm-uuid=30675d24-bd74-409c-ba36-4c59a72283ec
type=dpdkvhostuser
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed
(Connection refused)\n':
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error
while running command.
2021-03-09 06:30:36.594 3282 ERROR vif_plug_ovs.ovsdb.impl_vsctl
[req-a514ba2a-7ada-4a35-944e-0c861055112e
eacae20dfeb9442eb8643bb8784b03d1 caee3c14634f42438ec8479eb49ba388 -
default default] Unable to execute ['ovs-vsctl', '--timeout=120',
'--oneline', '--format=json', '--db=tcp:127.0.0.1:6640', '--',
'--columns=mtu_request', 'list', 'Interface']. Exception: Unexpected
error while running command.
Command: ovs-vsctl --timeout=120 --oneline --format=json
--db=tcp:127.0.0.1:6640 -- --columns=mtu_request list Interface
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed
(Connection refused)\n':
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error
while running command.
2021-03-09 06:30:36.594 3282 ERROR os_vif
[req-a514ba2a-7ada-4a35-944e-0c861055112e
eacae20dfeb9442eb8643bb8784b03d1 caee3c14634f42438ec8479eb49ba388 -
default default] Failed to plug vif
VIFVHostUser(active=False,address=fa:16:3e:8f:b1:a1,has_traffic_filtering=False,id=5e8b5246-cd05-4735-9692-29d51db6e72d,mode='client',network=Network(6be19cbf-c6a7-4735-a0ee-42e1c5a664e2),path='/var/run/openvswitch/vhu5e8b5246-cd',plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=True,vif_name='vhu5e8b5246-cd'):
oslo_concurrency.processutils.ProcessExecutionError: Unexpected error
while running command.
Command: ovs-vsctl --timeout=120 --oneline --format=json
--db=tcp:127.0.0.1:6640 -- --columns=mtu_request list Interface
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: tcp:127.0.0.1:6640: database connection failed
(Connection refused)\n'

=============================

ovs information:

===========================

_uuid               : 4a56f11f-f839-4032-ba85-2eec47260144
bridges             : [1a600d85-9849-459a-8453-9a28eee63dc4,
f21a1111-3c01-4988-99b7-ca5ab4a73051]
cur_cfg             : 653
datapath_types      : [netdev, system]
db_version          : "7.16.1"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname="cmp001",
"odl_os_hostconfig_config_odl_l2"="{\"allowed_network_types\":
[\"local\", \"flat\", \"vlan\", \"vxlan\", \"gre\"],
\"bridge_mappings\": {}, \"datapath_type\": \"netdev\",
\"supported_vnic_types\": [{\"vif_details\": {\"uuid\":
\"4a56f11f-f839-4032-ba85-2eec47260144\", \"host_addresses\":
[\"cmp001\"], \"has_datapath_type_netdev\": true,
\"support_vhost_user\": true, \"port_prefix\": \"vhu\",
\"vhostuser_socket_dir\": \"/var/run/openvswitch\",
\"vhostuser_ovs_plug\": true, \"vhostuser_mode\": \"client\",
\"vhostuser_socket\": \"/var/run/openvswitch/vhu$PORT_ID\"},
\"vif_type\": \"vhostuser\", \"vnic_type\": \"normal\"}]}",
odl_os_hostconfig_hostid="cmp001", rundir="/var/run/openvswitch",
system-id="ec007ae5-4b61-4f8e-af2b-b1b97413d20d"}
iface_types         : [erspan, geneve, gre, internal, "ip6erspan",
"ip6gre", lisp, patch, stt, system, tap, vxlan]
manager_options     : [11b3ada9-d9b3-42bb-97fc-d254364acecd]
next_cfg            : 653
other_config        : {local_ip="10.1.0.5",
provider_mappings="physnet1:br-floating"}
ovs_version         : "2.11.1"
ssl                 : []
statistics          : {}
system_type         : ubuntu
system_version      : "18.04"

===========================

# ovs-vsctl get bridge br-int datapath_type

system
===========================

*COMPUTE NODE 2:*

nova-compute.log

===========================

ERROR nova.compute.manager Failed to build and run instance:
libvirt.libvirtError: internal error: process exited while connecting
to monitor:
qemu-system-x86_64: -chardev
socket,id=charnet0,path=/var/run/openvswitch/vhub701eeae-c0: Failed to
connect socket /var/run/openvswitch/vhub701eeae-c0: No such file or
directory
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/nova/compute/manager.py", line
2353, in _build_and_run_instance
    block_device_info=block_device_info)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py",
line 3204, in spawn
    destroy_disks_on_failure=True)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py",
line 5724, in _create_domain_and_network
    destroy_disks_on_failure)
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line
220, in __exit__
    self.force_reraise()
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line
196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py",
line 5693, in _create_domain_and_network
    post_xml_callback=post_xml_callback)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/driver.py",
line 5627, in _create_domain
    guest.launch(pause=pause)
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py",
line 144, in launch
    self._encoded_xml, errors='ignore')
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line
220, in __exit__
    self.force_reraise()
  File "/usr/lib/python3/dist-packages/oslo_utils/excutils.py", line
196, in force_reraise
    six.reraise(self.type_, self.value, self.tb)
  File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/nova/virt/libvirt/guest.py",
line 139, in launch
    return self._domain.createWithFlags(flags)
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 190, in doit
    result = proxy_call(self._autowrap, f, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 148,
in proxy_call
    rv = execute(f, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 129, in execute
    six.reraise(c, e, tb)
  File "/usr/lib/python3/dist-packages/six.py", line 693, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/eventlet/tpool.py", line 83, in tworker
    rv = meth(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/libvirt.py", line 1110, in
createWithFlags
    if ret == -1: raise libvirtError ('virDomainCreateWithFlags()
failed', dom=self)
libvirt.libvirtError: internal error: process exited while connecting
to monitor: 2021-03-10T07:59:04.064555Z qemu-system-x86_64: -chardev
socket,id=charnet0,path=/var/run/openvswitch/vhub701eeae-c0: Failed to
connect socket /var/run/openvswitch/vhub701eeae-c0: No such file or
directory
INFO nova.compute.manager Took 0.68 seconds to deallocate network for instance.
INFO nova.scheduler.client.report Deleted allocation for instance
308de941-1192-4dbd-8c4d-d7860139d6d4

===========================

ovs information:

===========================

_uuid               : b8f5b869-e794-4052-8896-3ee5d323a0b7
bridges             : [5b6ab0c7-d8f1-43b0-9ab7-1e77280cd3ff,
70810190-6796-4500-bda4-dae856bcede1]
cur_cfg             : 1267
datapath_types      : [netdev, system]
db_version          : "7.16.1"
dpdk_initialized    : false
dpdk_version        : none
external_ids        : {hostname="cmp002",
"odl_os_hostconfig_config_odl_l2"="{\"allowed_network_types\":
[\"local\", \"flat\", \"vlan\", \"vxlan\", \"gre\"],
\"bridge_mappings\": {}, \"datapath_type\": \"netdev\",
\"supported_vnic_types\": [{\"vif_details\": {\"uuid\":
\"b8f5b869-e794-4052-8896-3ee5d323a0b7\", \"host_addresses\":
[\"cmp002\"], \"has_datapath_type_netdev\": true,
\"support_vhost_user\": true, \"port_prefix\": \"vhu\",
\"vhostuser_socket_dir\": \"/var/run/openvswitch\",
\"vhostuser_ovs_plug\": true, \"vhostuser_mode\": \"client\",
\"vhostuser_socket\": \"/var/run/openvswitch/vhu$PORT_ID\"},
\"vif_type\": \"vhostuser\", \"vnic_type\": \"normal\"}]}",
odl_os_hostconfig_hostid="cmp002", rundir="/var/run/openvswitch",
system-id="dddc6d6c-0e7c-4813-97c9-7c72eaf9f46b"}
iface_types         : [erspan, geneve, gre, internal, "ip6erspan",
"ip6gre", lisp, patch, stt, system, tap, vxlan]
manager_options     : [29b4ef4b-e2ba-43f3-aa42-b1bacaf9dd19,
75ae95f6-d251-4152-9e0d-5b65e5f7bb77]
next_cfg            : 1267
other_config        : {local_ip="10.1.0.6",
provider_mappings="physnet1:br-floating"}
ovs_version         : "2.11.1"
ssl                 : []
statistics          : {}
system_type         : ubuntu
system_version      : "18.04"

===========================

# ovs-vsctl get bridge br-int datapath_type
netdev

===========================


We don't use DPDK on our Openvswitch.
Can anyone give us some advice on this issue?

Thanks for helping us.


[image: Mailtrack]
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
Sender
notified by
Mailtrack
<https://mailtrack.io?utm_source=gmail&utm_medium=signature&utm_campaign=signaturevirality5&>
03/10/21,
04:09:46 PM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20210310/8a152292/attachment-0001.html>


More information about the openstack-discuss mailing list