[Openstack] OVS-DPDK with NetVirt

d.lake at surrey.ac.uk d.lake at surrey.ac.uk
Mon Aug 13 23:08:33 UTC 2018


I'm really getting nowhere fast with this.

The latest in set of issues appears to be related to the "Permission denied" on the socket for qemu.

Just to reprise - this is OVS with DPDK, All-In-One with Intel NICs and ODL NetVirt.

Can ANYONE shed any light on this please - I can't believe that this isn't a very standard deployment and given that it works without DPDK on OVS I can't believe that it hasn't been seen hundreds of times beore.

Thanks

David

From: Lake D Mr (PG/R - Elec Electronic Eng)
Sent: 13 August 2018 16:35
To: 'Venkatrangan G - ERS, HCL Tech' <venkatrangang at hcl.com>; dayavanti.gopal.kamath at ericsson.com; netvirt-dev at lists.opendaylight.org
Subject: RE: OVS-DPDK with NetVirt

Hi

OK - I found some more guides which told me I needed to add:

[ovs]
datapath_type=netdev


to ML2_conf which I have done with an extra line in local.conf.

Now I am seeing the ports trying to be added as vhost-user ports.

BUT.  I am seeing this issue in the log:

qemu-kvm: -chardev socket,id=charnet0,path=/var/run/openvswitch/vhuab608c58-ae: Failed to connect socket /var/run/openvswitch/vhuab608c58-ae: Permission denied\n']#033[00m

Any ideas?   This is on an all-in-one system using CentOS 7.5

Thanks

David

From: Venkatrangan G - ERS, HCL Tech <venkatrangang at hcl.com<mailto:venkatrangang at hcl.com>>
Sent: 13 August 2018 10:36
To: Lake D Mr (PG/R - Elec Electronic Eng) <d.lake at surrey.ac.uk<mailto:d.lake at surrey.ac.uk>>; dayavanti.gopal.kamath at ericsson.com<mailto:dayavanti.gopal.kamath at ericsson.com>; netvirt-dev at lists.opendaylight.org<mailto:netvirt-dev at lists.opendaylight.org>
Subject: RE: OVS-DPDK with NetVirt

Hi David,

I think you can run this ommand on your control node



 sudo neutron-odl-ovs-hostconfig --config-file=/etc/neutron/neutron.conf --debug --ovs_dpdk --bridge_mappings=physnet1:br-physnet1


(Not exactly sure of all the arguments, Please run this command in the control node with dpdk option, I think that should help)



Regards,
Venkat G
(When there is no wind....row!!!)

From: netvirt-dev-bounces at lists.opendaylight.org<mailto:netvirt-dev-bounces at lists.opendaylight.org> <netvirt-dev-bounces at lists.opendaylight.org<mailto:netvirt-dev-bounces at lists.opendaylight.org>> On Behalf Of d.lake at surrey.ac.uk<mailto:d.lake at surrey.ac.uk>
Sent: 13 August 2018 14:01
To: dayavanti.gopal.kamath at ericsson.com<mailto:dayavanti.gopal.kamath at ericsson.com>; netvirt-dev at lists.opendaylight.org<mailto:netvirt-dev at lists.opendaylight.org>
Subject: Re: [netvirt-dev] OVS-DPDK with NetVirt

Good morning all

I wonder if someone could help with this please.

I don't know whether I need to add anything into ML2 to have the br-int installed in netdev mode or whether something else is wrong.

Thank you in advance

David

Sent from my iPhone
________________________________
From: Lake D Mr (PG/R - Elec Electronic Eng)
Sent: Friday, August 10, 2018 10:57:02 PM
To: Dayavanti Gopal Kamath; netvirt-dev at lists.opendaylight.org<mailto:netvirt-dev at lists.opendaylight.org>
Subject: RE: OVS-DPDK with NetVirt

Hi

The first link you sent doesn't work?

I've no idea what a pseudoagent binding driver is....

All I've done is to follow the instructions for moving to DPDK on my existing ODL+OpenStack system which uses Devstack to install.

My understanding is that I needed to enable DPDK in OVS.  I do that with the following command:

                ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true

I then unbound the DPDK NICs from the kernel mode driver and bound them to vfio-pci using "dpdk-devbind."

Once that is done, I created 4 bridges in OVS which all use the netdev datapath:

                ovs-vsctl add-br br-dpdk1 -- set bridge br-dpdk1 datapath_type=netdev
ovs-vsctl add-br br-dpdk2 -- set bridge br-dpdk2 datapath_type=netdev
ovs-vsctl add-br br-dpdk3 -- set bridge br-dpdk3 datapath_type=netdev
ovs-vsctl add-br br-dpdk4 -- set bridge br-dpdk4 datapath_type=netdev


Then I added the ports for the NICs to each bridge:

sudo ovs-vsctl add-port br-dpdk1 dpdk-p1 -- set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:04:00.0
sudo ovs-vsctl add-port br-dpdk2 dpdk-p2 -- set Interface dpdk-p2 type=dpdk options:dpdk-devargs=0000:04:00.1
sudo ovs-vsctl add-port br-dpdk3 dpdk-p3 -- set Interface dpdk-p3 type=dpdk options:dpdk-devargs=0000:05:00.0
sudo ovs-vsctl add-port br-dpdk4 dpdk-p4 -- set Interface dpdk-p4 type=dpdk options:dpdk-devargs=0000:05:00.1

Having done that, I can verify that I can see traffic in the bridge using ovs-tcpdump so I know that the data is reaching OVS from the wire.

Then I run Devstack stack.sh and I get a working system with four physical networks.

However, this blog - https://joshhershberg.wordpress.com/2017/03/07/opendaylight-netvirt-dpdk-plumbing-how-it-all-works-together/<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjoshhershberg.wordpress.com%2F2017%2F03%2F07%2Fopendaylight-netvirt-dpdk-plumbing-how-it-all-works-together%2F&data=02%7C01%7Cvenkatrangang%40hcl.com%7Cc6b5accb8e794361475108d600f71878%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636697458633205453&sdata=69o0NMiSzo0mYbUI0oTHFb7uX1dctilETuo8GfHmr%2Bs%3D&reserved=0> seems to indicate that the br-int should be automatically created by ODL as part of the installation process in netdev mode by virtue of the fact that it has read the datapath type from OVSDB and would therefore ensure that all ports are created with netdev.

But this doesn't appear to be happening because I see messages in karaf.log telling me that the ports are NOT in dpdk mode.

The symptom is that when I create a VM, a TAP interface is built and I can see traffic into OVS and to/from the netns qdhcp, but traffic is not crossing between the br-dpdk ports and the ports associated with the VMs.

I've also read this note https://software.intel.com/en-us/forums/networking/topic/704506<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsoftware.intel.com%2Fen-us%2Fforums%2Fnetworking%2Ftopic%2F704506&data=02%7C01%7Cvenkatrangang%40hcl.com%7Cc6b5accb8e794361475108d600f71878%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636697458633215465&sdata=B8lGtKa0Odhx0gqZFkGMDyJT%2FyNtOaGO0Q5REzsRgy0%3D&reserved=0> which seems to indicate some additional ML2 configuration is required but that would seem to run counter to the instructions given in the blog referenced earlier!

I'm loathed to start manually changing anything in the OVS table because last time I asked a question about adding OVS rules to do routing across OVS I was told that really one should not touch the OVS tables manually if integrated with ODL and NetVirt.

This is all rather confusing.

David

From: Dayavanti Gopal Kamath <dayavanti.gopal.kamath at ericsson.com<mailto:dayavanti.gopal.kamath at ericsson.com>>
Sent: 10 August 2018 19:03
To: Lake D Mr (PG/R - Elec Electronic Eng) <d.lake at surrey.ac.uk<mailto:d.lake at surrey.ac.uk>>; netvirt-dev at lists.opendaylight.org<mailto:netvirt-dev at lists.opendaylight.org>
Subject: RE: OVS-DPDK with NetVirt

Hi david,
Are you using the pseudoagent binding driver for binding the vif? In that case, ovsdb openvswitch table needs to be populated with host config information-
https:/github/.com/openstack/networking-odl/blob/master/doc/source/devref/hostconfig.rst<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenstack%2Fnetworking-odl%2Fblob%2Fmaster%2Fdoc%2Fsource%2Fdevref%2Fhostconfig.rst&data=02%7C01%7Cvenkatrangang%40hcl.com%7Cc6b5accb8e794361475108d600f71878%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636697458633215465&sdata=C9OPlyVbsr2VOBOkMUOoHa%2BqhRAnhy%2FW0T2rFF1czd8%3D&reserved=0>
https://blueprints.launchpad.net/networking-odl/+spec/pseudo-agentdb-binding<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblueprints.launchpad.net%2Fnetworking-odl%2F%2Bspec%2Fpseudo-agentdb-binding&data=02%7C01%7Cvenkatrangang%40hcl.com%7Cc6b5accb8e794361475108d600f71878%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636697458633225470&sdata=KmdKNMECn%2BqY6C6atCnL7QKhHxDoOkpsKgLxfAMec10%3D&reserved=0>

for netdev, your openvswitch table could look like this -
external_ids: odl_os_hostconfig_hostid=<host_name>
external_ids: host_type= ODL_L2
external_ids: odl_os_hostconfig_config_odl_l2 = "{"supported_vnic_types": [{"vnic_type": ["normal"], "vif_type": "vhostuser", "vif_details": {"uuid": "TEST_UUID", "has_datapath_type_netdev": True, "support_vhost_user": True, "port_prefix": "vhu", "vhostuser_socket_dir": "/var/run/openvswitch", "vhostuser_ovs_plug": True, "vhostuser_mode": "server", "vhostuser_socket": "/var/run/openvswitch/vhu$PORT_ID"} }], "allowed_network_types": ["vlan", "vxlan"], "bridge_mappings": {" physnet1":"br-ex"}}"

From: netvirt-dev-bounces at lists.opendaylight.org<mailto:netvirt-dev-bounces at lists.opendaylight.org> [mailto:netvirt-dev-bounces at lists.opendaylight.org] On Behalf Of d.lake at surrey.ac.uk<mailto:d.lake at surrey.ac.uk>
Sent: Friday, August 10, 2018 7:59 PM
To: netvirt-dev at lists.opendaylight.org<mailto:netvirt-dev at lists.opendaylight.org>
Subject: [netvirt-dev] OVS-DPDK with NetVirt

Hello

I have installed OVS with DPDK support and created bridges to map my DPDK-mode interfaces to provider networks as below:

ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true

ovs-vsctl add-br br-dpdk1 -- set bridge br-dpdk1 datapath_type=netdev
ovs-vsctl add-br br-dpdk2 -- set bridge br-dpdk2 datapath_type=netdev
ovs-vsctl add-br br-dpdk3 -- set bridge br-dpdk3 datapath_type=netdev
ovs-vsctl add-br br-dpdk4 -- set bridge br-dpdk4 datapath_type=netdev

sudo ovs-vsctl add-port br-dpdk1 dpdk-p1 -- set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:04:00.0
sudo ovs-vsctl add-port br-dpdk2 dpdk-p2 -- set Interface dpdk-p2 type=dpdk options:dpdk-devargs=0000:04:00.1
sudo ovs-vsctl add-port br-dpdk3 dpdk-p3 -- set Interface dpdk-p3 type=dpdk options:dpdk-devargs=0000:05:00.0
sudo ovs-vsctl add-port br-dpdk4 dpdk-p4 -- set Interface dpdk-p4 type=dpdk options:dpdk-devargs=0000:05:00.1


I have ODL provider mappings between physnet1:br-dpdk1 etc and I can create flat networks  using the provider network names.

BUT.   I am still seeing the tap interfaces in the ovs-vsctl show and in karaf.log it appears that the VM interfaces are NOT being created as type vhostuser.

This blog - https://joshhershberg.wordpress.com/2017/03/07/opendaylight-netvirt-dpdk-plumbing-how-it-all-works-together/<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fjoshhershberg.wordpress.com%2F2017%2F03%2F07%2Fopendaylight-netvirt-dpdk-plumbing-how-it-all-works-together%2F&data=02%7C01%7Cvenkatrangang%40hcl.com%7Cc6b5accb8e794361475108d600f71878%7C189de737c93a4f5a8b686f4ca9941912%7C0%7C0%7C636697458633235478&sdata=aL6SMxLq%2FNk0JZz1O22hV1JMB7oV0UFLygBTL7RqJXo%3D&reserved=0> - seems to suggest that the br-int should be created as a netdev but I don't think this is happening.

Is there any config change I need to make to ML2 to make br-int into a netdev datapath?

Thanks

David

::DISCLAIMER::
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents (with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates. Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of this message without the prior written consent of authorized representative of HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately. Before opening any email and/or attachments, please check them for viruses and other defects.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20180813/e8a3632a/attachment-0001.html>


More information about the Openstack mailing list