[openstack-dev] [networking-ovs-dpdk]
Prathyusha Guduri
prathyushaconnects at gmail.com
Tue Nov 24 10:32:37 UTC 2015
Hi Sean,
Thanks for you kind help.
I did the following.
# apt-get install ubuntu-cloud-keyring
# echo "deb http://ubuntu-cloud.archive.canonical.com/ubuntu
<http://www.google.com/url?q=http%3A%2F%2Fubuntu-cloud.archive.canonical.com%2Fubuntu&sa=D&sntz=1&usg=AFQjCNGlCfoplf1zSbILNxCSVK23zRxa2A>"
\
"trusty-updates/kilo main" > /etc/apt/sources.list.d/cloudarchive-kilo.list
# apt-get update && apt-get dist-upgrade
and then uninstalled the libvirt and qemu that were installed manually and
then ran stack.sh after cleaning and unstacking.
Now fortunately libvirt and qemu satisfy minimum requirements.
$ virsh --version
1.2.12
$ kvm --version
/usr/bin/kvm: line 42: /tmp/qemu.orig: Permission denied
QEMU emulator version 2.2.0 (Debian 1:2.2+dfsg-5expubuntu9.3~cloud0),
Copyright (c) 2003-2008 Fabrice Bellard
Am using an ubuntu 14.04 system
$ uname -a
Linux ubuntu-Precision-Tower-5810 3.13.0-24-generic #46-Ubuntu SMP Thu Apr
10 19:11:08 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
After stack.sh which was successful, tried creating a new instance - which
gave an ERROR again.
$ nova list
+--------------------------------------+----------------+--------+------------+-------------+-------------------------------------------------------+
| ID | Name | Status | Task
State | Power State | Networks
|
+--------------------------------------+----------------+--------+------------+-------------+-------------------------------------------------------+
| 31a7e160-d04c-4216-91cf-30ce86c2b1fa | demo-instance1 | ERROR |
- | NOSTATE | private=10.0.0.3,
fd34:f4c5:412:0:f816:3eff:fea4:b9fe |
$ sudo service ovs-dpdk status
sourcing config
ovs alive
VHOST_CONFIG: bind to /var/run/openvswitch/vhufb8052e5-d3
2015-11-24T10:23:25Z|00126|dpdk|INFO|Socket
/var/run/openvswitch/vhufb8052e5-d3 created for vhost-user port
vhufb8052e5-d3
2015-11-24T10:23:25Z|00004|dpif_netdev(pmd18)|INFO|Core 2 processing port
'vhufb8052e5-d3'
2015-11-24T10:23:25Z|00002|dpif_netdev(pmd19)|INFO|Core 8 processing port
'dpdk0'
2015-11-24T10:23:25Z|00127|bridge|INFO|bridge br-int: added interface
vhufb8052e5-d3 on port 6
2015-11-24T10:23:25Z|00005|dpif_netdev(pmd18)|INFO|Core 2 processing port
'dpdk0'
2015-11-24T10:23:26Z|00128|connmgr|INFO|br-int<->unix: 1 flow_mods in the
last 0 s (1 deletes)
2015-11-24T10:23:26Z|00129|ofp_util|INFO|normalization changed ofp_match,
details:
2015-11-24T10:23:26Z|00130|ofp_util|INFO| pre:
in_port=5,nw_proto=58,tp_src=136
2015-11-24T10:23:26Z|00131|ofp_util|INFO|post: in_port=5
2015-11-24T10:23:26Z|00132|connmgr|INFO|br-int<->unix: 1 flow_mods in the
last 0 s (1 deletes)
2015-11-24T10:23:26Z|00133|connmgr|INFO|br-int<->unix: 1 flow_mods in the
last 0 s (1 deletes)
2015-11-24T10:23:29Z|00134|bridge|WARN|could not open network device
vhufb8052e5-d3 (No such device)
VHOST_CONFIG: socket created, fd:52
VHOST_CONFIG: bind to /var/run/openvswitch/vhufb8052e5-d3
2015-11-24T10:23:29Z|00135|dpdk|INFO|Socket
/var/run/openvswitch/vhufb8052e5-d3 created for vhost-user port
vhufb8052e5-d3
2015-11-24T10:23:29Z|00006|dpif_netdev(pmd18)|INFO|Core 2 processing port
'vhufb8052e5-d3'
2015-11-24T10:23:29Z|00003|dpif_netdev(pmd19)|INFO|Core 8 processing port
'dpdk0'
2015-11-24T10:23:29Z|00136|bridge|INFO|bridge br-int: added interface
vhufb8052e5-d3 on port 7
2015-11-24T10:23:30Z|00007|dpif_netdev(pmd18)|INFO|Core 2 processing port
'dpdk0'
0
I understand that ovs-dpdk is running. The error log of n-cpu.log is
2015-11-24 15:47:27.957 ^[[01;31mERROR nova.compute.manager
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone
None^[[01;31m] ^[[01;35m^[[01;31mNo compute node record for host
ubuntu-Precision-Tower-5810^[[00m
2015-11-24 15:47:27.964 ^[[01;33mWARNING nova.compute.monitors
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone
None^[[01;33m] ^[[01;35m^[[01;33mExcluding nova.compute.monitors.cpu
monitor virt_driver. Not in the list of enabled monitors
(CONF.compute_monitors).^[[00m
2015-11-24 15:47:27.964 ^[[00;36mINFO nova.compute.resource_tracker
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone
None^[[00;36m] ^[[01;35m^[[00;36mAuditing locally available compute
resources for node ubuntu-Precision-Tower-5810^[[00m
2015-11-24 15:47:28.087 ^[[00;32mDEBUG nova.compute.resource_tracker
[^[[01;36mreq-24fc3f16-ccd5-4e2d-b583-60ade23bc1ed ^[[00;36mNone
None^[[00;32m] ^[[01;35m^[[00;32mHypervisor: free VCPUs: 9^[[00m
^[[00;33mfrom (pid=7880) _report_hypervisor_resource_view
/opt/stack/nova/nova/compute/resource_tracker.py:588^[[00m
I dont understand where am I missing out now.
On Mon, Nov 23, 2015 at 7:50 PM, Mooney, Sean K <sean.k.mooney at intel.com>
wrote:
> Hi
>
> qemu version was 2.0.0 does not support mapping hugepage as shared.
> as a result the dpdk implementation of vhost-user cannot function with
> this version.
> Similarly libvirt 1.2.2 has no knowledge of vhost-user.
>
> If you are on fedora 21 then the virt-preview repo packages the required
> Libvirt and qemu.
>
> On Ubuntu the kilo cloud archive also packages version that meet the
> minimum versions.
>
> Looking at the log I would agree that this error is probably related to
> manually installing Libvirt and
> not adding the appropriate libvirtd configuration options.
>
> Devstack does not have native support for installing a vhost-user
> compatible Libvirt or qemu.
> when using our networking-ovs-dpdk plugin we ask you to enable the
> virt-prevew or cloud archive
> Before stacking so that when devestack install Libvirt/qemu it get
> compatible versions.
> for centos as we have been unable to find an equivalent we explicitly
> install Libvirt/qemu.
>
> In general we don’t provide support for installing the required version
> Libvirt/qemu in
> the networking-ovs-dpdk devstack plugin.
>
> What os are you currently using? ubuntu?
> in your case enabling the kilo cloud archive, uinstalling Libvirt/qemu and
> then restacking
> should provide the appropriate packages.
>
> In liberty we use that standard neutron openvswich agent binary and set
> the agent type to DPDK OVS Agent to enable
> the ovsdpdk ml2 driver to manage the node. In mitaka the standard ovs
> neutron agent and standard openvswitch ml2 drivers will be used instead.
>
>
>
> To confirm if ovs-dpdk is running you can use the following command
>
> sudo service ovs-dpdk status
>
> this will check if ovs-dpdk is running by reading the pid files and
> checking if the ovsdb and ovs-vswitchd
> process are running and tail the last 20ish lines of the vswitchd log file.
>
> note this is not a systemd service so the systemd equivalent command will
> not work.
>
>
>
> Alternative you can user ps and grep
>
> ps aux | grep ovs
>
>
>
> regards
>
> sean.
>
>
>
>
>
>
>
> *From:* Prathyusha Guduri [mailto:prathyushaconnects at gmail.com]
> *Sent:* Monday, November 23, 2015 9:16 AM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [networking-ovs-dpdk]
>
>
>
> Hi Sean,
>
> Thanks for your help before. It would be great if you look into another
> issue too.
>
> Am able to run stack.sh successfully and all services are up. But,
>
> libvirt version was 1.2.2 and qemu version was 2.0.0
>
> To satisfy the minimum requirement of qemu- version >=2.1 and
> libvirt-version >= 1.2.10
>
> I manually installed qemu and libvirt from respective sources.
>
> Now
> $ kvm --version
> /usr/bin/kvm: line 42: /tmp/qemu.orig: Permission denied
> QEMU emulator version 2.1.3, Copyright (c) 2003-2008 Fabrice Bellard
>
> $ virsh --version
> 1.2.10
>
> So basic requirement is satisfied.
>
> Before creating an instance ran the below command,
> $ nova flavor-key m1.tiny set "hw:mem_page_size=large"
>
> Now created an instance
> $ nova boot --flavor m1.tiny --image cirros-0.3.4-x86_64-uec --nic
> net-id=445e2dc5-221b-48ea-aea4-d04dee12fc7f --security-group default
> demo-instance1
>
> It gives the ERROR :
>
> 2015-11-23 13:19:59.654 ERROR nova.virt.libvirt.host
> [req-2d9d060d-1934-4e9e-af1c-010e177bea11 None None] Connection to
> libvirt failed: error from service: CheckAuthorization: Did not receive a
> reply. Possible causes include: the remote application did not send a
> reply, the message bus security policy blocked the reply, the reply timeout
> expired, or the network connection was broken.
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host Traceback (most
> recent call last):
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/opt/stack/nova/nova/virt/libvirt/host.py", line 527, in get_connection
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host conn =
> self._get_connection()
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/opt/stack/nova/nova/virt/libvirt/host.py", line 514, in _get_connection
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host wrapped_conn =
> self._get_new_connection()
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/opt/stack/nova/nova/virt/libvirt/host.py", line 466, in
> _get_new_connection
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host wrapped_conn =
> self._connect(self._uri, self._read_only)
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/opt/stack/nova/nova/virt/libvirt/host.py", line 320, in _connect
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host libvirt.openAuth,
> uri, auth, flags)
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 141, in
> proxy_call
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host rv = execute(f,
> *args, **kwargs)
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 122, in
> execute
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host six.reraise(c, e,
> tb)
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/usr/local/lib/python2.7/dist-packages/eventlet/tpool.py", line 80, in
> tworker
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host rv = meth(*args,
> **kwargs)
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host File
> "/usr/local/lib/python2.7/dist-packages/libvirt.py", line 105, in openAuth
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host if ret is
> None:raise libvirtError('virConnectOpenAuth() failed')
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host libvirtError: error
> from service: CheckAuthorization: Did not receive a reply. Possible causes
> include: the remote application did not send a reply, the message bus
> security policy blocked the reply, the reply timeout expired, or the
> network connection was broken.
> 2015-11-23 13:19:59.654 TRACE nova.virt.libvirt.host
> Traceback (most recent call last):
>
> I suspect this is because, I manually installed libvirt and qemu. My doubt
> is why devstack is not installing a correct version when it is supposed to.
> why a version less than min requirement is being installed???
>
> Now that because am installing manually, there might be a problem with
> groups - devstack creates some group and installs but manual installation
> doesn't bother about that groups.
> Can you please suggest a way on how do avoid that???
>
>
>
> Also, I just want to make sure that the agent running is
> neutron-openvswitch only. No ovsdpdk agent running.
>
> $ ps -Al | grep neutron
> 0 S 1000 8882 8859 3 80 0 - 49946 ep_pol pts/34 00:02:24
> neutron-openvsw
>
> But
>
> $ neutron agent-list
> 3385a430-5738-43cb-b853-059add5ab602 | DPDK OVS Agent |
> ubuntu-Precision-Tower-5810 | :-) | True |
> neutron-openvswitch-agent
>
> So this implies that dpdk agent is running right??? I remember reading in
> launchpad bugs that ovsdpdk agent is removed and that now openvswitch takes
> care of everything. Just wanted to confirm that my setup has ovs-dpdk
> running.
>
> Regards,
>
> Prathyusha
>
>
>
>
>
> On Wed, Nov 18, 2015 at 7:23 PM, James Page <james.page at ubuntu.com> wrote:
>
> Hi Sean
>
>
>
> On Wed, Nov 18, 2015 at 12:30 PM, Mooney, Sean K <sean.k.mooney at intel.com>
> wrote:
>
> Hi james
>
> Yes we are planning on testing the packaged release to see if it is
> compatible with our ml2 driver and the
>
> Changes we are submitting upstream. If it is we will add a use binary flag
> to our devstack plugin to skip the
>
> Compilation step and use that instead on 15.10 or 14.04
> cloud-archive:liberty
>
>
>
> Excellent.
>
>
>
> As part of your packaging did ye fix pciutils to correctly report the
> unused drivers when an interface is bound
>
> The dpdk driver? Also does it support both igb_uio and/or vfio-pci drivers
> for dpdk interface?
>
>
>
> Re pcituils, we've not done any work in that area - can you give an
> example of what you would expect?
>
>
>
> The dpdk package supports both driver types in /etc/dpdk/interfaces - when
> you declare an adapter for use, you get to specify the module you want to
> use as well; we're relying the in-tree kernel drivers (uio-pci-generic and
> vfio-pci) right now.
>
>
>
>
>
> Anyway yes I hope to check it out and seeing what ye have done. When
> ovs-dpdk starts getting packaged in more operating systems
>
> We will probably swap our default to the binary install though we will
> keep the source install option as it allows us to work on new features
>
> Before they are packaged and to have better performance.
>
>
>
> That sounds sensible; re 'better performance' - yeah we do have to
> baseline the optimizations at compile time right now (ssse3 only right now)
> , but I really hope that does change so that we can move to a runtime CPU
> feature detection model, allowing the best possible performance through the
> packages we have in Ubuntu (or any other distribution for that matter).
>
>
>
>
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-dev/attachments/20151124/38d01575/attachment.html>
More information about the OpenStack-dev
mailing list