[Openstack] High Latency to VMs

mad Engineer themadengin33r at gmail.com
Thu Dec 11 09:13:33 UTC 2014


Not in openstack,i had performance issue, with OVS and bursty traffic
upgrading to later version improved the performance.A lot of
performance features have been added in  2.1.3.

Do you have lots of lost: value in
ovs-dpctl show


On Thu, Dec 11, 2014 at 2:33 AM, André Aranha <andre.f.aranha at gmail.com> wrote:
> Yes, we are using version 2.0.2.
> The process uses only about 0.3% on network node and compute node.
> Did you have the same issue?
>
> On 10 December 2014 at 14:31, mad Engineer <themadengin33r at gmail.com> wrote:
>>
>> are you using openvswitch? which version?
>> if yes,is it consuming a lot of CPU?
>>
>> On Wed, Dec 10, 2014 at 7:45 PM, André Aranha <andre.f.aranha at gmail.com>
>> wrote:
>> > Well, here we are using de Icehouse with Ubuntu 14.04 LTS
>> >
>> > We found this thread in the community  and we apply the changes in the
>> > compute nodes (change VHOST_NET_ENABLED to 1 in /etc/default/qemu-kvm).
>> > After do this, a few instances the problem doesn't exists anymore. This
>> > link
>> > show an investigation to find the problem.
>> >
>> > About the MTU in our cloud (using iperf),
>> >
>> > 1-from any the Desktop to the Network Node
>> > MSS size 1448 bytes (MTU 1500 bytes, ethernet)
>> >
>> > 2-from any Desktop to the instance
>> > MSS size 1348 bytes (MTU 1388 bytes, unknown interface)
>> >
>> > 3- from any instance to the Network Node
>> > MSS size 1348 bytes (MTU 1388 bytes, unknown interface)
>> >
>> > 4- from any instance to the Desktop
>> > MSS size 1348 bytes (MTU 1388 bytes, unknown interface)
>> >
>> > 5-from Network Node to any ComputeNode
>> > MSS size 1448 bytes (MTU 1500 bytes, ethernet)
>> >
>> > 6-from any ComputeNode to NetworkNode
>> > MSS size 1448 bytes (MTU 1500 bytes, ethernet)
>> >
>> > On 10 December 2014 at 10:31, somshekar kadam <som_kadam at yahoo.co.in>
>> > wrote:
>> >>
>> >> Sorry for wrong post mail chain.
>> >>
>> >>
>> >> Regards
>> >> Neelu
>> >>
>> >>
>> >> On Wednesday, 10 December 2014 6:59 PM, somshekar kadam
>> >> <som_kadam at yahoo.co.in> wrote:
>> >>
>> >>
>> >> Hi All,
>> >>
>> >> Please recommend which stable Host OS to use for Controller and Compute
>> >> node.
>> >> I have tried Fedora20 seems lot of tweaking is required, corerct me If
>> >> I
>> >> am wrong.
>> >> I see that most of it is tested on ubuntu and centos.
>> >> I am planning to use JUNO stable version.
>> >> Please help on this
>> >>
>> >>
>> >> Regards
>> >> Neelu
>> >>
>> >>
>> >> On Wednesday, 10 December 2014 5:42 PM, Hannah Fordham
>> >> <hfordham at radiantworlds.com> wrote:
>> >>
>> >>
>> >> I'm afraid we didn't, we're still struggling with some VMs with this
>> >> problem. Sorry!
>> >>
>> >> On 9 December 2014 14:09:32 GMT+00:00, "André Aranha"
>> >> <andre.f.aranha at gmail.com> wrote:
>> >>
>> >> Hi,
>> >>
>> >> We are with the same issue here, and already try some solutions that
>> >> didn't work at all. Did you solved this problem?
>> >>
>> >> Thank you,
>> >> Andre Aranha
>> >>
>> >> On 27 August 2014 at 08:17, Hannah Fordham <hfordham at radiantworlds.com>
>> >> wrote:
>> >>
>> >> I’ve been trying to figure this one out for a while, so I’ll try and be
>> >> as
>> >> thorough as possible in this post but apologies if I miss anything
>> >> pertinent
>> >> out.
>> >>
>> >> First off, I’m running a set up with one control node and 5 compute
>> >> nodes,
>> >> all created using the Stackgeek scripts -
>> >> http://www.stackgeek.com/guides/gettingstarted.html. The first two
>> >> (compute1
>> >> and compute 2) were created at the same time, compute3, 4 and 5 were
>> >> added
>> >> as needed later. My VMs are predominantly CentOS, while my Openstack
>> >> nodes
>> >> are Ubuntu 14.04.1
>> >>
>> >> The symptom: irregular high latency/packet loss to VMs on all compute
>> >> boxes except compute3. Mostly a pain when trying to do anything via ssh
>> >> on a
>> >> VM because the lag makes it difficult to do anything, but it shows
>> >> itself
>> >> quite nicely through pings as well:
>> >> --- 10.0.102.47 ping statistics ---
>> >> 111 packets transmitted, 103 received, 7% packet loss, time 110024ms
>> >> rtt min/avg/max/mdev = 0.096/367.220/5593.100/1146.920 ms, pipe 6
>> >>
>> >>
>> >> I have tested these pings:
>> >> VM to itself (via its external IP) seems fine
>> >> VM to another VM is not fine
>> >> Hosting compute node to VM is not fine
>> >> My PC to VM is not fine (however the other way round works fine)
>> >>
>> >>
>> >> Top on a (32 core) compute node with laggy VMs:
>> >> top - 12:09:20 up 33 days, 21:35,  1 user,  load average: 2.37, 4.95,
>> >> 6.23
>> >> Tasks: 431 total,   2 running, 429 sleeping,   0 stopped,   0 zombie
>> >> %Cpu(s):  0.6 us,  3.4 sy,  0.0 ni, 96.0 id,  0.0 wa,  0.0 hi,  0.0 si,
>> >> 0.0 st
>> >> KiB Mem:  65928256 total, 44210348 used, 21717908 free,   341172
>> >> buffers
>> >> KiB Swap:  7812092 total,  1887864 used,  5924228 free.  7134740 cached
>> >> Mem
>> >>
>> >> And for comparison, on the one compute node that doesn’t seem to be
>> >> suffering from this:
>> >> top - 12:12:20 up 33 days, 21:38,  1 user,  load average: 0.28, 0.18,
>> >> 0.15
>> >> Tasks: 399 total,   3 running, 396 sleeping,   0 stopped,   0 zombie
>> >> %Cpu(s):  0.3 us,  0.1 sy,  0.0 ni, 98.9 id,  0.6 wa,  0.0 hi,  0.0 si,
>> >> 0.0 st
>> >> KiB Mem:  65928256 total, 49986064 used, 15942192 free,   335788
>> >> buffers
>> >> KiB Swap:  7812092 total,   919392 used,  6892700 free. 39272312 cached
>> >> Mem
>> >>
>> >> Top on a laggy VM:
>> >> top - 11:02:53 up 27 days, 33 min,  3 users,  load average: 0.00, 0.00,
>> >> 0.00
>> >> Tasks:  91 total,   1 running,  90 sleeping,   0 stopped,   0 zombie
>> >> Cpu(s):  0.2%us,  0.1%sy,  0.0%ni, 99.5%id,  0.1%wa,  0.0%hi,  0.0%si,
>> >> 0.0%st
>> >> Mem:   1020400k total,   881004k used,   139396k free,   162632k
>> >> buffers
>> >> Swap:  1835000k total,    14984k used,  1820016k free,   220644k cached
>> >>
>> >> http://imgur.com/blULjDa shows the hypervisor panel of Horizon. As you
>> >> can
>> >> see, Compute 3 has fewer resources used, but none of the compute nodes
>> >> should be anywhere near overloaded from what I can tell.
>> >>
>> >> Any ideas? Let me know if I’m missing anything obvious that would help
>> >> with figuring this out!
>> >>
>> >> Hannah
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> ***********
>> >>
>> >> Radiant Worlds Limited is registered in England (company no: 07822337).
>> >> This message is intended solely for the addressee and may contain
>> >> confidential information. If you have received this message in error
>> >> please
>> >> send it back to us and immediately and permanently delete it from your
>> >> system. Do not use, copy or disclose the information contained in this
>> >> message or in any attachment. Please also note that transmission cannot
>> >> be
>> >> guaranteed to be secure or error-free.
>> >>
>> >> _______________________________________________
>> >> Mailing list:
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >> Post to     : openstack at lists.openstack.org
>> >> Unsubscribe :
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >>
>> >>
>> >>
>> >> --
>> >> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>> >>
>> >>
>> >> ***********
>> >>
>> >> Radiant Worlds Limited is registered in England (company no: 07822337).
>> >> This message is intended solely for the addressee and may contain
>> >> confidential information. If you have received this message in error
>> >> please
>> >> send it back to us and immediately and permanently delete it from your
>> >> system. Do not use, copy or disclose the information contained in this
>> >> message or in any attachment. Please also note that transmission cannot
>> >> be
>> >> guaranteed to be secure or error-free.
>> >>
>> >> _______________________________________________
>> >> Mailing list:
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >> Post to    : openstack at lists.openstack.org
>> >> Unsubscribe :
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >>
>> >>
>> >>
>> >> _______________________________________________
>> >> Mailing list:
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >> Post to    : openstack at lists.openstack.org
>> >> Unsubscribe :
>> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >>
>> >>
>> >
>> >
>> > _______________________________________________
>> > Mailing list:
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> > Post to     : openstack at lists.openstack.org
>> > Unsubscribe :
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>> >
>
>




More information about the Openstack mailing list