<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; font-family: Calibri, sans-serif; ">
<div>> We had to do the same downgrade with openvswitch, the newest version, under heavy load, corrupts packets in-transit, but we do not have the time to investigate the issue further.</div>
<div><br>
</div>
<div>Daniele, what was the openvswitch version before and after the upgrade? And which ethernet drivers do you have? The corruption maybe related to the drivers you have (the issues may be triggered by the way openvswitch flows are configured in Icehouse
vs Havana).</div>
<div><br>
</div>
<div>Thanks.</div>
<div><br>
</div>
<span id="OLK_SRC_BODY_SECTION">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Daniele Venzano <<a href="mailto:daniele.venzano@eurecom.fr">daniele.venzano@eurecom.fr</a>><br>
<span style="font-weight:bold">Organization: </span>Eurecom<br>
<span style="font-weight:bold">Date: </span>Sunday, October 19, 2014 11:46 PM<br>
<span style="font-weight:bold">To: </span>"<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>" <<a href="mailto:openstack-operators@lists.openstack.org">openstack-operators@lists.openstack.org</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Openstack-operators] qemu 1.x to 2.0<br>
</div>
<div><br>
</div>
<div>
<div bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">We have the same setup (Icehouse on Ubuntu 12.04) and had similar issues. We downgraded qemu from 2.x to 1.x, as we cannot terminate all VMs for all users. We had non-resumable VMs also in the middle of the 1.x series and nothing
was documented in the changlelog.<br>
We had to do the same downgrade with openvswitch, the newest version, under heavy load, corrupts packets in-transit, but we do not have the time to investigate the issue further.<br>
<br>
We plan to warn our users in time for the next major upgrade to Juno that all VMs need to be terminated, probably during the Christmas holidays. I do not think they will be happy.<br>
Seeing also all the problems we had upgrading Neutron from OVS to ML2, terminating all VMs is probably the best policy anyway during an OpenStack upgrade. Or you do lots of migrations and upgrade qemu one compute host at the time, but if something goes wrong
you end-up with an angry user and a stuck VM.<br>
<br>
It certainly is a big deal.<br>
<br>
On 10/20/14 00:59, Joe Topjian wrote:<br>
</div>
<blockquote cite="mid:CA+y7hvhC0FnDR2M=tzYJchMQbLPLac78EwwhJLw0kS2KfihhBA@mail.gmail.com" type="cite">
<div dir="ltr">Hello,
<div><br>
</div>
<div>We recently upgraded an OpenStack Grizzly environment to Icehouse (doing a quick stop-over at Havana). This environment is still running Ubuntu 12.04.</div>
<div><br>
</div>
<div>The <a moz-do-not-send="true" href="https://wiki.ubuntu.com/TrustyTahr/ReleaseNotes#Ubuntu_Server">
Ubuntu 14.04 release notes</a> make mention of incompatibilities with 12.04 and moving to 14.04 and qemu 2.0. I didn't think that this would apply for upgrades staying on 12.04, but it indeed does.</div>
<div><br>
</div>
<div>We found that existing instances could not be live migrated (as per the release notes). Additionally, instances that were hard-rebooted and had the libvirt xml file rebuilt could no longer start, either.</div>
<div><br>
</div>
<div>The exact error message we saw was:</div>
<div><br>
</div>
<div>"Length mismatch: vga.vram: 1000000 in != 800000"</div>
<div><br>
</div>
<div>I found a few bugs that are related to this, but I don't think they're fully relevant to the issue I ran into:</div>
<div><br>
</div>
<div><a moz-do-not-send="true" href="https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1308756">https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1308756</a><br>
</div>
<div><a moz-do-not-send="true" href="https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1291321">https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1291321</a><br>
</div>
<div><a moz-do-not-send="true" href="https://bugs.launchpad.net/nova/+bug/1312133">https://bugs.launchpad.net/nova/+bug/1312133</a><br>
</div>
<div><br>
</div>
<div>We ended up downgrading to the stock Ubuntu 12.04 qemu 1.0 packages and everything is working nicely.</div>
<div><br>
</div>
<div>I'm wondering if anyone else has run into this issue and how they dealt with it or plan to deal with it.</div>
<div><br>
</div>
<div>Also, I'm curious as to why exactly qemu 1.x to 2.0 are incompatible with each other. Is this just an Ubuntu issue? Or is this native of qemu?</div>
<div><br>
</div>
<div>Unless I'm missing something, this seems like a big deal. If we continue to use Ubuntu's OpenStack packages, we're basically stuck at 12.04 and Icehouse unless we have all users snapshot their instance and re-launch in a new cloud.</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Joe</div>
<div><br>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset> <br>
<pre wrap="">_______________________________________________
OpenStack-operators mailing list
<a class="moz-txt-link-abbreviated" href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><a class="moz-txt-link-freetext" href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a></pre>
</blockquote>
<br>
<br>
</div>
</div>
</span>
</body>
</html>