<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman";}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Courier;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1895896519;
mso-list-type:hybrid;
mso-list-template-ids:-1561549142 -1657361318 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\.\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">As a follow up on this. You can configure the host to shutdown and start up in a way that all the VM’s are shutdown and started up automatically.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">To do this you need to do a few things:<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Calibri"><span style="mso-list:Ignore">1.)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:Calibri">Ensure that nova-compute is configured to stopped before libvirt-guests. Make sure libvirt-guests is enabled.<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Calibri"><span style="mso-list:Ignore">2.)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:Calibri">Allow libvirt-guests to shutdown the vm’s that are running (I recommend avoiding suspending the vm’s. As this will lead to in vm clock sync issues): ON_SHUTDOWN=shutdown<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Calibri"><span style="mso-list:Ignore">3.)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:Calibri">Ensure that libvirt-guests is configure with: ON_BOOT=ignore<o:p></o:p></span></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="font-size:11.0pt;font-family:Calibri"><span style="mso-list:Ignore">4.)<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:11.0pt;font-family:Calibri">Set [DEFAULT] resume_guests_state_on_host_boot=true in nova.conf<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">This config will gracefully shutdown the running vm’s via a normal host shutdown, preserving the running state in nova. This will then cause nova to bring the vm’s online when nova-compute
starts on host start up. This also works with ungraceful power downs. The key is that nova needs to be the one that starts the VM’s. Because libvirt-guests will not be able to successfully start the vm’s, because neutron eeds to plug the vifs for the vms.
As long as the state of the VM is “running” inside the DB, this config will work.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">NB: if you do chassis swaps and for some reason the OS comes up in a config that no longer works, all of the VM’s will go to error. You will need to fix whatever issue prevented the vm’s
from starting and manually reset the state and start the vms.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Some examples that we have seen: vtx extensions disabled on new server. Replacement server has different cpu’s and no longer matches either the numa config or the new processors do not
have the same cpu extensions as the old processors.<o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Calibri;color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Calibri;color:black">___________________________________________________________________<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Calibri;color:black">Kris Lindgren<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Calibri;color:black">Senior Linux Systems Engineer<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Calibri;color:black">GoDaddy</span><span style="font-size:11.0pt;font-family:Calibri"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span>
</b><span style="font-family:Calibri;color:black">Adam Thurlow <thurloat@gmail.com><br>
<b>Date: </b>Wednesday, November 16, 2016 at 9:58 PM<br>
<b>To: </b>"openstack-operators@lists.openstack.org" <openstack-operators@lists.openstack.org><br>
<b>Subject: </b>Re: [Openstack-operators] sync power states and externally shut off VMs<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p>If you are interested in manually mucking around with local virsh domain states, and you don't want nova to interfere, you can just stop the local nova-compute service and it won't be doing any syncing. Once you get those instances back into their desired
state, you can restart nova-compute and it won't be any wiser.<o:p></o:p></p>
<p>You can obviously shoot yourself in the foot using this method, but I can understand in some cases that large hammers and manual virsh commands are necessary.<o:p></o:p></p>
<p class="MsoNormal" style="margin-bottom:12.0pt">Cheers!<o:p></o:p></p>
<div>
<p class="MsoNormal">On 2016-11-16 17:32, Mohammed Naser wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">Typically, you should not be managing your VMs by virsh. After a power outage, I would recommend sending a start API call to instances that are housed on that specific hypervisor<br>
<br>
Sent from my iPhone<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
On Nov 16, 2016, at 4:26 PM, Gustavo Randich <<a href="mailto:gustavo.randich@gmail.com">gustavo.randich@gmail.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal">When a VM is shutdown without using nova API (kvm process down, libvirt failed to start instance on host boot, etc.), Openstack "freezes" the shutdown power state in the DB, and then re-applies it if the VM is not started via API, e.g.:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0in">
<div>
<p class="MsoNormal"># virsh shutdown <domain><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[ sync power states -> stop instance via API ], because hypervisor rules ("power_state is always updated from hypervisor to db")<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"># virsh startup <domain><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">[ sync power states -> stop instance via API ], because database rules<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I understand this behaviour is "by design", but I'm confused about the asymmetry: if VM is shutdown without using nova API, should I not be able to start it up again without nova API?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This is a common scenario in power outages or failures external to Openstack, when VMs fail to start and we need to start them up again using virsh.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">_______________________________________________<br>
OpenStack-operators mailing list<br>
<a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><br>
<a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><o:p></o:p></p>
</div>
</blockquote>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>OpenStack-operators mailing list<o:p></o:p></pre>
<pre><a href="mailto:OpenStack-operators@lists.openstack.org">OpenStack-operators@lists.openstack.org</a><o:p></o:p></pre>
<pre><a href="http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators">http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators</a><o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
</div>
</body>
</html>