<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:564534996;
mso-list-type:hybrid;
mso-list-template-ids:419227936 134807567 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi All,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I’m having a few issues with live migration at the moment. This is an openstack train deployed with rdo packages.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]> when migrating some VMs the migration succeeds but the VM doesn’t move and the following error can be found in the nova logs with nothing that seems to correspond in the neutron logs:<br>
2022-09-30 11:04:04.069 716669 INFO nova.compute.manager [-] [instance: c10d1d9e-2baf-4ab5-ba3a-2f60f521aea6] Took 76.49 seconds for pre_live_migration on destination host hv548.nubes.rl.<br>
ac.uk.<br>
2022-09-30 11:04:04.720 716669 ERROR nova.virt.libvirt.driver [-] [instance: c10d1d9e-2baf-4ab5-ba3a-2f60f521aea6] Live Migration failure: 'fa:ca:ad:1f:71:41'<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-18.0pt;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">2.<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>When migrating some VMs the migration is marked as complete and the VM is moved from the source to the destination hypervisor but nova keeps the migrating state and doesn’t update the host or the node fields. The following error is found
in nova with no corresponding error that I can find in neutron. The config is the same across all hypervisors.<o:p></o:p></p>
<p class="MsoListParagraph">2022-09-13 10:37:23.709 12387 ERROR nova.network.neutronv2.api [req-ee51a8b4-911c-4f92-860b-9d6a7a24c786 27be800439c244c1bd742ff056cf700c 4de86830e89b4a46b590536571b6ccd4 - 38372510d9bb4ac7916178b062d387de 38372510d9bb4ac7916178b062d387de]
Neutron client was not able to generate a valid admin token, please verify Neutron admin credential located in nova.conf: neutronclient.common.exceptions.Unauthorized: 401-{'error': {'message': 'The request you have made requires authentication.', 'code':
401, 'title': 'Unauthorized'}}<o:p></o:p></p>
<p class="MsoListParagraph">2022-09-13 10:37:23.845 12387 ERROR oslo_messaging.rpc.server [req-ee51a8b4-911c-4f92-860b-9d6a7a24c786 27be800439c244c1bd742ff056cf700c 4de86830e89b4a46b590536571b6ccd4 - 38372510d9bb4ac7916178b062d387de 38372510d9bb4ac7916178b062d387de]
Exception during message handling: nova.exception.NeutronAdminCredentialConfigurationInvalid: Networking client is experiencing an unauthorized exception.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Regards<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">Alexander Dibbo – Cloud Architect / Cloud Operations Group Leader<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">For STFC Cloud Documentation visit
<a href="https://stfc-cloud-docs.readthedocs.io/"><span style="color:#0563C1">https://stfc-cloud-docs.readthedocs.io</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To raise a support ticket with the cloud team please email
<a href="mailto:cloud-support@stfc.ac.uk"><span style="color:#0563C1">cloud-support@stfc.ac.uk</span></a>
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To receive notifications about the service please subscribe to our mailing list at:
<a href="https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD"><span style="color:#0563C1">https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=STFC-CLOUD</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-GB">To receive fast notifications or to discuss usage of the cloud please join our Slack:
<a href="https://stfc-cloud.slack.com/"><span style="color:#0563C1">https://stfc-cloud.slack.com/</span></a>
<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p><span style="font-size: 6pt;">This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. </span></p></body>
</html>