<div dir="ltr">We'll probably need a few more details<div><ul><li>How was this deployed?</li><li>What version of Openstack are you running?</li></ul><div>Openstack is usually pretty good about failing and reverting when one step of the operation chain failed. That said, this is anecdotal but the most amount of mismatch between the VM and the DB is always around Cinder. Which kind of makes sense since it's probably volume attach + detach that are the most common. Maybe port operations after.</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 26, 2021 at 12:24 PM Aleksander Wojtal <<a href="mailto:aleksander.wojtal@tietoevry.com">aleksander.wojtal@tietoevry.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-US" style="overflow-wrap: break-word;">
<div class="gmail-m_-2942710927436140877WordSection1">
<p class="MsoNormal">Hello,<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">During our testing we perform reboot of compute during volume attached task. Result is as follows.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Volume is marked as attached to VM (also in DB)<u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Lucida Console";color:black">ceeinfra@lcm1:~> openstack volume list</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+--------------------------------------+-----------------------------------------------------------------------------+-----------+------+--------------------------------------------------------------------------------------------------------------+</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| ID | Name | Status | Size | Attached to
|</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+--------------------------------------+-----------------------------------------------------------------------------+-----------+------+--------------------------------------------------------------------------------------------------------------+</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| c52d406c-5587-445d-9da1-436ffbbe3541 | vol_neXt-488 | available | 10 |
|</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| f3e45efc-35eb-4f95-8e71-f50b5cb69028 | RebootOfComputeHostWhilePerformingVolumeOperations-Volume-0618_19_45_12_669 | in-use | 20 | Attached to <span style="background:yellow">neXt-377_VM1--RebootOfComputeHostWhilePerformingVolumeOperations--0618_19_44_16_775</span> on
/dev/vdb |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| 1bd81392-e2c9-4725-93c8-baa484924f21 | vol_neXt-488 | available | 10 |
|</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| 919db17c-0729-409c-bf59-74539edcea47 | vol_neXt-488 | available | 10 |
|</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+--------------------------------------+-----------------------------------------------------------------------------+-----------+------+--------------------------------------------------------------------------------------------------------------+<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Lucida Console";color:black"><u></u> <u></u></span></p>
<p class="MsoNormal">From VM perspective too.<u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Lucida Console";color:black">ceeinfra@lcm1:~> openstack server show neXt-377_VM1--RebootOfComputeHostWhilePerformingVolumeOperations--0618_19_44_16_775</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+-------------------------------------+-------------------------------------------------------------------------------------+</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| Field | Value |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+-------------------------------------+-------------------------------------------------------------------------------------+</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-DCF:diskConfig | MANUAL |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-AZ:availability_zone | nova |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-SRV-ATTR:host | <a href="http://compute-0-11.k2.ericsson.se" target="_blank">compute-0-11.k2.ericsson.se</a> |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-SRV-ATTR:hypervisor_hostname | <a href="http://compute-0-11.k2.ericsson.se" target="_blank">compute-0-11.k2.ericsson.se</a> |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-SRV-ATTR:instance_name | <span style="background:yellow">instance-00001edc</span> |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-STS:power_state | Running |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-STS:task_state | None |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-EXT-STS:vm_state | active |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-SRV-USG:launched_at | 2021-06-18T14:14:58.000000 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| OS-SRV-USG:terminated_at | None |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| accessIPv4 | |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| accessIPv6 | |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| addresses | NovaController-Network-0618_19_44_16_852=10.0.0.5 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| config_drive | True |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| created | 2021-06-18T14:14:19Z |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| flavor | m1.small (2) |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| hostId | 5078bfd4fabd8b2e7c9f4554b3ac08d536ceebb5f393db4d38c8ff60 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| id | 95ed49ed-b3f8-46ae-80a0-544b939c50b7 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| image | JCAT_Common_CirrOS_i386 (b71f3fbd-59f1-4c29-a296-5065fe0406f7) |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| key_name | None |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| name | neXt-377_VM1--RebootOfComputeHostWhilePerformingVolumeOperations--0618_19_44_16_775 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| progress | 0 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| project_id | b63725c8ebaf43ebabbed497f41eb71c |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| properties | ha-policy='managed-on-host' |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| scheduler_hints | {} |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| security_groups | name='default' |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| status | ACTIVE |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| updated | 2021-06-18T14:22:07Z |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| user_id | f490f07b813748698e56ca7641d46a72 |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">| volumes_attached | id='f3e45efc-35eb-4f95-8e71-f50b5cb69028' |</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">+-------------------------------------+-------------------------------------------------------------------------------------+</span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">From Libvirt perspective, volume is not attached to VM<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Lucida Console";color:black">compute-0-11:/var/log # virsh dumpxml <span style="background:yellow">instance-00001edc</span></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"><domain type='kvm' id='1'></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"> <name>instance-00001edc</name></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"> <uuid>95ed49ed-b3f8-46ae-80a0-544b939c50b7</uuid></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">(…)</span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"> <devices></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"> <emulator>/usr/bin/qemu-system-x86_64</emulator></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <disk type='file' device='disk'></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <driver name='qemu' type='qcow2' cache='directsync'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <source file='/var/lib/nova/instances/95ed49ed-b3f8-46ae-80a0-544b939c50b7/disk' index='2'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <backingStore type='file' index='3'></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <format type='raw'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <source file='/var/lib/nova/instances/_base/af82cd8216adb002916bb9e422a3fbb637022ec6'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <backingStore/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> </backingStore></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <target dev='vda' bus='virtio'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <alias name='virtio-disk0'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> </disk></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <disk type='file' device='cdrom'></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <driver name='qemu' type='raw' cache='directsync'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <source file='/var/lib/nova/instances/95ed49ed-b3f8-46ae-80a0-544b939c50b7/disk.config' index='1'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <backingStore/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <target dev='hdd' bus='ide'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <readonly/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <alias name='ide0-1-1'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> <address type='drive' controller='0' bus='1' target='0' unit='1'/></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black;background:yellow"> </disk></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black"> <controller type='usb' index='0' model='piix3-uhci'></span><span style="font-size:10pt;font-family:Arial,sans-serif;color:black"><br>
</span><span style="font-size:9pt;font-family:"Lucida Console";color:black">(…)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:9pt;font-family:"Lucida Console";color:black"></domain></span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">System does not provide any indication of the problem. There should be some kind of indication to user that volume attachment was not completed.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><span style="font-size:10pt;font-family:Arial,sans-serif;color:rgb(80,80,80)">Pozdrawiam / Best regards,
<br>
<b><br>
Aleksander Wojtal</b>, Junior Software Engineer <br>
<br>
<b>TietoEVRY<br>
BU Telco Cloud Infra & Applications</b><br>
email <a href="mailto:aleksander.wojtal@tietoevry.com" target="_blank">aleksander.wojtal@tietoevry.com</a><br>
al. Piastów 30, 71-064 Szczecin, <a href="http://tietoevry.com" target="_blank">tietoevry.com</a> </span><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(80,80,80)"><br>
<br>
</span><span lang="PL" style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(80,80,80)"><img width="119" height="48" style="width: 1.2395in; height: 0.5in;" id="gmail-m_-2942710927436140877Picture_x0020_1" src="cid:17ae4be7bfc4cff311"></span><span style="font-size:10.5pt;font-family:Arial,sans-serif;color:rgb(80,80,80)"><br>
<br>
</span><span style="font-size:8pt;font-family:Arial,sans-serif;color:rgb(80,80,80)">Please note: The information contained in this message may be legally privileged,
<br>
confidential and protected from disclosure. If you received this in error, please notify
<br>
the sender immediately and delete the message from your computer. Thank you.</span><u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</blockquote></div>