Open Stack

After moving from CentOS 7.3 to 7.4, I’ve had trouble getting live migration to work when a volume is attached.  As it turns out when a live migration takes place the libvirt driver rewrites portions of the xml definition for the destination hypervisor and gets it wrong.  Here is an example.

It turns this:

disk device="disk" type="network">
	<driver cache="writeback" name="qemu" type="raw"/>
	<auth username="cinder">
		<secret type="ceph" uuid="1a790a26-dd49-4825-8d16-3dd627cf05a9"/>
	</auth>
	<source name="cinder-volumes/volume-de400476-b68a-45a2-b04f-739313f42bef" protocol="rbd">
		<host name="172.16.128.101" port="6789"/>
		<host name="172.16.128.121" port="6789"/>
		<host name="172.16.128.130" port="6789"/>
	</source>
	<target bus="scsi" dev="sdb"/>
	<serial>de400476-b68a-45a2-b04f-739313f42bef</serial>
	<address bus="0" controller="0" target="0" type="drive" unit="1"/>
</disk>

Into this:

<disk device="disk" type="network">
	<driver cache="writeback" name="qemu" type="raw"/>
	<auth username="cinder">
		<secret type="ceph" uuid="1a790a26-dd49-4825-8d16-3dd627cf05a9"/>
	</auth>
	<source name="cinder-volumes/volume-de400476-b68a-45a2-b04f-739313f42bef" protocol="rbd">
		<host name="172.16.128.101" port="6789"/>
		<host name="172.16.128.121" port="6789"/>
		<host name="172.16.128.130" port="6789"/>
	</source>
	<target bus="scsi" dev="sdb"/>
	<serial>de400476-b68a-45a2-b04f-739313f42bef</serial>
	<address controller="0" type="drive"/>
</disk>

Note the address change, libvirt will attempt to put this undefined disk at 0:0:0 which conflicts with the root disk and causes the migration to fail.  While this may be a libvirt bug I can’t imagine anything good would come from changing the scsi address of your disks mid flight so steps should be taken in nova if rewriting the definition to preserve the address.

Here is my question, when exactly would you want to change the definition of an attached volume?