- Create a VM
- Attach a Cinder Volume.  This will create a new Storage Controller.
- Start writing data to the new disk.  We used FIO.  Let FIO run until it starts reporting IOPS data.
- Live migrate the VM
- The VM will migrate and FIO will continue to run.  FIO will show 0 or no IOPS
- lspci output will show that the new storage device is in error.
- The VM will remain in this state until the user issues a hard reset command
- Once the VM is rebooted, the issue can not be reproduced until another device is attached

This also happens if we attach a new VNIC.

Prior to migration, this is what we see: 

root@test-vm-cd:~# cat before_migration_lspci.txt | grep block

05:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01)

      Subsystem: Red Hat, Inc. Virtio block device

08:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev 01)

      Subsystem: Red Hat, Inc. Virtio block device

 Post migration we see this:

08:00.0 SCSI storage controller: Red Hat, Inc. Virtio block device (rev ff) (prog-if ff)

        !!! Unknown header type 7f

        Kernel driver in use: virtio-pci

The original SCSI controller is fine.  Outside of the above error, we don't see any other errors.  From Openstack's perspective, the migration went fine.  Any suggestions on where to look? Has anyone else seen this? 

Thanks,

Chris