Open Stack

Hi,

I am running OpenStack nova 2.17.0 on Ubuntu 14.04.1 with libvirt 1.2.2
and QEMU 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.5).

When attempting to nova rescue an instance, the instance boots with
updated libvirt XML but ends up booting to the old, non-rescue disk
(second disk in XML) 80% of the time. Occasionally after several reboots
(for example, via ctrl+alt+del in VNC) I can randomly get it to boot to
the rescue disk.

The XML looks like this:

    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback'/>
      <source file='/var/lib/nova/instances/INSTANCE_ID/disk.rescue'/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
function='0x0'/>
    </disk>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='writeback'/>
      <source file='/var/lib/nova/instances/INSTANCE_ID/disk'/>
      <target dev='vdb' bus='virtio'/>
      <alias name='virtio-disk1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
function='0x0'/>
    </disk>

And the command line:

/usr/bin/qemu-system-x86_64 -name instance-00000a65 -S -machine
pc-i440fx-trusty,accel=kvm,usb=off -m 512 -realtime mlock=off -smp
1,sockets=1,cores=1,threads=1 -uuid e9019a9b-03de-47e9-b372-673305ea5c66
-smbios type=1,manufacturer=OpenStack Foundation,product=OpenStack
Nova,version=2014.1.2,serial=44454c4c-3700-1056-8053-b6c04f504e31,uuid=e9019a9b-03de-47e9-b372-673305ea5c66
-no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/instance-00000a65.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet
-no-shutdown -boot strict=on -device
piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -drive
file=/var/lib/nova/instances/e9019a9b-03de-47e9-b372-673305ea5c66/disk.rescue,if=none,id=drive-virtio-disk0,format=qcow2,cache=writeback
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1
-drive
file=/var/lib/nova/instances/e9019a9b-03de-47e9-b372-673305ea5c66/disk,if=none,id=drive-virtio-disk1,format=qcow2,cache=writeback
-device
virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk1,id=virtio-disk1
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=fa:16:3e:ac:13:08,bus=pci.0,addr=0x3
-chardev
file,id=charserial0,path=/var/lib/nova/instances/e9019a9b-03de-47e9-b372-673305ea5c66/console.log
-device isa-serial,chardev=charserial0,id=serial0 -chardev
pty,id=charserial1 -device isa-serial,chardev=charserial1,id=serial1
-device usb-tablet,id=input0 -vnc 127.0.0.1:0 -k en-us -device
cirrus-vga,id=video0,bus=pci.0,addr=0x2 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

So everything looks like it should boot to the rescue disk (strict boot
on and boot index set for the desired device), yet usually it still ends
up booting to the old OS. I tried manually specifying boot index on
second device to 2 and it still fails.

Any ideas what's going on with this? All the packages are also
up-to-date on the host node; in this case the VM has Ubuntu 14.04 OS but
I don't think that affects the boot order.

Thanks,
- Favyen Bastani