<div dir="ltr">Please guys,  Any ideas. I have not been able to fix the issue till now.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 6, 2015 at 1:33 PM, Vijaya Bhaskar <span dir="ltr"><<a href="mailto:vijayabhaskar@sparksupport.com" target="_blank">vijayabhaskar@sparksupport.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi all,<br><br></div>I have an openstack setup with ceph as the storage backend and linuxbridge plugin for networking. When I try to take a snapshot of an instance that is running, the image is getting created, but it gets deleted as soon as the creation starts. When I checked the nova logs I found these:<br><br></div>logs<br>-------<br><br>2015-02-06 13:10:35.863 8030 AUDIT nova.compute.manager [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] instance snapshotting<br>2015-02-06 13:10:36.196 8030 INFO nova.compute.manager [-] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] VM Paused (Lifecycle Event)<br>2015-02-06 13:10:36.382 8030 INFO nova.compute.manager [-] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance has a pending task (image_snapshot). Skip.<br>2015-02-06 13:10:37.813 8030 INFO nova.compute.manager [-] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] VM Stopped (Lifecycle Event)<br>2015-02-06 13:10:37.818 8030 INFO nova.virt.libvirt.driver [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] Beginning cold snapshot process<br>2015-02-06 13:10:37.822 8030 INFO oslo.messaging._drivers.impl_rabbit [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Connecting to AMQP server on controller1:5672<br>2015-02-06 13:10:37.840 8030 INFO oslo.messaging._drivers.impl_rabbit [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Connected to AMQP server on controller1:5672<br>2015-02-06 13:10:38.042 8030 INFO nova.compute.manager [-] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance has a pending task (image_snapshot). Skip.<br>2015-02-06 13:10:40.647 8030 INFO nova.compute.manager [-] [instance: 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance has a pending task (image_pending_upload). Skip.<br>2015-02-06 13:10:46.309 8030 ERROR nova.virt.libvirt.driver [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] Error launching a defined domain with XML: <domain type='kvm'><br>  <name>instance-00000097</name><br>  <uuid>172cd3ae-c4a2-4559-abcb-31e81132acba</uuid><br>  <metadata><br>    <nova:instance xmlns:nova="<a href="http://openstack.org/xmlns/libvirt/nova/1.0" target="_blank">http://openstack.org/xmlns/libvirt/nova/1.0</a>"><br>      <nova:package version="2014.2.1"/><br>      <nova:name>CIRROS-TEST</nova:name><br>      <nova:creationTime>2015-02-06 07:40:12</nova:creationTime><br>      <nova:flavor name="micro"><br>        <nova:memory>256</nova:memory><br>        <nova:disk>1</nova:disk><br>        <nova:swap>0</nova:swap><br>        <nova:ephemeral>0</nova:ephemeral><br>        <nova:vcpus>1</nova:vcpus><br>      </nova:flavor><br>      <nova:owner/><br>      <nova:root type="image" uuid="95b4a0ff-9007-440b-bcdf-0caeab529de4"/><br>    </nova:instance><br>  </metadata><br>  <memory unit='KiB'>262144</memory><br>  <currentMemory unit='KiB'>262144</currentMemory><br>  <vcpu placement='static' cpuset='0'>1</vcpu><br>  <sysinfo type='smbios'><br>    <system><br>      <entry name='manufacturer'>OpenStack Foundation</entry><br>      <entry name='product'>OpenStack Nova</entry><br>      <entry name='version'>2014.2.1</entry><br>      <entry name='serial'>a7a9fb94-4349-9d21-ef9f-40fa6edf982f</entry><br>      <entry name='uuid'>172cd3ae-c4a2-4559-abcb-31e81132acba</entry><br>    </system><br>  </sysinfo><br>  <os><br>    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type><br>    <boot dev='hd'/><br>    <smbios mode='sysinfo'/><br>  </os><br>  <features><br>    <acpi/><br>    <apic/><br>  </features><br>  <cpu><br>    <topology sockets='1' cores='1' threads='1'/><br>  </cpu><br>  <clock offset='utc'><br>    <timer name='pit' tickpolicy='delay'/><br>    <timer name='rtc' tickpolicy='catchup'/><br>    <timer name='hpet' present='no'/><br>  </clock><br>  <on_poweroff>destroy</on_poweroff><br>  <on_reboot>restart</on_reboot><br>  <on_crash>destroy</on_crash><br>  <devices><br>    <emulator>/usr/bin/qemu-system-x86_64</emulator><br>    <disk type='network' device='disk'><br>      <driver name='qemu' type='raw' cache='none'/><br>      <auth username='cinder'><br>        <secret type='ceph' uuid='a3c760c0-536b-4be7-97b8-d59b0948c050'/><br>      </auth><br>      <source protocol='rbd' name='vms/172cd3ae-c4a2-4559-abcb-31e81132acba_disk'><br>        <host name='10.0.11.11' port='6789'/><br>        <host name='10.0.11.12' port='6789'/><br>        <host name='10.0.11.13' port='6789'/><br>      </source><br>      <target dev='vda' bus='virtio'/><br>      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/><br>    </disk><br>    <controller type='usb' index='0'><br>      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/><br>    </controller><br>    <controller type='pci' index='0' model='pci-root'/><br>    <interface type='bridge'><br>      <mac address='fa:16:3e:48:11:dd'/><br>      <source bridge='brq6913fc1c-69'/><br>      <target dev='tape3360180-4d'/><br>      <model type='rtl8139'/><br>      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/><br>    </interface><br>    <serial type='file'><br>      <source path='/var/lib/nova/instances/172cd3ae-c4a2-4559-abcb-31e81132acba/console.log'/><br>      <target port='0'/><br>    </serial><br>    <serial type='pty'><br>      <target port='1'/><br>    </serial><br>    <console type='file'><br>      <source path='/var/lib/nova/instances/172cd3ae-c4a2-4559-abcb-31e81132acba/console.log'/><br>      <target type='serial' port='0'/><br>    </console><br>    <input type='tablet' bus='usb'/><br>    <input type='mouse' bus='ps2'/><br>    <input type='keyboard' bus='ps2'/><br>    <graphics type='vnc' port='-1' autoport='yes' listen='10.0.11.17' keymap='en-us'><br>      <listen type='address' address='10.0.11.17'/><br>    </graphics><br>    <video><br>      <model type='cirrus' vram='9216' heads='1'/><br>      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/><br>    </video><br>    <memballoon model='virtio'><br>      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/><br>      <stats period='10'/><br>    </memballoon><br>  </devices><br></domain><br><br>2015-02-06 13:10:46.935 8030 INFO nova.scheduler.client.report [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] Compute_service record updated for ('compute', 'compute')<br>2015-02-06 13:10:46.939 8030 ERROR oslo.messaging.rpc.dispatcher [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Exception during message handling: Cannot get interface MTU on 'brq6913fc1c-69': No such device<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 134, in _dispatch_and_reply<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     incoming.message))<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 177, in _dispatch<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 123, in _do_dispatch<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     payload)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 303, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     pass<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 289, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 331, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     kwargs['instance'], e, sys.exc_info())<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 379, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     instance=instance)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 369, in decorated_function<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     *args, **kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3027, in snapshot_instance<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     task_states.IMAGE_SNAPSHOT)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3058, in _snapshot_instance<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     update_task_state)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1733, in snapshot<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     new_dom = self._create_domain(domain=virt_dom)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4338, in _create_domain<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     LOG.error(err)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 82, in __exit__<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4329, in _create_domain<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     domain.createWithFlags(launch_flags)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     result = proxy_call(self._autowrap, f, *args, **kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     rv = execute(f,*args,**kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     rv = meth(*args,**kwargs)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)<br>2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher libvirtError: Cannot get interface MTU on 'brq6913fc1c-69': No such device<br><br></div>The issue is that nova is trying to freeze the instance before snapshot(I think), but unable to resume the instance due to missing bridge , and the snapshot image is deleted automatically. However, the instances can suspend, soft reboot and hard reboot successfully.<br><br></div>Please help.<br></div>
</blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature">Systems Engineer<br>SparkSupport<br></div>
</div>