[Openstack] Unable to create snapshots.
Vijaya Bhaskar
vijayabhaskar at sparksupport.com
Mon Feb 9 04:39:55 UTC 2015
Please guys, Any ideas. I have not been able to fix the issue till now.
On Fri, Feb 6, 2015 at 1:33 PM, Vijaya Bhaskar <
vijayabhaskar at sparksupport.com> wrote:
> Hi all,
>
> I have an openstack setup with ceph as the storage backend and linuxbridge
> plugin for networking. When I try to take a snapshot of an instance that is
> running, the image is getting created, but it gets deleted as soon as the
> creation starts. When I checked the nova logs I found these:
>
> logs
> -------
>
> 2015-02-06 13:10:35.863 8030 AUDIT nova.compute.manager
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] instance snapshotting
> 2015-02-06 13:10:36.196 8030 INFO nova.compute.manager [-] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] VM Paused (Lifecycle Event)
> 2015-02-06 13:10:36.382 8030 INFO nova.compute.manager [-] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance
> has a pending task (image_snapshot). Skip.
> 2015-02-06 13:10:37.813 8030 INFO nova.compute.manager [-] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] VM Stopped (Lifecycle Event)
> 2015-02-06 13:10:37.818 8030 INFO nova.virt.libvirt.driver
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] Beginning cold snapshot process
> 2015-02-06 13:10:37.822 8030 INFO oslo.messaging._drivers.impl_rabbit
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Connecting to AMQP server on
> controller1:5672
> 2015-02-06 13:10:37.840 8030 INFO oslo.messaging._drivers.impl_rabbit
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Connected to AMQP server on
> controller1:5672
> 2015-02-06 13:10:38.042 8030 INFO nova.compute.manager [-] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance
> has a pending task (image_snapshot). Skip.
> 2015-02-06 13:10:40.647 8030 INFO nova.compute.manager [-] [instance:
> 172cd3ae-c4a2-4559-abcb-31e81132acba] During sync_power_state the instance
> has a pending task (image_pending_upload). Skip.
> 2015-02-06 13:10:46.309 8030 ERROR nova.virt.libvirt.driver
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] Error launching a defined
> domain with XML: <domain type='kvm'>
> <name>instance-00000097</name>
> <uuid>172cd3ae-c4a2-4559-abcb-31e81132acba</uuid>
> <metadata>
> <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.0
> ">
> <nova:package version="2014.2.1"/>
> <nova:name>CIRROS-TEST</nova:name>
> <nova:creationTime>2015-02-06 07:40:12</nova:creationTime>
> <nova:flavor name="micro">
> <nova:memory>256</nova:memory>
> <nova:disk>1</nova:disk>
> <nova:swap>0</nova:swap>
> <nova:ephemeral>0</nova:ephemeral>
> <nova:vcpus>1</nova:vcpus>
> </nova:flavor>
> <nova:owner/>
> <nova:root type="image" uuid="95b4a0ff-9007-440b-bcdf-0caeab529de4"/>
> </nova:instance>
> </metadata>
> <memory unit='KiB'>262144</memory>
> <currentMemory unit='KiB'>262144</currentMemory>
> <vcpu placement='static' cpuset='0'>1</vcpu>
> <sysinfo type='smbios'>
> <system>
> <entry name='manufacturer'>OpenStack Foundation</entry>
> <entry name='product'>OpenStack Nova</entry>
> <entry name='version'>2014.2.1</entry>
> <entry name='serial'>a7a9fb94-4349-9d21-ef9f-40fa6edf982f</entry>
> <entry name='uuid'>172cd3ae-c4a2-4559-abcb-31e81132acba</entry>
> </system>
> </sysinfo>
> <os>
> <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
> <boot dev='hd'/>
> <smbios mode='sysinfo'/>
> </os>
> <features>
> <acpi/>
> <apic/>
> </features>
> <cpu>
> <topology sockets='1' cores='1' threads='1'/>
> </cpu>
> <clock offset='utc'>
> <timer name='pit' tickpolicy='delay'/>
> <timer name='rtc' tickpolicy='catchup'/>
> <timer name='hpet' present='no'/>
> </clock>
> <on_poweroff>destroy</on_poweroff>
> <on_reboot>restart</on_reboot>
> <on_crash>destroy</on_crash>
> <devices>
> <emulator>/usr/bin/qemu-system-x86_64</emulator>
> <disk type='network' device='disk'>
> <driver name='qemu' type='raw' cache='none'/>
> <auth username='cinder'>
> <secret type='ceph' uuid='a3c760c0-536b-4be7-97b8-d59b0948c050'/>
> </auth>
> <source protocol='rbd'
> name='vms/172cd3ae-c4a2-4559-abcb-31e81132acba_disk'>
> <host name='10.0.11.11' port='6789'/>
> <host name='10.0.11.12' port='6789'/>
> <host name='10.0.11.13' port='6789'/>
> </source>
> <target dev='vda' bus='virtio'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x04'
> function='0x0'/>
> </disk>
> <controller type='usb' index='0'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x01'
> function='0x2'/>
> </controller>
> <controller type='pci' index='0' model='pci-root'/>
> <interface type='bridge'>
> <mac address='fa:16:3e:48:11:dd'/>
> <source bridge='brq6913fc1c-69'/>
> <target dev='tape3360180-4d'/>
> <model type='rtl8139'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x03'
> function='0x0'/>
> </interface>
> <serial type='file'>
> <source
> path='/var/lib/nova/instances/172cd3ae-c4a2-4559-abcb-31e81132acba/console.log'/>
> <target port='0'/>
> </serial>
> <serial type='pty'>
> <target port='1'/>
> </serial>
> <console type='file'>
> <source
> path='/var/lib/nova/instances/172cd3ae-c4a2-4559-abcb-31e81132acba/console.log'/>
> <target type='serial' port='0'/>
> </console>
> <input type='tablet' bus='usb'/>
> <input type='mouse' bus='ps2'/>
> <input type='keyboard' bus='ps2'/>
> <graphics type='vnc' port='-1' autoport='yes' listen='10.0.11.17'
> keymap='en-us'>
> <listen type='address' address='10.0.11.17'/>
> </graphics>
> <video>
> <model type='cirrus' vram='9216' heads='1'/>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x02'
> function='0x0'/>
> </video>
> <memballoon model='virtio'>
> <address type='pci' domain='0x0000' bus='0x00' slot='0x05'
> function='0x0'/>
> <stats period='10'/>
> </memballoon>
> </devices>
> </domain>
>
> 2015-02-06 13:10:46.935 8030 INFO nova.scheduler.client.report
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 None] Compute_service record
> updated for ('compute', 'compute')
> 2015-02-06 13:10:46.939 8030 ERROR oslo.messaging.rpc.dispatcher
> [req-b7effc42-8f02-433e-9500-f068b2a827b4 ] Exception during message
> handling: Cannot get interface MTU on 'brq6913fc1c-69': No such device
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher Traceback
> (most recent call last):
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 134, in _dispatch_and_reply
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> incoming.message))
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 177, in _dispatch
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> return self._do_dispatch(endpoint, method, ctxt, args)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line
> 123, in _do_dispatch
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> result = getattr(endpoint, method)(ctxt, **new_args)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> payload)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 82, in __exit__
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> return f(self, context, *args, **kw)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 303, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher pass
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 82, in __exit__
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 289, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> return function(self, context, *args, **kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 331, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> kwargs['instance'], e, sys.exc_info())
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 82, in __exit__
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 319, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> return function(self, context, *args, **kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 379, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> instance=instance)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 82, in __exit__
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 369, in
> decorated_function
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> *args, **kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3027, in
> snapshot_instance
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> task_states.IMAGE_SNAPSHOT)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3058, in
> _snapshot_instance
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> update_task_state)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 1733,
> in snapshot
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> new_dom = self._create_domain(domain=virt_dom)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4338,
> in _create_domain
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> LOG.error(err)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line
> 82, in __exit__
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> six.reraise(self.type_, self.value, self.tb)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 4329,
> in _create_domain
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> domain.createWithFlags(launch_flags)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> result = proxy_call(self._autowrap, f, *args, **kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in
> proxy_call
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher rv =
> execute(f,*args,**kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher rv =
> meth(*args,**kwargs)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher File
> "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher if
> ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed',
> dom=self)
> 2015-02-06 13:10:46.939 8030 TRACE oslo.messaging.rpc.dispatcher
> libvirtError: Cannot get interface MTU on 'brq6913fc1c-69': No such device
>
> The issue is that nova is trying to freeze the instance before snapshot(I
> think), but unable to resume the instance due to missing bridge , and the
> snapshot image is deleted automatically. However, the instances can
> suspend, soft reboot and hard reboot successfully.
>
> Please help.
>
--
Systems Engineer
SparkSupport
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20150209/15604b97/attachment.html>
More information about the Openstack
mailing list