[Openstack-security] [Bug 1252519] Re: Live migration failed because of file permission changed
Maurice Leeflang
maurice at leeflang.net
Thu Aug 14 09:19:08 UTC 2014
I have the same problem.
I am currently trying to isolate the cause to it.
The first live migration of an instance works, but the second one (back to the first node) fails.
The ownership changed, but even manually as root I am not allowed to read the disk file.
Only when kvm closes the fd's on these files (when I suspend the instance, for example), the files can be read again. The files can still be read when the instance is resumed, even a live migration is possible again, then. The ownership after a resume is qemu again (not root), so I can understand why Barrow points that way.
It smells like some locking situation on the gluster side, but I am not able to pinpoint it to a configuration option or bug yet
I will do some more tests to see why (and how) gluster is spoiling the fun.
Please do not put the blame on some gluster bug or behaviour yet.
The fact that the ownership of instance files on shared storage changes indicates that there is still a resize or something being done (post migration), which, IMHO, is not needed in the case of a live migration with de nova instances directory on shared storage for all compute nodes participating.
--
You received this bug notification because you are a member of OpenStack
Security Group, which is subscribed to OpenStack.
https://bugs.launchpad.net/bugs/1252519
Title:
Live migration failed because of file permission changed
Status in OpenStack Compute (Nova):
Triaged
Bug description:
Openstack : Havana
OS : CentOS 6.4
Shared storage with GlusterFS : /var/lib/nova/instances mounted on glusterfs shared
Instance start up fine on node01. When live migration happen, it moved to node02 but failed with the following error
2013-11-18 16:27:37.813 9837 ERROR nova.openstack.common.periodic_task [-] Error during ComputeManager.update_available_resource: Unexpected error while running command.
Command: env LC_ALL=C LANG=C qemu-img info /var/lib/nova/instances/aa1deb40-ae1d-45e4-a37e-7b0607df372f/disk
Exit code: 1
Stdout: ''
Stderr: "qemu-img: Could not open '/var/lib/nova/instances/aa1deb40-ae1d-45e4-a37e-7b0607df372f/disk'\n"
2013-11-18 16:27:37.813 9837 TRACE nova.openstack.common.periodic_task Traceback (most recent call last):
2013-11-18 16:27:37.813 9837 TRACE nova.openstack.common.periodic_task File "/usr/lib/python2.6/site-packages/nova/openstack/common/periodic_task.py", line 180, in run_periodic_tasks
2013-11-18 16:27:37.813 9837 TRACE nova.openstack.common.periodic_task task(self, context)
The problem is with the file ownership of "console.log" and "disk".
Those file should be owned by user "qemu" and group "qemu" but after
the migration, both files are owned by root
drwxr-xr-x 2 nova nova 53 Nov 18 13:40 .
drwxr-xr-x 6 nova nova 110 Nov 18 13:43 ..
-rw-rw---- 1 root root 1546 Nov 18 13:43 console.log
-rw-r--r-- 1 root root 12058624 Nov 18 13:42 disk
-rw-r--r-- 1 nova nova 1569 Nov 18 13:42 libvirt.xml
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1252519/+subscriptions
More information about the Openstack-security
mailing list