[Openstack] Error during live migration

Guilherme Russi luisguilherme.cr at gmail.com
Mon Aug 19 10:44:41 UTC 2013


Here is my permissions inside the folder:


CN-01:
root at caos:/var/lib/nova# ls -la .ssh/
total 24
drwx------  2 nova nova   89 Ago 15 14:54 .
drwxr-xr-x 10 nova  129 4096 Ago 15 10:39 ..
-rw-------  1 nova nova  599 Ago 15 14:54 authorized_keys
-rw-------  1 nova nova   25 Ago 15 10:39 config
-rw-------  1 nova nova  668 Ago 15 10:20 id_dsa
-rw-------  1 nova nova  599 Ago 15 10:20 id_dsa.pub
-rw-------  1 nova nova  444 Ago 15 10:21 known_hosts


CN-02:
root at tiresias:/var/lib/nova# ls -la .ssh/
total 16
drwx------  2 nova nova  76 Ago 15 14:54 .
drwxr-xr-x 10 nova  129 138 Ago 15 10:39 ..
-rw-------  1 nova nova 599 Ago 15 14:54 authorized_keys
-rw-------  1 nova nova  25 Ago 15 10:39 config
-rw-------  1 nova nova 599 Ago 15 10:22 id_dsa.pub
-rw-------  1 nova nova 222 Ago 15 12:25 known_hosts

And the folder permission:
drwx------  2 nova nova     89 Ago 15 14:54 .ssh

Regards.

Guilherme.


2013/8/17 Razique Mahroua <razique.mahroua at gmail.com>

> can you show me the permissions inside?
> BTW, no need to give full 777 to the directory, as long as it belongs to
> the user nova.
> What about the rights in /var/lib/nova?
>
> *Razique Mahroua** - **Nuage & Co*
> razique.mahroua at gmail.com
> Tel : +33 9 72 37 94 15
>
>
> Le 16 août 2013 à 14:53, Guilherme Russi <luisguilherme.cr at gmail.com> a
> écrit :
>
> Hello Razique,
>
>  Here is my directory configs:
>
> drwxrwxrwx  2 nova nova     76 Ago 15 14:54 .ssh
>
> I made chmod a+xwr at the folder at all computers, I made it because I
> couldn't find the reason I was no allowed to migrate, can it be like that?
>
> Regards.
>
> Guilherme.
>
>
> 2013/8/16 Razique Mahroua <razique.mahroua at gmail.com>
>
>> Ok now, make sure on every CN the directory /var/lib/nova/.ssh belongs to
>> the user nova
>> $ chown -R nova.nova /var/lib/nova.ssh
>>
>> and it should work :)
>>
>> *Razique Mahroua** - **Nuage & Co*
>> razique.mahroua at gmail.com
>> Tel : +33 9 72 37 94 15
>>
>> <NUAGECO-LOGO-Fblan_petit.jpg>
>>
>> Le 15 août 2013 à 16:21, Guilherme Russi <luisguilherme.cr at gmail.com> a
>> écrit :
>>
>>  Right, I've done all again, and now things starting change but, I'm
>> getting this now:
>>
>> 2013-08-15 11:18:05.705 ERROR nova.compute.manager
>> [req-dbbe3889-acd0-4c99-b22d-68c7005901a3 c402785616534f2096b34ce132b7d3f2
>> d532a4fc2e9e4b5f83b6dec7085237e5] [instance:
>> 53f0a8ff-cd3b-4ddc-be9c-76655e8b8354] Unexpected error while running
>> command.
>> Stderr: 'Bad owner or permissions on /var/lib/nova/.ssh/config\r\n'.
>> Setting instance vm_state to ERROR
>> 2013-08-15 11:18:06.198 ERROR nova.openstack.common.rpc.amqp
>> [req-dbbe3889-acd0-4c99-b22d-68c7005901a3 c402785616534f2096b34ce132b7d3f2
>> d532a4fc2e9e4b5f83b6dec7085237e5] Exception during message handling
>>
>>
>> My config file content:
>>
>> StrictHostKeyChecking no
>>
>> and its properties:
>>
>> -rw-rw----  1 nova nova  25 Ago 15 10:39 config
>>
>> What Am I missing?
>>
>> Regards.
>>
>> Guilherme.
>>
>>
>>
>>
>> 2013/8/15 Guilherme Russi <luisguilherme.cr at gmail.com>
>>
>>> Ok, let me write down what I've done:
>>>
>>> #1 - Logged with nova's user at my first CN and generate a key with
>>> ssh-keygen -t dsa at the /var/lib/nova/.ssh location;
>>> #2 - Copied the id_dsa.pub from my first CN to my second CN at the
>>> /var/lib/nova/.ssh location;
>>> #3 - I'm sure I don't have te directive "AllowUsers" in my
>>> /etc/ssh/sshd_config
>>> #4 - At my two CN I have inside my /etc/passwd,
>>> nova:x:123:131::/var/lib/nova:/bin/sh
>>>
>>> Another questions, do* *I need to copy the .pub key to my controller
>>> node too? And, do I need to create one key to each compute node and copy
>>> the .pub to another one, or just create one key and copy the .pub?
>>>
>>> I'm making some mass with the key's thing :(
>>>
>>> Thank you all.
>>>
>>> Guilherme.
>>>
>>>
>>> 2013/8/15 Razique Mahroua <razique.mahroua at gmail.com>
>>>
>>>> I was so convinced we resolved your issue Guilherme
>>>> but maybe it was someone else :)
>>>> the error simply means the user "nova" from the first CN cannot connect
>>>> as the user "nova" to the second CN it needs to send the image to
>>>> Here are few checks :
>>>> #1 - Exchange all nova's public keys between all compute nodes
>>>> #2 - Make sure the connection with/ from that user is allowed (that you
>>>> don't have directive such as "AllowUsers" in /etc/ssh/sshd_config)
>>>> #3 - Make sure in /etc/passwd, the nova user has a shell
>>>>
>>>>
>>>> *Razique Mahroua** - **Nuage & Co*
>>>> razique.mahroua at gmail.com
>>>> Tel : +33 9 72 37 94 15
>>>>
>>>> <NUAGECO-LOGO-Fblan_petit.jpg>
>>>>
>>>> Le 15 août 2013 à 09:39, james <jshaw90 at gmail.com> a écrit :
>>>>
>>>> Pigging backing on the answer before me but yes, i had the same
>>>> problem. Following the setup of ssh keys for the user 'Nova' between nodes
>>>> will resolve the problem for you.
>>>>
>>>>
>>>> Kind Regards,
>>>>
>>>> James Shaw
>>>>
>>>>
>>>> On 15 August 2013 02:10, Md. Maruful Hassan <mrf.mnm at gmail.com> wrote:
>>>>
>>>>> I have not done the live-migration setup before but looking at the log
>>>>> it seems like you don't have ssh setup right between your compute nodes.
>>>>>
>>>>> 013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>> Command:* ssh 10.3.77.52 mkdir -p
>>>>> /var/lib/nova/instances/e04986a1-8f56-4dd9-9995-419e05430da3*
>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp Exit
>>>>> code: 255
>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>> Stdout: ''
>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp S*tderr:
>>>>> 'Permission denied, please try again.\r\nPermission denied, please try
>>>>> again.\r\nPermission denied (publickey,password).\r\n'*
>>>>> *
>>>>> *
>>>>> Setup ssh-key based passwordless access for user 'nova' (or whatever
>>>>> user nova process is running as ) between nodes. Then try again.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> m at ruf
>>>>>
>>>>>
>>>>> On Thu, Aug 15, 2013 at 4:52 AM, Guilherme Russi <
>>>>> luisguilherme.cr at gmail.com> wrote:
>>>>>
>>>>>> Hello guys,
>>>>>>
>>>>>>  I've been facing an error with live migration since last week and I
>>>>>> wonder why I can't migrate my instances. I'm using the Grizzly release and
>>>>>> I got this error in my both compute nodes:
>>>>>>
>>>>>>
>>>>>> 2013-08-14 15:49:47.788 ERROR nova.compute.manager
>>>>>> [req-1fad8da2-2682-48a4-a390-64cb00036568 c402785616534f2096b34ce132b7d3f2
>>>>>> d532a4fc2e9e4b5f83b6dec7085237e5] [instance:
>>>>>> e04986a1-8f56-4dd9-9995-419e05430da3] Unexpected error while running
>>>>>> command.
>>>>>> Stderr: 'Permission denied, please try again.\r\nPermission denied,
>>>>>> please try again.\r\nPermission denied (publickey,password).\r\n'. Setting
>>>>>> instance vm_state to ERROR
>>>>>> 2013-08-14 15:49:48.294 ERROR nova.openstack.common.rpc.amqp
>>>>>> [req-1fad8da2-2682-48a4-a390-64cb00036568 c402785616534f2096b34ce132b7d3f2
>>>>>> d532a4fc2e9e4b5f83b6dec7085237e5] Exception during message handling
>>>>>>
>>>>>>
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> Traceback (most recent call last):
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/amqp.py",
>>>>>> line 430, in _process_data
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> rval = self.proxy.dispatch(ctxt, version, method, **args)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File
>>>>>> "/usr/lib/python2.7/dist-packages/nova/openstack/common/rpc/dispatcher.py",
>>>>>> line 133, in dispatch
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> return getattr(proxyobj, method)(ctxt, **kwargs)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 117, in
>>>>>> wrapped
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> temp_level, payload)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> self.gen.next()
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 94, in
>>>>>> wrapped
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> return f(self, context, *args, **kw)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 209,
>>>>>> in decorated_function
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> pass
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> self.gen.next()
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 195,
>>>>>> in decorated_function
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> return function(self, context, *args, **kwargs)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 260,
>>>>>> in decorated_function
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> function(self, context, *args, **kwargs)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 237,
>>>>>> in decorated_function
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> e, sys.exc_info())
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> self.gen.next()
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 224,
>>>>>> in decorated_function
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> return function(self, context, *args, **kwargs)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2354,
>>>>>> in resize_instance
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> block_device_info)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
>>>>>> 3467, in migrate_disk_and_power_off
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> inst_base_resize)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> self.gen.next()
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line
>>>>>> 3444, in migrate_disk_and_power_off
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> utils.execute('ssh', dest, 'mkdir', '-p', inst_base)
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> File "/usr/lib/python2.7/dist-packages/nova/utils.py", line 239, in execute
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> cmd=' '.join(cmd))
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> ProcessExecutionError: Unexpected error while running command.
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> Command: ssh 10.3.77.52 mkdir -p
>>>>>> /var/lib/nova/instances/e04986a1-8f56-4dd9-9995-419e05430da3
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> Exit code: 255
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> Stdout: ''
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>> Stderr: 'Permission denied, please try again.\r\nPermission denied, please
>>>>>> try again.\r\nPermission denied (publickey,password).\r\n'
>>>>>> 2013-08-14 15:49:48.294 1700 TRACE nova.openstack.common.rpc.amqp
>>>>>>
>>>>>> Can anybody help me?
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> Guilherme.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Mailing list:
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>> Post to     : openstack at lists.openstack.org
>>>>>> Unsubscribe :
>>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list:
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>> Post to     : openstack at lists.openstack.org
>>>>> Unsubscribe :
>>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list:
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>> Post to     : openstack at lists.openstack.org
>>>> Unsubscribe :
>>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
>>>>
>>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130819/2a15ddca/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: NUAGECO-LOGO-Fblan_petit.jpg
Type: image/jpeg
Size: 10122 bytes
Desc: not available
URL: <http://lists.openstack.org/pipermail/openstack/attachments/20130819/2a15ddca/attachment.jpg>


More information about the Openstack mailing list