On 06/06, Colby Walsworth wrote:

Hey Everyone,

We just upgraded from Wallaby to Xena. We use Ceph as our volume/image backend. All instances from before the upgrade seem to be unable to be shelved/unshelved or live migrated (have not tried offline migration yet). Any new instances that are created work fine and all those tasks work fine on those. The problem seems to be with cinder:

cinder.api.v3.attachments [req-2af34fab-a4b5-43f3-bd9a-94f360201533 e28435e0a66740968c523e6376c57f68 179f701c810c425ab004548cc1f76bc9 - default default] Unable to create attachment for volume.: oslo_messaging.exceptions.MessagingTimeout: Timed out waiting for a reply to message ID 8eea6a00568a477c89ced8db17f1179d

the migrations and shelve/unshelve processes start on the hypervisor but eventually error out with the above message. Any new instances I create do not have this issue at all. I have done all the db upgrades so I don't believe this is a db record update issue.

Any ideas of what might be causing this?

Thanks,

Colby

Hi,

Do you have DEBUG log levels for the request? It would help if we could have the logs from cinder-api and cinder-volume for the request.

I assume that the cinder-volume service doesn't appear as down in `cinder service-list`.

Cheers, Gorka.