[openstack-dev] [nova] [cinder] Issue with live migration of instances with encrypted volumes

Lee Yarwood lyarwood at redhat.com
Wed Nov 2 10:43:44 UTC 2016


On 02-11-16 08:55:08, Carlton, Paul (Cloud Services) wrote:
> Lee
> 
> I see this in a multiple node devstack without shared storage, although that shouldn't be relevant
> 
> I do a live migration of an instance
> 
> I then hard reboot it
> 
> I you are not seeing the same outcome I'll look at this again

Apologies if I'm not being clear here Paul but I'm asking if we can't
fix the hard reboot issue directly instead of reverting the serial
console fix. Given that you actually need the serial console fix to
avoid calling connect_volume multiple times on the destination host. 

Lee

> From: Lee Yarwood <lyarwood at redhat.com>
> Sent: 02 November 2016 08:17:35
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [nova] [cinder] Issue with live migration of instances with encrypted volumes
> 
> On 01-11-16 15:22:57, Carlton, Paul (Cloud Services) wrote:
> > Lee
> >
> > That change is in my test version or was till I reverted it with https://review.openstack.org/#/c/391418,
> >
> > If you live migrate with the change you mentioned the instance goes to error when you try to hard reboot
> 
> Hey Paul,
> 
> I can't see a bug referenced by the revert above, have you looked into
> why this is happening and if a full revert is really required? It might
> be easier to fix this corner case, leaving the new method of fetching
> the domain XML in post_live_migration_at_destination and thus working
> around your issue.
> 
> Lee
> 
> > From: Lee Yarwood <lyarwood at redhat.com>
> > Sent: 01 November 2016 14:58:58
> > To: OpenStack Development Mailing List (not for usage questions)
> > Subject: Re: [openstack-dev] [nova] [cinder] Issue with live migration of instances with encrypted volumes
> >
> > On 01-11-16 12:02:55, Carlton, Paul (Cloud Services) wrote:
> > > Daniel
> > >
> > > Yes, thanks, but the thing is this does not occur with regular volumes!
> > > The process seems to be you need to connect the volume then the encryptor.
> > > In pre migration at the destination I connect the volume and then setup the encryptor and that works fine, but in post migration
> > > at destination it rebuilds the instance xml and defines the vm which calls _get_guest_storage_config which does another call to
> > > connect_volume.  This seems redundant to me, because it is already connected,
> > > but it works for normal volumes and if I bypass it for encrypted volumes
> > > it just fails with the same error when the same function is
> > > called as part of a subsequent hard reboot.
> >
> > Try rebasing on the following change that reworked
> > post_live_migration_at_destination to fetch the domain XML from libvirt
> > instead of asking Nova to rebuild it :
> >
> > libvirt: fix serial console not correctly defined after live-migration
> > https://review.openstack.org/#/c/356335/
> >
> > I think you've highlighted that this caused issues with hard rebooting
> > elsewhere right?
> >
> > Lee
> >
> > > From: Daniel P. Berrange <berrange at redhat.com>
> > > Sent: 01 November 2016 11:29:51
> > > To: OpenStack Development Mailing List (not for usage questions)
> > > Subject: Re: [openstack-dev] [nova] [cinder] Issue with live migration of instances with encrypted volumes
> > >
> > > On Tue, Nov 01, 2016 at 11:22:25AM +0000, Carlton, Paul (Cloud Services) wrote:
> > > > I'm working on a bug https://bugs.launchpad.net/nova/+bug/1633033 with the live migration of
> > > >
> > > > instances with encrypted volumes. I've submitted a work in progress version of a patch
> > > >
> > > > https://review.openstack.org/#/c/389608 but I can't overcome an issue with an iscsi command
> > > >
> > > > failure that only occurs for encrypted volumes during the post migration processing, see
> > > >
> > > > http://paste.openstack.org/show/587535/
> > > >
> > > >
> > > > Does anyone have any thoughts on how to proceed with this issue?
> > >
> > > No particular ideas, but I wanted to point out that the scsi_id command
> > > shown in that stack trace has a device path that points to the raw
> > > iSCSI LUN, not to the dm-crypt overlay. So it looks like you're hitting
> > > a failure before you get the encryption part, so encryption might be
> > > unrelated.
> 
> --
> Lee Yarwood
> Senior Software Engineer
> Red Hat
> 
> PGP : A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672 2D76
> 
> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

> __________________________________________________________________________
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request at lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Lee Yarwood
Senior Software Engineer
Red Hat

PGP : A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672 2D76



More information about the OpenStack-dev mailing list