Open Stack

Hello folks,

On Thu, Jul 29, 2021 at 6:24 PM Sean Mooney <smooney at redhat.com> wrote:
>
> On Thu, 2021-07-29 at 15:28 +0000, Jiri Stransky wrote:
> > OS Migrate might be worth a look, it's an Ansible collection. It does
> > cold migration that does not require admin privileges (is runnable by
> > tenants who are sufficiently technically savvy).
> >
>
> default nova policy requrie admin right for cold migration
> https://github.com/openstack/nova/blob/master/nova/policies/migrate_server.py#L25-L36
>
> os-migrate is not useing nova migration apis its migrating the data and vms behind novas back.

I used the term "cold migration" in the generic sense (migration of
VMs while they're stopped), not in the particular "Nova cold
migration" sense. While we're not using any migration-specific APIs
(not sure if these would work *between* clouds, incl. copying Cinder
volumes), we do still interact with OpenStack only through the normal
REST APIs, we do not touch the backing MariaDB database directly. (So
i do not see our approach as migrating "behind Nova's back".)

>
>
>
> >  It can copy VMs with
> > attached volumes. If boot disk copy is desired, then the destination
> > instance will be created as boot-from-volume (the alternative option
> > being re-imaging the instance from a pre-existing Glance image in the
> > destination cloud, which is good for instances which store important
> > data only in volumes, and can be rebuilt as in `openstack server
> > rebuild`). Aside from migrating VMs, it should also help you migrate
> > prerequisites for VMs, e.g. tenant networking resources or security
> > groups.
> >
> > Perhaps even if you end up doing the migration on behalf of tenants,
> > at least you wouldn't have to do it fully by hand, and you'd have
> > something for the migration of attached volumes.
>
> os-migrate is a tool for moveing betweeen different cloud deployemnets isntead of doing a upgrade you deploy a parralel
> cloud and then you use os-migrate to move workloads.

+1.

>
> while that sound nice in practic it will break any automatin that site on top of openstack ans the workload will not
> be moved to a new cloud so so if you have deployed yoru vms with heat or use an external VNFM you willl nolonger be abel to manage the vms.

The data is copied to destination and a new VM is created using the
data copy (assuming the two different migration modes i described in
previous e-mails and in the blog). Floating IP addresses will also
differ in the destination since generally different clouds have
different FIP ranges. Whether this is a suitable approach or not
depends per use case expectations and constraints.

>
> os migrate will not be transparent to the enduser
> for example there keystone project id and user id will change.
> this is because keysotne does not allow you to choose a uuid when create user or projects
> https://docs.openstack.org/api-ref/identity/v3/index.html?expanded=create-user-detail#create-user
> https://docs.openstack.org/api-ref/identity/v3/index.html?expanded=create-project-detail#create-project

Indeed as we're recreating the resources using APIs rather than a
direct DB access, IDs in the destination are different, but on the
other hand going trough the normal REST APIs (rather than trying to
force IDs by DB editing for example) considerably lowers the risk of
creating inconsistent state of OpenStack services. Again, whether this
is suitable or not depends per use case. I would say the migration
procedure is transparent to the user (e.g. the resources are
serialized into readable and editable YAML files, which are then used
when creating resources in destination).

>
> while some resouces can be recarted with the same external ids resouces like flaover obviously need admin privladges.
> nova lso caches the version of the imave and flavor the server was booted with.
>
> if you have modifed the flaovr or image extra spces since the vms were booted the vms that are create on the destination
> may be diffeente then the souce and that may or may not break the newly created vm.

Indeed this is something to watch out for if the src/dst flavors or
images happen to differ somehow. I would not say differences are
entirely disqualifying the migration, but care needs to be taken that
workloads' requirements are still satisfied.

>
> so while it might be useful in limited cases you shoudl be aware the way it works is generally not supprotable in public cloud or by normal users
> its a tool that a cloud operator can use.

As i wrote earlier, the user needs to be somewhat technically savvy
(most importantly, be able to work with Ansible and have at least
high-level understanding of OpenStack), but admin privileges are not
required unless you are trying to migrate resources that only admins
can create (public images, users, projects, flavors, ...). We tested
it with a cloud where we didn't have admin access.

>
> as a reulst it is portentially more dangourous then just a normal upgades as its more like that the vm abi and resouce uuids will change.

Upgrade is less invasive on the level of individual resources, but
migration allows you to scope risk better. In other words: when
upgrading, the entire cloud is at some level of risk. When migrating
tenant content with tenant credentials, only one tenant in the source
and one tenant in the destination are being affected at a given time.
Which approach is considered less dangerous will depend on use cases.

> im also not sure if it will work with ironic i suspect not.

I would hope at least the re-imaging mode of migration would work, but
we did not test migrating instances deployed on Ironic nodes, so i'm
also not sure.

>
> it also cannot handel all resouces for all service for example it cannot tansfer octaivia loadblanceser, barbical secreats, swift objecst or trove db instance or heat stacks to name but a few
> service that are not supported.

The supported resources are: users, projects, networks, subnets,
routers, security groups, images, nova flavors and key pairs, VMs +
attached volumes + FIP creation.


Have a good day,

Jirka