Adding the response to this thread, as I replied in the wrong one. Sorry for the confusion.

Hello Sylvain Bauza and Sean Mooney,

The patch pointed out in [1] is doing exactly what you said that we should not do. I mean, changing the AZ in the request spec of the virtual machine (VM) that the user created. The patch we propose in [2] is intended to avoid exactly that.

As pointed out by Florian, the configuration "cross_az_attach" is a constraint of the cloud environment, and as such it should be considered when selecting hosts to execute VMs migrations. Therefore, for those situations, and only for those (cross_az_attach=False), we send the current AZ of the VM to placement, to enable it (placement) to filter out hosts that are not from the current AZ of the VM.

Looking at the patch suggested by you in [1], and later testing it, we can confirm that the problem is still there in main/upstream. This happens because the patch in [1] is only addressing the cases when the VM is created based on volumes, and then it sets the AZ of the volumes in the request spec of the VM. That is why everything works for the setups where cross_az_attach=False. However, if we create a VM based on an image, and then it (Nova) creates a new volume in Cinder, the AZ is not set in the request spec (but it is used to execute the first call to placement to select the hosts); thus, the issues described in [2] can still happen.

Anyways, the proposal presented in [2] is simpler and works nicely. We can discuss it further in the patchset then, if you guys think it is worth it.

[1] https://review.opendev.org/c/openstack/nova/+/469675/12/nova/compute/api.py#1173
[2] https://review.opendev.org/c/openstack/nova/+/864760

On Wed, Mar 29, 2023 at 10:26 AM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:

"If they *don't* provide this parameter, then depending on the default_schedule_zone config option, either the instance will eventually use a specific AZ (and then it's like if the enduser was asking for this AZ), or none of AZ is requested and then the instance can be created and moved between any hosts within *all* AZs."

I ask aftet that, although without az when launch instances but they still have az. But i still mv to diffent host in diffent az when mirgrating or spawn which masakari. i am not clear, I tested.

On Wed, Mar 29, 2023, 7:38 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:

Yes. Thanks, but the things I would like to know: after instances are created, how do we know if it was launched with specified AZ or without it? I mean the way to distinguish between specified instances and non specified instances?

Nguyen Huu Khoi

On Wed, Mar 29, 2023 at 5:05 PM Sylvain Bauza <sbauza@redhat.com> wrote:

Le mer. 29 mars 2023 à 08:06, Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> a écrit :

Hello.

I have one question.

Follow this 

https://docs.openstack.org/nova/latest/admin/availability-zones.html

If the server was not created in a specific zone then it is free to be moved to other zones. but when I use 

openstack server show [server id]

I still see the "OS-EXT-AZ:availability_zone" value belonging to my instance.

Correct, this is normal. If the operators creates some AZs, then the enduser should see where the instance in which AZ.

 

Could you tell the difference which causes "if the server was not created in a specific zone then it is free to be moved to other zones."

To be clear, an operator can create Availability Zones. Those AZs can then be seen by an enduser using the os-availability-zones API [1]. Then, either the enduser wants to use a specific AZ for their next instance creation (and if so, he/she adds --availability-zone parameter to their instance creation client) or they don't want and then they don't provide this parameter.

If they provide this parameter, then the server will be created only in one host in the specific AZ and then when moving the instance later, it will continue to move to any host within the same AZ.

If they *don't* provide this parameter, then depending on the default_schedule_zone config option, either the instance will eventually use a specific AZ (and then it's like if the enduser was asking for this AZ), or none of AZ is requested and then the instance can be created and moved between any hosts within *all* AZs.

That being said, as I said earlier, the enduser can still verify the AZ from where the instance is by the server show parameter you told.

We also have a documentation explaining about Availability Zones, maybe this would help you more to understand about AZs : https://docs.openstack.org/nova/latest/admin/availability-zones.html

[1] https://docs.openstack.org/api-ref/compute/#availability-zones-os-availability-zone (tbc, the enduser won't see the hosts, but they can see the list of existing AZs)

 

Nguyen Huu Khoi

On Mon, Mar 27, 2023 at 8:37 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:

Hello guys.

I just suggest to openstack nova works better. My story because 

1. The server was created in a specific zone with the POST /servers request containing the availability_zone parameter.

It will be nice when we attach randow zone when we create instances then It will only move to the same zone when migrating or masakari ha.

Currently we can force it to zone by default zone shedule in nova.conf.

Sorry because I am new to Openstack and I am just an operator. I try to verify some real cases. 

 

Nguyen Huu Khoi

On Mon, Mar 27, 2023 at 7:43 PM Sylvain Bauza <sbauza@redhat.com> wrote:

Le lun. 27 mars 2023 à 14:28, Sean Mooney <smooney@redhat.com> a écrit :

On Mon, 2023-03-27 at 14:06 +0200, Sylvain Bauza wrote:
> Le lun. 27 mars 2023 à 13:51, Sean Mooney <smooney@redhat.com> a écrit :
>
> > On Mon, 2023-03-27 at 10:19 +0200, Sylvain Bauza wrote:
> > > Le dim. 26 mars 2023 à 14:30, Rafael Weingärtner <
> > > rafaelweingartner@gmail.com> a écrit :
> > >
> > > > Hello Nguyễn Hũu Khôi,
> > > > You might want to take a look at:
> > > > https://review.opendev.org/c/openstack/nova/+/864760. We created a
> > patch
> > > > to avoid migrating VMs to any AZ, once the VM has been bootstrapped in
> > an
> > > > AZ that has cross zone attache equals to false.
> > > >
> > > >
> > > Well, I'll provide some comments in the change, but I'm afraid we can't
> > > just modify the request spec like you would want.
> > >
> > > Anyway, if you want to discuss about it in the vPTG, just add it in the
> > > etherpad and add your IRC nick so we could try to find a time where we
> > > could be discussing it : https://etherpad.opendev.org/p/nova-bobcat-ptg
> > > Also, this kind of behaviour modification is more a new feature than a
> > > bugfix, so fwiw you should create a launchpad blueprint so we could
> > better
> > > see it.
> >
> > i tought i left review feedback on that too that the approch was not
> > correct.
> > i guess i did not in the end.
> >
> > modifying the request spec as sylvain menthioned is not correct.
> > i disucssed this topic on irc a few weeks back with mohomad for vxhost.
> > what can be done is as follows.
> >
> > we can add a current_az field  to the Destination object
> >
> > https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L1092-L1122
> > The conductor can read the instance.AZ and populate it in that new field.
> > We can then add a new weigher to prefer hosts that are in the same az.
> >
> >
>
> I tend to disagree this approach as people would think that the
> Destination.az field would be related to the current AZ for an instance,
> while we only look at the original AZ.
> That being said, we could have a weigher that would look at whether the
> host is in the same AZ than the instance.host.
you miss understood what i wrote

i suggested addint Destination.current_az to store teh curernt AZ of the instance before scheduling.

so my proposal is if RequestSpec.AZ is not set and Destination.current_az is set then the new
weigher would prefer hosts that are in the same az as Destination.current_az

we coudl also call Destination.current_az Destination.prefered_az

I meant, I think we don't need to provide a new field, we can already know about what host an existing instance uses if we want (using [1])

Anyway, let's stop to discuss about it here, we should rather review that for a Launchpad blueprint or more a spec.

-Sylvain

[1] https://github.com/openstack/nova/blob/b9a49ffb04cb5ae2d8c439361a3552296df02988/nova/scheduler/host_manager.py#L369-L370

>
>
> This will provide soft AZ affinity for the vm and preserve the fact that if
> > a vm is created without sepcifying
> > An AZ the expectaiton at the api level woudl be that it can migrate to any
> > AZ.
> >
> > To provide hard AZ affintiy we could also add prefileter that would use
> > the same data but instead include it in the
> > placement query so that only the current AZ is considered. This would have
> > to be disabled by default.
> >
> >
> Sure, we could create a new prefilter so we could then deprecate the
> AZFilter if we want.
we already have an AZ prefilter and the AZFilter is deprecate for removal
i ment to delete it in zed but did not have time to do it in zed of Antielope
i deprecated the AZ| filter in https://github.com/openstack/nova/commit/7c7a2a142d74a7deeda2a79baf21b689fe32cd08
xena when i enabeld the az prefilter by default.

Ah whoops, indeed I forgot the fact we already have the prefilter, so the hard support for AZ is already existing.

 

i will try an delete teh AZ filter before m1 if others dont.

OK.

 

>
>
> > That woudl allow operators to choose the desired behavior.
> > curret behavior (disable weigher and dont enabel prefilter)
> > new default, prefer current AZ (weigher enabeld prefilter disabled)
> > hard affintiy(prefilter enabled.)
> >
> > there are other ways to approch this but updating the request spec is not
> > one of them.
> > we have to maintain the fact the enduser did not request an AZ.
> >
> >
> Anyway, if folks want to discuss about AZs, this week is the good time :-)
>
>
> > >
> > > -Sylvain
> > >
> > >
> > >
> > > > On Sun, Mar 26, 2023 at 8:20 AM Nguyễn Hữu Khôi <
> > nguyenhuukhoinw@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello guys.
> > > > > I playing with Nova AZ and Masakari
> > > > >
> > > > > https://docs.openstack.org/nova/latest/admin/availability-zones.html
> > > > >
> > > > > Masakari will move server by nova scheduler.
> > > > >
> > > > > Openstack Docs describe that:
> > > > >
> > > > > If the server was not created in a specific zone then it is free to
> > be
> > > > > moved to other zones, i.e. the AvailabilityZoneFilter
> > > > > <
> > https://docs.openstack.org/nova/latest/admin/scheduling.html#availabilityzonefilter>
> > is
> > > > > a no-op.
> > > > >
> > > > > I see that everyone usually creates instances with "Any Availability
> > > > > Zone" on Horzion and also we don't specify AZ when creating
> > instances by
> > > > > cli.
> > > > >
> > > > > By this way, when we use Masakari or we miragrated instances( or
> > > > > evacuate) so our instance will be moved to other zones.
> > > > >
> > > > > Can we attach AZ to server create requests API based on Any
> > > > > Availability Zone to limit instances moved to other zones?
> > > > >
> > > > > Thank you. Regards
> > > > >
> > > > > Nguyen Huu Khoi
> > > > >
> > > >
> > > >
> > > > --
> > > > Rafael Weingärtner
> > > >
> >
> >

--

Rafael Weingärtner