Bug report created at: https://bugs.launchpad.net/nova/+bug/2018318 On Wed, Apr 26, 2023 at 12:23 PM Sylvain Bauza <sbauza@redhat.com> wrote:
Le mer. 26 avr. 2023 à 14:46, Rafael Weingärtner < rafaelweingartner@gmail.com> a écrit :
Adding the response to this thread, as I replied in the wrong one. Sorry for the confusion.
Hello Sylvain Bauza and Sean Mooney,
The patch pointed out in [1] is doing exactly what you said that we should not do. I mean, changing the AZ in the request spec of the virtual machine (VM) that the user created. The patch we propose in [2] is intended to avoid exactly that.
As pointed out by Florian, the configuration "cross_az_attach" is a constraint of the cloud environment, and as such it should be considered when selecting hosts to execute VMs migrations. Therefore, for those situations, and only for those (cross_az_attach=False), we send the current AZ of the VM to placement, to enable it (placement) to filter out hosts that are not from the current AZ of the VM.
Looking at the patch suggested by you in [1], and later testing it, we can confirm that the problem is still there in main/upstream. This happens because the patch in [1] is only addressing the cases when the VM is created based on volumes, and then it sets the AZ of the volumes in the request spec of the VM. That is why everything works for the setups where cross_az_attach=False. However, if we create a VM based on an image, and then it (Nova) creates a new volume in Cinder, the AZ is not set in the request spec (but it is used to execute the first call to placement to select the hosts); thus, the issues described in [2] can still happen.
Anyways, the proposal presented in [2] is simpler and works nicely. We can discuss it further in the patchset then, if you guys think it is worth it.
As I replied in the gerrit change https://review.opendev.org/c/openstack/nova/+/864760/comments/4a302ce3_9805e... then you should create a Launchpad bug report but fwiw, you should also modify the implementation as it would rather do the same for image metadata that what we do for volumes with [1]
-Sylvain
[1]
https://review.opendev.org/c/openstack/nova/+/469675/12/nova/compute/api.py#... [2] https://review.opendev.org/c/openstack/nova/+/864760
On Wed, Mar 29, 2023 at 10:26 AM Nguyễn Hữu Khôi < nguyenhuukhoinw@gmail.com> wrote:
"If they *don't* provide this parameter, then depending on the default_schedule_zone config option, either the instance will eventually use a specific AZ (and then it's like if the enduser was asking for this AZ), or none of AZ is requested and then the instance can be created and moved between any hosts within *all* AZs."
I ask aftet that, although without az when launch instances but they still have az. But i still mv to diffent host in diffent az when mirgrating or spawn which masakari. i am not clear, I tested.
On Wed, Mar 29, 2023, 7:38 PM Nguyễn Hữu Khôi <nguyenhuukhoinw@gmail.com> wrote:
Yes. Thanks, but the things I would like to know: after instances are created, how do we know if it was launched with specified AZ or without it? I mean the way to distinguish between specified instances and non specified instances?
Nguyen Huu Khoi
On Wed, Mar 29, 2023 at 5:05 PM Sylvain Bauza <sbauza@redhat.com> wrote:
Le mer. 29 mars 2023 à 08:06, Nguyễn Hữu Khôi < nguyenhuukhoinw@gmail.com> a écrit :
Hello. I have one question. Follow this
https://docs.openstack.org/nova/latest/admin/availability-zones.html
If the server was not created in a specific zone then it is free to be moved to other zones. but when I use
openstack server show [server id]
I still see the "OS-EXT-AZ:availability_zone" value belonging to my instance.
Correct, this is normal. If the operators creates some AZs, then the enduser should see where the instance in which AZ.
Could you tell the difference which causes "if the server was not created in a specific zone then it is free to be moved to other zones. "
To be clear, an operator can create Availability Zones. Those AZs can then be seen by an enduser using the os-availability-zones API [1]. Then, either the enduser wants to use a specific AZ for their next instance creation (and if so, he/she adds --availability-zone parameter to their instance creation client) or they don't want and then they don't provide this parameter.
If they provide this parameter, then the server will be created only in one host in the specific AZ and then when moving the instance later, it will continue to move to any host within the same AZ. If they *don't* provide this parameter, then depending on the default_schedule_zone config option, either the instance will eventually use a specific AZ (and then it's like if the enduser was asking for this AZ), or none of AZ is requested and then the instance can be created and moved between any hosts within *all* AZs.
That being said, as I said earlier, the enduser can still verify the AZ from where the instance is by the server show parameter you told.
We also have a documentation explaining about Availability Zones, maybe this would help you more to understand about AZs : https://docs.openstack.org/nova/latest/admin/availability-zones.html
[1] https://docs.openstack.org/api-ref/compute/#availability-zones-os-availabili... (tbc, the enduser won't see the hosts, but they can see the list of existing AZs)
Nguyen Huu Khoi
On Mon, Mar 27, 2023 at 8:37 PM Nguyễn Hữu Khôi < nguyenhuukhoinw@gmail.com> wrote:
> Hello guys. > > I just suggest to openstack nova works better. My story because > > > 1. > > The server was created in a specific zone with the POST /servers request > containing the availability_zone parameter. > > It will be nice when we attach randow zone when we create instances > then It will only move to the same zone when migrating or masakari ha. > > Currently we can force it to zone by default zone shedule in > nova.conf. > > Sorry because I am new to Openstack and I am just an operator. I try > to verify some real cases. > > > > Nguyen Huu Khoi > > > On Mon, Mar 27, 2023 at 7:43 PM Sylvain Bauza <sbauza@redhat.com> > wrote: > >> >> >> Le lun. 27 mars 2023 à 14:28, Sean Mooney <smooney@redhat.com> a >> écrit : >> >>> On Mon, 2023-03-27 at 14:06 +0200, Sylvain Bauza wrote: >>> > Le lun. 27 mars 2023 à 13:51, Sean Mooney <smooney@redhat.com> >>> a écrit : >>> > >>> > > On Mon, 2023-03-27 at 10:19 +0200, Sylvain Bauza wrote: >>> > > > Le dim. 26 mars 2023 à 14:30, Rafael Weingärtner < >>> > > > rafaelweingartner@gmail.com> a écrit : >>> > > > >>> > > > > Hello Nguyễn Hũu Khôi, >>> > > > > You might want to take a look at: >>> > > > > https://review.opendev.org/c/openstack/nova/+/864760. We >>> created a >>> > > patch >>> > > > > to avoid migrating VMs to any AZ, once the VM has been >>> bootstrapped in >>> > > an >>> > > > > AZ that has cross zone attache equals to false. >>> > > > > >>> > > > > >>> > > > Well, I'll provide some comments in the change, but I'm >>> afraid we can't >>> > > > just modify the request spec like you would want. >>> > > > >>> > > > Anyway, if you want to discuss about it in the vPTG, just >>> add it in the >>> > > > etherpad and add your IRC nick so we could try to find a >>> time where we >>> > > > could be discussing it : >>> https://etherpad.opendev.org/p/nova-bobcat-ptg >>> > > > Also, this kind of behaviour modification is more a new >>> feature than a >>> > > > bugfix, so fwiw you should create a launchpad blueprint so >>> we could >>> > > better >>> > > > see it. >>> > > >>> > > i tought i left review feedback on that too that the approch >>> was not >>> > > correct. >>> > > i guess i did not in the end. >>> > > >>> > > modifying the request spec as sylvain menthioned is not >>> correct. >>> > > i disucssed this topic on irc a few weeks back with mohomad >>> for vxhost. >>> > > what can be done is as follows. >>> > > >>> > > we can add a current_az field to the Destination object >>> > > >>> > > >>> https://github.com/openstack/nova/blob/master/nova/objects/request_spec.py#L... >>> > > The conductor can read the instance.AZ and populate it in that >>> new field. >>> > > We can then add a new weigher to prefer hosts that are in the >>> same az. >>> > > >>> > > >>> > >>> > I tend to disagree this approach as people would think that the >>> > Destination.az field would be related to the current AZ for an >>> instance, >>> > while we only look at the original AZ. >>> > That being said, we could have a weigher that would look at >>> whether the >>> > host is in the same AZ than the instance.host. >>> you miss understood what i wrote >>> >>> i suggested addint Destination.current_az to store teh curernt AZ >>> of the instance before scheduling. >>> >>> so my proposal is if RequestSpec.AZ is not set and >>> Destination.current_az is set then the new >>> weigher would prefer hosts that are in the same az as >>> Destination.current_az >>> >>> we coudl also call Destination.current_az Destination.prefered_az >>> >>> >> I meant, I think we don't need to provide a new field, we can >> already know about what host an existing instance uses if we want (using >> [1]) >> Anyway, let's stop to discuss about it here, we should rather >> review that for a Launchpad blueprint or more a spec. >> >> -Sylvain >> >> [1] >> https://github.com/openstack/nova/blob/b9a49ffb04cb5ae2d8c439361a3552296df02... >> >>> > >>> > >>> > This will provide soft AZ affinity for the vm and preserve the >>> fact that if >>> > > a vm is created without sepcifying >>> > > An AZ the expectaiton at the api level woudl be that it can >>> migrate to any >>> > > AZ. >>> > > >>> > > To provide hard AZ affintiy we could also add prefileter that >>> would use >>> > > the same data but instead include it in the >>> > > placement query so that only the current AZ is considered. >>> This would have >>> > > to be disabled by default. >>> > > >>> > > >>> > Sure, we could create a new prefilter so we could then deprecate >>> the >>> > AZFilter if we want. >>> we already have an AZ prefilter and the AZFilter is deprecate for >>> removal >>> i ment to delete it in zed but did not have time to do it in zed >>> of Antielope >>> i deprecated the AZ| filter in >>> https://github.com/openstack/nova/commit/7c7a2a142d74a7deeda2a79baf21b689fe3... >>> xena when i enabeld the az prefilter by default. >>> >>> >> Ah whoops, indeed I forgot the fact we already have the prefilter, >> so the hard support for AZ is already existing. >> >> >>> i will try an delete teh AZ filter before m1 if others dont. >>> >> >> OK. >> >> >>> > >>> > >>> > > That woudl allow operators to choose the desired behavior. >>> > > curret behavior (disable weigher and dont enabel prefilter) >>> > > new default, prefer current AZ (weigher enabeld prefilter >>> disabled) >>> > > hard affintiy(prefilter enabled.) >>> > > >>> > > there are other ways to approch this but updating the request >>> spec is not >>> > > one of them. >>> > > we have to maintain the fact the enduser did not request an AZ. >>> > > >>> > > >>> > Anyway, if folks want to discuss about AZs, this week is the >>> good time :-) >>> > >>> > >>> > > > >>> > > > -Sylvain >>> > > > >>> > > > >>> > > > >>> > > > > On Sun, Mar 26, 2023 at 8:20 AM Nguyễn Hữu Khôi < >>> > > nguyenhuukhoinw@gmail.com> >>> > > > > wrote: >>> > > > > >>> > > > > > Hello guys. >>> > > > > > I playing with Nova AZ and Masakari >>> > > > > > >>> > > > > > >>> https://docs.openstack.org/nova/latest/admin/availability-zones.html >>> > > > > > >>> > > > > > Masakari will move server by nova scheduler. >>> > > > > > >>> > > > > > Openstack Docs describe that: >>> > > > > > >>> > > > > > If the server was not created in a specific zone then it >>> is free to >>> > > be >>> > > > > > moved to other zones, i.e. the AvailabilityZoneFilter >>> > > > > > < >>> > > >>> https://docs.openstack.org/nova/latest/admin/scheduling.html#availabilityzon... >>> > >>> > > is >>> > > > > > a no-op. >>> > > > > > >>> > > > > > I see that everyone usually creates instances with "Any >>> Availability >>> > > > > > Zone" on Horzion and also we don't specify AZ when >>> creating >>> > > instances by >>> > > > > > cli. >>> > > > > > >>> > > > > > By this way, when we use Masakari or we miragrated >>> instances( or >>> > > > > > evacuate) so our instance will be moved to other zones. >>> > > > > > >>> > > > > > Can we attach AZ to server create requests API based on >>> Any >>> > > > > > Availability Zone to limit instances moved to other >>> zones? >>> > > > > > >>> > > > > > Thank you. Regards >>> > > > > > >>> > > > > > Nguyen Huu Khoi >>> > > > > > >>> > > > > >>> > > > > >>> > > > > -- >>> > > > > Rafael Weingärtner >>> > > > > >>> > > >>> > > >>> >>>
-- Rafael Weingärtner
-- Rafael Weingärtner