[nova] Strict isolation of group of hosts for image and flavor, modifying command 'nova-manage placement sync_aggregates'
Hi,
Working on implementation of 'Support filtering of allocation_candidates by forbidden aggregates' spec.
Need discussion particularly for point [1] where traits needs to be sync along with aggregates at placement.
Master implementation for 'nova-manage placement sync_aggregates' command is to sync the nova host aggregates.
Modifying this command to sync trait metadata of aggregate at placement.
Below are the aggregate restful APIs which currently supports:
1. 'POST'-- /os-aggregates/{aggregate_id}/action(Add host) getting synced on the placement service 2. 'POST'-- /os-aggregates/{aggregate_id}/action(Remove host) getting synced on the placement service 3. 'POST'-- /os-aggregates/{aggregate_id}/action(set metadata) Doesn't get sync on the placement service.
4. 'POST'-- /os-aggregates/{aggregate_id}/action(unset metadata) Doesn't get sync on the placement service.
I have added code to sync traits for below APIs and I don't see any issues there:
1. 'POST'-- /os-aggregates/{aggregate_id}/action(Add host) 2. 'POST'-- /os-aggregates/{aggregate_id}/action(set metadata)
But there is an issue while removing traits for below APIs:
1. 'POST'-- /os-aggregates/{aggregate_id}/action(Remove host) 2. 'POST'-- /os-aggregates/{aggregate_id}/action(unset metadata)
Ideally, we should remove traits set in the aggregate metadata from the resource providers associated with the aggregate for above two APIs but it could cause a problem for below scenario:-
For example:
1. Create two aggregates 'agg1' and 'agg2' by using:
'POST'-- /os-aggregates(Create aggregate)
2. Associate above aggregates to host 'RP1' by using:
'POST'-- /os-aggregates/{aggregate_id}/action(Add host)
3. Setting metadata (trait:STORAGE_DISK_SSD='required') on the aggregate agg1 by using:
'POST'-- /os-aggregates/{aggregate_id}/action(set metadata)
4. Setting metadata (trait:STORAGE_DISK_SSD='required', trait:HW_CPU_X86_SGX='required') on the aggregate agg2 by using:
'POST'-- /os-aggregates/{aggregate_id}/action(set metadata)
Traits set to 'RP1' are:
STORAGE_DISK_SSD
HW_CPU_X86_SGX
Note: Here trait 'STORAGE_DISK_SSD' is set on both agg1 and agg2.
Now, If we remove host 'RP1' from 'agg1' then the trait 'STORAGE_DISK_SSD' set to `RP1` also needs to be removed but since 'RP1' is also assigned to 'agg2', removing 'STORAGE_DISK_SSD' trait from 'RP1' is not correct.
I have discussed about syncing traits issues with Eric on IRC [2], he has suggested few approaches as below:
- Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
- Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
- Define a "namespace" - a trait substring - and remove only traits in that namespace.
If I'm not wrong, for last two approaches, we would need to change RestFul APIs.
Need your feedback whether traits should be deleted from resource provider or not for below two cases?
1. 'POST'-- /os-aggregates/{aggregate_id}/action(Remove host)
2. 'POST'-- /os-aggregates/{aggregate_id}/action(unset metadata)
[1]: https://review.opendev.org/#/c/609960/8/specs/train/approved/placement-req-f... [2]: http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2...
Thanks & Regards, Vrushali Kamde
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
Let me TL;DR this:
The forbidden aggregates filter spec [1] says when we put trait metadata onto a host aggregate, we should add the same trait to the compute node RPs for all the hosts in that aggregate, so that the feature actually works when we use it.
But we never talked about what to do when we *remove* a trait from such an aggregate, or trash an aggregate with traits, or remove a host from such an aggregate.
Here are the alternatives, as Vrushali laid them out (letters added by me):
(a) Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
(b) Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only traits in that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If I’m not wrong, for last two approaches, we would need to change RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
And when we do any of the removal things, we always and only remove any trait containing the substring _AGGREGATE_ISOLATION_ (assuming it's not asserted by other aggregates in which the host is also a member, yatta yatta).
IMO (a) and (c) both suck, but (c) is a slightly better experience for the user.
efried
[1] http://specs.openstack.org/openstack/nova-specs/specs/train/approved/placeme...
P.S.
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
Hear that, pipermail?
On Thu, 2019-06-06 at 16:52 -0500, Eric Fried wrote:
Let me TL;DR this:
The forbidden aggregates filter spec [1] says when we put trait metadata onto a host aggregate, we should add the same trait to the compute node RPs for all the hosts in that aggregate, so that the feature actually works when we use it.
But we never talked about what to do when we *remove* a trait from such an aggregate, or trash an aggregate with traits, or remove a host from such an aggregate.
Here are the alternatives, as Vrushali laid them out (letters added by me):
(a) Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
(b) Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only traits in that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If I’m not wrong, for last two approaches, we would need to change RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
And when we do any of the removal things, we always and only remove any trait containing the substring _AGGREGATE_ISOLATION_ (assuming it's not asserted by other aggregates in which the host is also a member, yatta yatta).
IMO (a) and (c) both suck, but (c) is a slightly better experience for the user.
c is only a good option if we are talking about a specific set of new traits for this usecase e.g. CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE but if we want to allow setting tratis genericaly on hosts via host-aggrages its not really that usefull.
for example we talked about useing a hyperthreading trait in the cpu pinning spec which will not be managed by the compute driver. host aggages would be a convient way to be able to manage that trait if this was a generic feature.
for c you still have to deal with the fact a host can be in multiple host aggrates too by the way so jsut because a thread is namespace d and it is removed from an aggrate does not mean its correcct to remove it from a host.
On Fri, Jun 7, 2019 at 1:31 AM, Sean Mooney smooney@redhat.com wrote:
On Thu, 2019-06-06 at 16:52 -0500, Eric Fried wrote:
Let me TL;DR this:
The forbidden aggregates filter spec [1] says when we put trait metadata onto a host aggregate, we should add the same trait to the compute node RPs for all the hosts in that aggregate, so that the feature actually works when we use it.
But we never talked about what to do when we *remove* a trait from such an aggregate, or trash an aggregate with traits, or remove a host from such an aggregate.
Here are the alternatives, as Vrushali laid them out (letters added by me):
(a) Leave all traits alone. If they need to be removed, it would
have to
be manually via a separate step.
(b) Support a new option so the caller can dictate whether the
operation
should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only
traits in
that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If Iʼm not wrong, for last two approaches, we would need to
change
RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
Besides possible performance impacts, I think this would be the logical behavior from nova to do.
cheers, gibi
On Fri, 2019-06-07 at 07:38 +0000, Balázs Gibizer wrote:
On Fri, Jun 7, 2019 at 1:31 AM, Sean Mooney smooney@redhat.com wrote:
On Thu, 2019-06-06 at 16:52 -0500, Eric Fried wrote:
Let me TL;DR this:
The forbidden aggregates filter spec [1] says when we put trait metadata onto a host aggregate, we should add the same trait to the compute node RPs for all the hosts in that aggregate, so that the feature actually works when we use it.
But we never talked about what to do when we *remove* a trait from such an aggregate, or trash an aggregate with traits, or remove a host from such an aggregate.
Here are the alternatives, as Vrushali laid them out (letters added by me):
(a) Leave all traits alone. If they need to be removed, it would
have to
be manually via a separate step.
(b) Support a new option so the caller can dictate whether the
operation
should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only
traits in
that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If Iʼm not wrong, for last two approaches, we would need to
change
RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
Besides possible performance impacts, I think this would be the logical behavior from nova to do.
option d is worstcase aproxmatly O(NlogN) but is technicall between O(n) and O(NM) where N is the number of instance and M is the maxium number of aggrates a host is a memeber of.
so it grows non linearly but the plroblem is not quadratic and is much closer to O(N) or O(NlogN) then it is to O(N^2) so long if we are smart about how we look up the data form the db its probably ok.
we are basically asking for all the host in this aggrate, give me the hosts that dont have another aggrate with the trait i am about to remove form this aggragte. those host are the set of host we need to update in placemnt and sql is good at anserwing that type of question. if we do this in python it will make me sad.
cheers, gibi
(a) Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
(b) Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only traits in that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If I’m not wrong, for last two approaches, we would need to change RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
Actually, it means we *can*, whereas (b) and (d) mean we *can't*. That's the whole point. If we want to isolate our hyperthreading hosts, we put them in an aggregate with HW_CPU_HYPERTHREADING on it. The sync here should be a no-op because those hosts should already have HW_CPU_HYPERTHREADING on them. And then if we decide to remove such a host, or destroy the aggregate, or whatever, we *don't want* HW_CPU_HYPERTHREADING to be removed from the providers, because they can still do that.
(Unless you mean we can't make a standard trait that we can use for isolation that gets (conditionally) removed in these scenarios? There's nothing preventing us from creating a standard trait called COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE, which would work just the same.)
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
Sorry I wasn't clear, (b) also does this ^ but with the condition that also checks for the _AGGREGATE_ISOLATION_ infix.
for c you still have to deal with the fact a host can be in multiple
host aggrates too by
the way so jsut because a thread is namespace d and it is removed from
an aggrate does not
mean its correcct to remove it from a host.
Right - in reality, there should be one algorithm, idempotent, to sync host RP traits when anything happens to aggregates. It always goes out and does the appropriate {set} math to decide which traits should exist on which hosts and effects any necessary changes.
And yes, the performance will suck in a large deployment, because we have to get all the compute RPs in all the aggregates (even the ones with no trait metadata) to do that calculation. But aggregate operations are fairly rare, aren't they?
Perhaps this is where we provide a nova-manage tool to do (b)'s sync manually (which we'll surely have to do anyway as a "heal" or for upgrades). So if you're not using the feature, you don't suffer the penalty.
for example we talked about useing a hyperthreading trait in the cpu pinning spec which will not be managed by the compute driver. host aggages would be a convient way to be able to manage that trait if this was a generic feature.
Oh, okay, yeah, I don't accept this as a use case for this feature. It will work, but we shouldn't recommend it precisely because it's asymmetrical (you can't remove the trait by removing it from the aggregate). There are other ways to add a random trait to all hosts in an aggregate (for host in `get providers in aggregate`; do openstack resource provider trait add ...; done).
But for the sake of discussion, what about:
(e) Fully manual. Aggregate operations never touch (add or remove) traits on host RPs. You always have to do that manually. As noted above, it's easy to do - and we could make it easier with a tiny wrapper that takes an aggregate, a list of traits, and an --add/--remove command. So initially, setting up aggregate isolation is a two-step process, and in the future we can consider making new API/CLI affordance that combines the steps.
efried .
On Fri, 2019-06-07 at 09:05 -0500, Eric Fried wrote:
(a) Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
(b) Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only traits in that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If I’m not wrong, for last two approaches, we would need to change RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
Actually, it means we *can*, whereas (b) and (d) mean we *can't*. That's the whole point. If we want to isolate our hyperthreading hosts, we put them in an aggregate with HW_CPU_HYPERTHREADING on it. The sync here should be a no-op because those hosts should already have HW_CPU_HYPERTHREADING on them. And then if we decide to remove such a host, or destroy the aggregate, or whatever, we *don't want* HW_CPU_HYPERTHREADING to be removed from the providers, because they can still do that.
in the cpu pinning spec we said HW_CPU_HYPERTHREADING was not going to be managed by the virt driver so it wont be reported unless the admin manulaly adds it.
https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cpu... "The HW_CPU_HYPERTHREADING trait will need to be among the traits that the virt driver cannot always override, since the operator may want to indicate that a single NUMA node on a multi-NUMA-node host is meant for guests that tolerate hyperthread siblings as dedicated CPUs."
so i was suggesting this was a way to enable that the operator to manage whic host report that trait although as the spec suggest we may want to report this differently per numa node which would still require you to use osc-placment or some other way to set it manually.
(Unless you mean we can't make a standard trait that we can use for isolation that gets (conditionally) removed in these scenarios? There's nothing preventing us from creating a standard trait called COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE, which would work just the same.)
im suggesting it woudl be nice to be able to use host aggates to manage statdard or custom traits on hosts that are not managed by the driver thwer ethat is a COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE trait or something else. so i was hoping to make this feature more reusable for other usecase in the future. for example it would be nice to be able to say this set of host has the CUSTOM_DPDK_NETWORKING trait by putting them in a host aggrage and then adding a forbindent trait to my non hugepage backed guests.
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
Sorry I wasn't clear, (b) also does this ^ but with the condition that also checks for the _AGGREGATE_ISOLATION_ infix.
for c you still have to deal with the fact a host can be in multiple
host aggrates too by
the way so jsut because a thread is namespace d and it is removed from
an aggrate does not
mean its correcct to remove it from a host.
Right - in reality, there should be one algorithm, idempotent, to sync host RP traits when anything happens to aggregates. It always goes out and does the appropriate {set} math to decide which traits should exist on which hosts and effects any necessary changes.
And yes, the performance will suck in a large deployment, because we have to get all the compute RPs in all the aggregates (even the ones with no trait metadata) to do that calculation. But aggregate operations are fairly rare, aren't they?
Perhaps this is where we provide a nova-manage tool to do (b)'s sync manually (which we'll surely have to do anyway as a "heal" or for upgrades). So if you're not using the feature, you don't suffer the penalty.
for example we talked about useing a hyperthreading trait in the cpu pinning spec which will not be managed by the compute driver. host aggages would be a convient way to be able to manage that trait if this was a generic feature.
Oh, okay, yeah, I don't accept this as a use case for this feature. It will work, but we shouldn't recommend it precisely because it's asymmetrical (you can't remove the trait by removing it from the aggregate).
why not we do not expect the virt driver to report the hypertreading trait since we said it can be extrenally managed. even if we allowed the virt drvier to conditionally report it only when it frst creates a RP it is not allowed to readd if it is remvoed by someone else.
There are other ways to add a random trait to all hosts in an aggregate (for host in `get providers in aggregate`; do openstack resource provider trait add ...; done).
But for the sake of discussion, what about:
(e) Fully manual. Aggregate operations never touch (add or remove) traits on host RPs. You always have to do that manually. As noted above, it's easy to do - and we could make it easier with a tiny wrapper that takes an aggregate, a list of traits, and an --add/--remove command. So initially, setting up aggregate isolation is a two-step process, and in the future we can consider making new API/CLI affordance that combines the steps.
efried .
On Fri, 2019-06-07 at 19:04 +0100, Sean Mooney wrote:
On Fri, 2019-06-07 at 09:05 -0500, Eric Fried wrote:
(a) Leave all traits alone. If they need to be removed, it would have to be manually via a separate step.
(b) Support a new option so the caller can dictate whether the operation should remove the traits. (This is all-or-none.)
(c) Define a "namespace" - a trait substring - and remove only traits in that namespace.
I'm going to -1 (b). It's too big a hammer, at too big a cost (including API changes).
If I’m not wrong, for last two approaches, we would need to change RestFul APIs.
No, (c) does not. By "define a namespace" I mean we would establish a naming convention for traits to be used with this feature. For example:
CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
i personaly dislike c as it means we cannot use any standard traits in host aggrates.
Actually, it means we *can*, whereas (b) and (d) mean we *can't*. That's the whole point. If we want to isolate our hyperthreading hosts, we put them in an aggregate with HW_CPU_HYPERTHREADING on it. The sync here should be a no-op because those hosts should already have HW_CPU_HYPERTHREADING on them. And then if we decide to remove such a host, or destroy the aggregate, or whatever, we *don't want* HW_CPU_HYPERTHREADING to be removed from the providers, because they can still do that.
in the cpu pinning spec we said HW_CPU_HYPERTHREADING was not going to be managed by the virt driver so it wont be reported unless the admin manulaly adds it.
https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cpu...
"The HW_CPU_HYPERTHREADING trait will need to be among the traits that the virt driver cannot always override, since the operator may want to indicate that a single NUMA node on a multi-NUMA-node host is meant for guests that tolerate hyperthread siblings as dedicated CPUs."
so i was suggesting this was a way to enable that the operator to manage whic host report that trait although as the spec suggest we may want to report this differently per numa node which would still require you to use osc-placment or some other way to set it manually.
(Unless you mean we can't make a standard trait that we can use for isolation that gets (conditionally) removed in these scenarios? There's nothing preventing us from creating a standard trait called COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE, which would work just the same.)
im suggesting it woudl be nice to be able to use host aggates to manage statdard or custom traits on hosts that are not managed by the driver thwer ethat is a COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE trait or something else. so i was hoping to make this feature more reusable for other usecase in the future. for example it would be nice to be able to say this set of host has the CUSTOM_DPDK_NETWORKING trait by putting them in a host aggrage and then adding a forbindent trait to my non hugepage backed guests.
there is also an option d. when you remove a trait form a host aggregate for each host in the aggregate check if that traits exists on another aggregate the host is a member of and remove it if not found on another aggregate.
Sorry I wasn't clear, (b) also does this ^ but with the condition that also checks for the _AGGREGATE_ISOLATION_ infix.
for c you still have to deal with the fact a host can be in multiple
host aggrates too by
the way so jsut because a thread is namespace d and it is removed from
an aggrate does not
mean its correcct to remove it from a host.
Right - in reality, there should be one algorithm, idempotent, to sync host RP traits when anything happens to aggregates. It always goes out and does the appropriate {set} math to decide which traits should exist on which hosts and effects any necessary changes.
And yes, the performance will suck in a large deployment, because we have to get all the compute RPs in all the aggregates (even the ones with no trait metadata) to do that calculation. But aggregate operations are fairly rare, aren't they?
Perhaps this is where we provide a nova-manage tool to do (b)'s sync manually (which we'll surely have to do anyway as a "heal" or for upgrades). So if you're not using the feature, you don't suffer the penalty.
for example we talked about useing a hyperthreading trait in the cpu pinning spec which will not be managed by the compute driver. host aggages would be a convient way to be able to manage that trait if this was a generic feature.
Oh, okay, yeah, I don't accept this as a use case for this feature. It will work, but we shouldn't recommend it precisely because it's asymmetrical (you can't remove the trait by removing it from the aggregate).
why not we do not expect the virt driver to report the hypertreading trait since we said it can be extrenally managed. even if we allowed the virt drvier to conditionally report it only when it frst creates a RP it is not allowed to readd if it is remvoed by someone else.
There are other ways to add a random trait to all hosts in an aggregate (for host in `get providers in aggregate`; do openstack resource provider trait add ...; done).
But for the sake of discussion, what about:
(e) Fully manual. Aggregate operations never touch (add or remove) traits on host RPs. You always have to do that manually. As noted above, it's easy to do - and we could make it easier with a tiny wrapper that takes an aggregate, a list of traits, and an --add/--remove command. So initially, setting up aggregate isolation is a two-step process, and in the future we can consider making new API/CLI affordance that combines the steps.
ya e could work too. melanie added a similar functionality to osc placment for managing the alloction ratios of specific resource classes per aggregate a few months ago https://review.opendev.org/#/c/640898/
we could proably provide somthing similar for managing traits but determining what RP to add the trait too would be a littel tricker. we would have to be able to filter to RP with either a specific inventory or with a specific trait or in a speicic subtree.
you could have a --root or somthing to jsut say add or remove the tratit from the root RPs in an aggregate. but yes you could certely automate this in a simile cli extention.
efried .
(e) Fully manual. Aggregate operations never touch (add or remove) traits on host RPs. You always have to do that manually.
I'm going to come down in favor of this option. It's the shortest path to getting something viable working, in a way that is simple to understand, despite lacking magical DWIM-ness.
As noted above, it's easy to do - and we could make it easier with a tiny wrapper that takes an aggregate, a list of traits, and an --add/--remove command. So initially, setting up aggregate isolation is a two-step process, and in the future we can consider making new API/CLI affordance that combines the steps.
ya e could work too. melanie added a similar functionality to osc placment for managing the alloction ratios of specific resource classes per aggregate a few months ago https://review.opendev.org/#/c/640898/
we could proably provide somthing similar for managing traits but determining what RP to add the trait too would be a littel tricker. we would have to be able to filter to RP with either a specific inventory or with a specific trait or in a speicic subtree.
We (Placement team) are still trying to figure out how to manage concepts like "resourceless request groups" and "traits/aggregates flow down". But for now, Nova is still always modeling VCPU/MEMORY_MB and traits on the root provider, so let's simply hit the providers in the aggregate (i.e. the root compute host RPs).
I'm putting this on the agenda for Thursday's nova meeting [1] to hopefully get some more Nova opinions on it.
efried
[1] https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
On Mon, 2019-06-10 at 17:38 -0500, Eric Fried wrote:
(e) Fully manual. Aggregate operations never touch (add or remove) traits on host RPs. You always have to do that manually.
I'm going to come down in favor of this option. It's the shortest path to getting something viable working, in a way that is simple to understand, despite lacking magical DWIM-ness.
As noted above, it's easy to do - and we could make it easier with a tiny wrapper that takes an aggregate, a list of traits, and an --add/--remove command. So initially, setting up aggregate isolation is a two-step process, and in the future we can consider making new API/CLI affordance that combines the steps.
ya e could work too. melanie added a similar functionality to osc placment for managing the alloction ratios of specific resource classes per aggregate a few months ago https://review.opendev.org/#/c/640898/
we could proably provide somthing similar for managing traits but determining what RP to add the trait too would be a littel tricker. we would have to be able to filter to RP with either a specific inventory or with a specific trait or in a speicic subtree.
We (Placement team) are still trying to figure out how to manage concepts like "resourceless request groups" and "traits/aggregates flow down". But for now, Nova is still always modeling VCPU/MEMORY_MB and traits on the root provider, so let's simply hit the providers in the aggregate (i.e. the root compute host RPs).
I'm putting this on the agenda for Thursday's nova meeting [1] to hopefully get some more Nova opinions on it.
for what its worth for the host-aggate case teh aplity to add or remvoe a trait form all root providers is likely enough, so that would make a cli much simpeler to create.
for the generic case of being able to add/remove a trait on an rp that could be anyhere in a nested tree for all trees in an aggaget, that is a much harder problem but we also do not need it to solve the usecase we have today so we can defer that until we actully need it and if we never need it we can defer it forever.
so +1 for keeping it simple and just updating the root RPs.
efried
[1] https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
We discussed this in the nova meeting today [1] with a little spillover in the -nova channel afterward [2].
The consensus was: Don't muck with resource provider traits at all during aggregate operations. The operator must do that bit manually. As a stretch goal, we can write a simple utility to help with this.
This was discussed as option (e) earlier in this thread.
The spec needs to get updated accordingly.
Thanks, efried
[1] http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-06-13-14.00.log.... [2] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2... (interleaved)
The consensus was: Don't muck with resource provider traits at all during aggregate operations. The operator must do that bit manually. As a stretch goal, we can write a simple utility to help with this.
This was discussed as option (e) earlier in this thread.
I have updated specs and uploaded a new patch for review. https://review.opendev.org/#/c/665605/
Regards, Tushar Patil
________________________________________ From: Eric Fried openstack@fried.cc Sent: Friday, June 14, 2019 3:36:04 AM To: openstack-discuss@lists.openstack.org Subject: Re: [nova] Strict isolation of group of hosts for image and flavor, modifying command 'nova-manage placement sync_aggregates'
We discussed this in the nova meeting today [1] with a little spillover in the -nova channel afterward [2].
The consensus was: Don't muck with resource provider traits at all during aggregate operations. The operator must do that bit manually. As a stretch goal, we can write a simple utility to help with this.
This was discussed as option (e) earlier in this thread.
The spec needs to get updated accordingly.
Thanks, efried
[1] http://eavesdrop.openstack.org/meetings/nova/2019/nova.2019-06-13-14.00.log.... [2] http://eavesdrop.openstack.org/irclogs/%23openstack-nova/%23openstack-nova.2... (interleaved)
Disclaimer: This email and any attachments are sent in strictest confidence for the sole use of the addressee and may contain legally privileged, confidential, and proprietary data. If you are not the intended recipient, please advise the sender by replying promptly to this email and then delete and destroy this email and any attachments without any further use, copying or forwarding.
participants (5)
-
Balázs Gibizer
-
Eric Fried
-
Kamde, Vrushali
-
Patil, Tushar
-
Sean Mooney