Open Stack

On Fri, 2019-06-07 at 09:05 -0500, Eric Fried wrote:
> > > > (a) Leave all traits alone. If they need to be removed, it would have to
> > > > be manually via a separate step.
> > > > 
> > > > (b) Support a new option so the caller can dictate whether the operation
> > > > should remove the traits. (This is all-or-none.)
> > > > 
> > > > (c) Define a "namespace" - a trait substring - and remove only traits in
> > > > that namespace.
> > > 
> > > I'm going to -1 (b). It's too big a hammer, at too big a cost (including
> > > API changes).
> > > 
> > > > If I’m not wrong, for last two approaches, we would need to change
> > > > RestFul APIs.
> > > 
> > > No, (c) does not. By "define a namespace" I mean we would establish a
> > > naming convention for traits to be used with this feature. For example:
> > > 
> > > CUSTOM_AGGREGATE_ISOLATION_WINDOWS_LICENSE
> > 
> > i personaly dislike c as it means we cannot use any standard traits in host
> > aggrates.
> 
> Actually, it means we *can*, whereas (b) and (d) mean we *can't*. That's
> the whole point. If we want to isolate our hyperthreading hosts, we put
> them in an aggregate with HW_CPU_HYPERTHREADING on it. The sync here
> should be a no-op because those hosts should already have
> HW_CPU_HYPERTHREADING on them. And then if we decide to remove such a
> host, or destroy the aggregate, or whatever, we *don't want*
> HW_CPU_HYPERTHREADING to be removed from the providers, because they can
> still do that.

in the cpu pinning spec we said HW_CPU_HYPERTHREADING was not going to be managed
by the virt driver so it wont be reported unless the admin manulaly adds it.

https://github.com/openstack/nova-specs/blob/master/specs/train/approved/cpu-resources.rst#add-hw_cpu_hyperthreading-trait
"The HW_CPU_HYPERTHREADING trait will need to be among the traits that the virt driver cannot always override, since the
operator may want to indicate that a single NUMA node on a multi-NUMA-node host is meant for guests that tolerate
hyperthread siblings as dedicated CPUs."

so i was suggesting this was a way to enable that the operator to manage whic host report that trait
although as the spec suggest we may  want to report this differently per numa node which would still
require you to use osc-placment or some other way to set it manually.
> 
> (Unless you mean we can't make a standard trait that we can use for
> isolation that gets (conditionally) removed in these scenarios? There's
> nothing preventing us from creating a standard trait called
> COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE, which would work just the
> same.)

im suggesting it woudl be nice to be able to use host aggates to manage statdard or custom traits on hosts that are
not managed by the driver thwer ethat is a COMPUTE_AGGREGATE_ISOLATION_WINDOWS_LICENSE trait or something
else. so i was hoping to make this feature more reusable for other usecase in the future. for example it would be nice
to be able to say this set of host has the CUSTOM_DPDK_NETWORKING trait by putting them in a host aggrage and then
adding a forbindent trait to my non hugepage backed guests.
> 
> > there is also an option d. when you remove a trait form a host aggregate for each host in
> > the aggregate check if that traits exists on another aggregate the host is a member of and remove
> > it if not found on another aggregate.
> 
> Sorry I wasn't clear, (b) also does this ^ but with the condition that
> also checks for the _AGGREGATE_ISOLATION_ infix.
> 
> > for c you still have to deal with the fact a host can be in multiple
> 
> host aggrates too by
> > the way so jsut because a thread is namespace d and it is removed from
> 
> an aggrate does not
> > mean its correcct to remove it from a host.
> 
> Right - in reality, there should be one algorithm, idempotent, to sync
> host RP traits when anything happens to aggregates. It always goes out
> and does the appropriate {set} math to decide which traits should exist
> on which hosts and effects any necessary changes.
> 
> And yes, the performance will suck in a large deployment, because we
> have to get all the compute RPs in all the aggregates (even the ones
> with no trait metadata) to do that calculation. But aggregate operations
> are fairly rare, aren't they?
> 
> Perhaps this is where we provide a nova-manage tool to do (b)'s sync
> manually (which we'll surely have to do anyway as a "heal" or for
> upgrades). So if you're not using the feature, you don't suffer the penalty.
> 
> > for example we talked about useing a hyperthreading trait in the cpu pinning spec which
> > will not be managed by the compute driver. host aggages would be a convient way
> > to be able to manage that trait if this was a generic feature.
> 
> Oh, okay, yeah, I don't accept this as a use case for this feature. It
> will work, but we shouldn't recommend it precisely because it's
> asymmetrical (you can't remove the trait by removing it from the
> aggregate).
why not we do not expect the virt driver to report the hypertreading trait
since we said it can be extrenally managed. even if we allowed the virt drvier to
conditionally report it only when it frst creates a RP it is not allowed to readd
if it is remvoed by someone else.

> There are other ways to add a random trait to all hosts in
> an aggregate (for host in `get providers in aggregate`; do openstack
> resource provider trait add ...; done).
> 
> But for the sake of discussion, what about:
> 
> (e) Fully manual. Aggregate operations never touch (add or remove)
> traits on host RPs. You always have to do that manually. As noted above,
> it's easy to do - and we could make it easier with a tiny wrapper that
> takes an aggregate, a list of traits, and an --add/--remove command. So
> initially, setting up aggregate isolation is a two-step process, and in
> the future we can consider making new API/CLI affordance that combines
> the steps.
> 
> efried
> .
>