<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Eric Fried <openstack@fried.cc> 于2019年4月17日周三 上午3:28写道：<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">> I'm not sure I understand your proposal. Would you introduce a VM <br>
> resource and then allocate 1 of that resource for each VM?<br>
<br>
This has been proposed before, somewhere: translating<br>
max_instances_per_host to an inventory of resource class "VM" on the<br>
compute node RP, and including resources:VM=1 in every GET /a_c request.<br></blockquote><div><br></div><div>Actually, I propose to attach traits to the RP which has the VCPU resource. In the case, we have NUMA in placement. We will attach traits to the numa node RP. I just try to explain why that may makes sense. Since the those traits should be attached to the "Compute" resource, and the VCPU is just that "Compute" resource.(yes, we have two numa nodes, then both two numa nodes has those traits, but it should be fine). When we are asking those traits, then we must asking the VCPU, right? If so, it sounds make sense. Or is there any case we only request a trait, totally no resource requesting? If yes, that may not works. But I think the case we begin to discussion is about the trait and resource aren't in the same RP. For the neutron bw case, we still attach nic type trait to the PF, not the agent, for the same reason.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
This would solve the class of use cases like:<br>
<br>
>> So we have a specific use case: COMPUTE_TRUSTED_CERTS + NUMA <br>
<br>
but wouldn't help us for:<br>
<br>
> So what we need to solve is. Two (or more) sets of resources where the <br>
> different sets requires different, contradicting traits, in a setup <br>
> where the trait is not on the RP where resource inventory is.<br>
> <br>
> compute RP<br>
> |<br>
> |<br>
> |____ OVS agent RP<br>
> | | * CUSTOM_VNIC_TYPE_NORMAL<br>
> | |<br>
> | |___________ br-int dev RP<br>
> | * CUSTOM_PHYSNET_PHYSNET0<br>
> | * NET_BW_EGR_KILOBIT_PER_SEC: 1000<br>
> |<br>
> |<br>
> |____ SRIOV agent RP<br>
> | | * CUSTOM_VNIC_TYPE_DIRECT<br>
> | |<br>
> | |<br>
> | |___________ esn1 dev RP<br>
> | | * CUSTOM_PHYSNET_PHYSNET0<br>
> | | * NET_BW_EGR_KILOBIT_PER_SEC: 10000<br>
> | |<br>
> | |___________ esn2 dev RP<br>
> | * CUSTOM_PHYSNET_PHYSNET1<br>
> | * NET_BW_EGR_KILOBIT_PER_SEC: 20000<br>
> <br>
> <br>
> Then having two neutron ports in a server create request:<br>
> * port-normal:<br>
> "resource_request": {<br>
> "resources": {<br>
> orc.NET_BW_EGR_KILOBIT_PER_SEC: 1000},<br>
> "required": ["CUSTOM_PHYSNET0", "CUSTOM_VNIC_TYPE_NORMAL"]<br>
> <br>
> * port-direct:<br>
> "resource_request": {<br>
> "resources": {<br>
> orc.NET_BW_EGR_KILOBIT_PER_SEC: 2000},<br>
> "required": ["CUSTOM_PHYSNET0", "CUSTOM_VNIC_TYPE_DIRECT"]<br>
<br>
...unless we contrive some inventory unit to put on the agent RPs. What<br>
would that be? VNIC? How would we know how many to create?<br>
<br>
Interestingly, the above is closely approaching the space we're<br>
exploring for "subtree affinity". I'm wondering if there's a Unified<br>
Solution...<br></blockquote><div><br></div><div>yea, if we are going to create a virtual resource 'VM', then we need 'VNIC', and then we need more. I don't like that.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> For example, if we said that "traits always flow down [4]" (the<br>
> phrase that entered my brain and got me to start this email, "down"<br>
> in this case is "in the direction of children") then some traits<br>
> could be on the compute node, but expressed in a numbered request<br>
> group if that happened to be more convenient.<br>
> <br>
> This mental model works well for me, because nested often represents<br>
> a _containing_ hierarchy [2].<br>
> <br>
> If the "compute RP has no resources to give [...] but it's still the thing<br>
> exposing traits we want to filter by" [3], if we make it so the children<br>
> inherit those traits (because they have flowed down and the children<br>
> are "inside" the thing) things feel a bit saner to me. Would be good<br>
> if Eric were able to express in more detail why inherit feels<br>
> "terrible" [3]. It could very well be.<br>
<br>
I also said "feels". I can't really explain it any better than I could<br>
explain why "using group numbers as values" gave me the ooks. And given<br>
we're coming up ugly with all other proposals, convince me that this one<br>
is practical and not fraught with peril and I'll quickly get over my<br>
discomfort. Right now I'm pretty close to that point because it<br>
elegantly solves both classes of problem described above, and I can't<br>
think of a way to break it that isn't ridiculously contrived.<br>
<br>
It's possible we punted on it before because a) we didn't have the<br>
concrete use cases we have now; and b) it was going to be pretty tricky<br>
to implement. More on that below.<br>
<br>
> Similarly, aggregate membership would flow down as well, because a<br>
> child is always in its parent's aggregate too because it is inside its<br>
> parent.<br>
<br>
This one I'm not so convinced about. Can we defer making changes here<br>
until we have similarly concrete use cases?<br>
<br>
> A numeric requiredN or member_ofN span would be capped by the resource<br>
> provider that satisfied resourcesN.<br>
<br>
Eh? I was following you up to this point. Do you just mean that we don't<br>
have to worry about ascending the tree looking for requiredN because the<br>
trait is implicitly on the provider with resourceN by virtue of being on<br>
its ancestor?<br>
<br>
> We need to work out a consistent and relatively easy to explain<br>
> mental model for this, because we need to be able to talk about it<br>
> with other people without them being required to re-experience all<br>
> the mental hurdles we are having to overcome.<br>
<br>
I think the hurdles are more around "why" and "are you sure you want to"<br>
- once we've made those decisions, IMO it can be understood fairly<br>
easily with one or both of "encapsulation" and "traits flow down" as<br>
you've explained them.<br>
<br>
> [4] A corollary could be "classes of inventory always flow up": If<br>
> you need a SRIOV_NET_VF, this root resource provider can provide it<br>
> because it has a great grandchild which has it.<br>
<br>
This one bakes my noodle pretty good. I have a harder time visualizing<br>
how the above use cases are satisfied by walking backwards up the tree<br>
accumulating resources (and you have to accumulate the traits as well,<br>
right?) until I hit a point where I've gathered everything I need.<br>
<br>
So I'll come down in favor of making "traits flow down" happen. Question<br>
is, how? (And I know we've talked about this before - maybe Queens+Denver?)<br>
<br>
(A) In the database.<br>
(i) Any time a trait is added to a provider, we create records for<br>
same trait for all descendants.<br>
(ii) Need a data migration to bring existing data into conformance with ^<br>
(iii) When a trait is deleted from a provider, I assume we need to<br>
recursively delete it from all descendants. If you didn't want that,<br>
you'd have to go back and re-add it to the descendants you wanted it on.<br>
<br>
Pros: Easy to do. We don't have to change any of the APIs' algorithms -<br>
they just work the way we want them to by virtue of the trait data being<br>
where we want it. Reporting (e.g. GET /rps and therefore CLI output)<br>
reflects "reality".<br>
Cons: Irreversible. Not backward compatible. Can't do it in a microversion.<br>
<br>
(B) In the algorithms.<br>
(i) GET /rps and GET /a_cs queries need JOINs I can't even begin to<br>
comprehend.<br>
(ii) Do we tweak the outputs (GET /rps response and GET /a_cs<br>
provider_summaries) to report the "inherited" traits as well?<br>
<br>
Pros: Can do it in a microversion.<br>
Cons: See "can't even begin to comprehend". Maybe I'm a dunce.<br>
<br>
Perhaps this suggests a hybrid approach:<br>
<br>
(C) Create a "ghost" table of inherited resource provider traits. If<br>
$old_microversion we ignore it; if $new_microversion we logically<br>
combine it with the existing rp traits table in all our queries.<br>
<br>
Thoughts?<br>
<br>
efried<br>
.<br>
<br>
</blockquote></div></div>