[nova] implementation options for nova spec: show-server-numa-topology
Hi, guys This spec needs input and discuss for move on. Currently the spec is under reviewing: https://review.openstack.org/#/c/612256/8 Plus with POC code: https://review.openstack.org/#/c/621476/3 and related stein PTG discuss: https://etherpad.openstack.org/p/nova-ptg-stein start from line 897 NUMA topology had lots of information to expose, for saving you time to jumping into to the spec, the information need to return include NUMA related like: numa_node,cpu_pinning,cpu_thread_policy,cpuset,siblings, mem,pagesize,sockets,cores, threads, and PCI device's information. Base on IRC's discuss, we may have 3 options about how to deal with those blobs: 1) include those directly in the server response details, like the released POC does: https://review.openstack.org/#/c/621476/3 2) add a new sub-resource endpoint to servers, most likely use key word 'topology' then: "GET /servers/{server_id}/topology" returns the NUMA information for one server. 3) put the NUMA info under existing 'diagnostics' API. "GET /servers/{server_id}/diagnostics" this is admin only API, normal user loss the possible to check their topology. when the information put into diagnostics, they will be look like: { .... "numa_topology": { cells [ { "numa_node" : 3 "cpu_pinning": {0:5, 1:6}, "cpu_thread_policy": "prefer", "cpuset": [0,1,2,3], "siblings": [[0,1],[2,3]], "mem": 1024, "pagesize": 4096, "sockets": 0, "cores": 2, "threads": 2, }, ... ] # cells } "emulator_threads_policy": "share" "pci_devices": [ { "address":"00:1a.0", "type": "VF", "vendor": "8086", "product": "1526" }, ] } Regards Yongli He
On 12/18/2018 2:20 AM, yonglihe wrote:
I would think these are potentially admin-level sensitive details as well and thus only exposed based on a policy check. A user requests a certain topology, but I'm not sure how low-level the user needs/should see what nova is doing for satisfying that topology, especially for things like pinning CPUs on the host. I thought the main use case for this spec (and several like these discussed at the PTG) was more about being able to get information (reporting) out of the REST API for debugging (by admins and/or support team members), less about user need.
Similar to (1) in that I'd think there would be a new policy check on this which defaults to admin-only. I think this would be better than (1) since it wouldnt' be confused with listing servers (GET /servers/detail).
By default it's an admin-only API, but that is configurable in policy, so if a given cloud wants to expose this for admin or owner of the instance, they can do that, or alternatively expose it to support team members via a special role in keystone.
I tend to prefer option (3) since it seems topology is a much more natural fit with the existing information (low-level CPU/RAM/disk usage) we expose out of the diagnostics API and is already restricted to admins by default in policy (but again as noted this can be configured). -- Thanks, Matt
On 12/20/2018 7:13 PM, yonglihe wrote:
I also commented in the spec today. I would also be OK(ish) with option 2. I'm mostly concerned about the performance implications of needing to fetch and process this data, including policy checks, when listing 1000 servers with details. The spec wasn't clear (to me) about where the data comes from exactly (do we join on the compute_nodes table?). I'm also unsure about how much end users need to see the NUMA/PCI information for their server (so is the admin-only policy sufficient for the diagnostics API?). I'd really like input from others here. I mostly just want users to have to opt in to getting this information, not nova needing to produce it in the main server resource response during show/list, so option 2 or 3 are preferable *to me*. I think option 3 is the safest one if we're unsure or deadlocked otherwise, but no one else has really said anything (outside of the spec anyway). -- Thanks, Matt
On 2018/12/21 上午9:28, Matt Riedemann wrote:
Spec patch set 10 address all your comments, and thanks a lot for all typo things. Spec path set 11, switch to implementation option 2, cause the data did come from another DB query, it's obviously impact the performance some how, especially on a batch operations. thanks. Regards Yongli he
On 2018/12/18 下午4:20, yonglihe wrote:
Hi, guys
This spec needs input and discuss for move on.
Jay suggest we might be good to use a new sub node to hold topology stuff, it's option 2, here. And split the PCI stuff out of this NUMA thing spec, use a /devices node to hold all 'devices' stuff instead, then this node is generic and not only for PCI itself. I'm OK for Jay's suggestion, it contains more key words and seems crystal clear and straight forward. The problem is we need aligned about this. This spec need gain more input thanks, Jay, Matt. Regards Yongli He
On 01/02/2019 10:15 PM, yonglihe wrote:
Also, I mentioned that you need not (IMHO) combine both PCI/devices and NUMA topology in a single spec. We could proceed with the /topology API endpoint and work out the more generic /devices API endpoint in a separate spec. Best, -jay
On 1/3/2019 6:39 AM, Jay Pipes wrote:
I said earlier in the email thread that I was OK with option 2 (sub-resource) or the diagnostics API, and leaned toward the diagnostics API since it was already admin-only. As long as this information is admin-only by default, not part of the main server response body and therefore not parting of listing servers with details (GET /servers/detail) then I'm OK either way and GET /servers/{server_id}/topology is OK with me also. -- Thanks, Matt
participants (3)
-
Jay Pipes
-
Matt Riedemann
-
yonglihe