[watcher] Question about baremetal node support in nova CDM
While working on [1] I noticed that the NovaHelper.get_compute_node_by_hostname method is making 3 API calls to get the details about a hypervisor: 1. listing hypervisors and then filtering client-side by the compute service host - this fails if there is not exactly one hypervisor for the given compute service host name 2. search for the hypervisor given the hypervisor_hostname to get the id 3. get the hypervisor details using the id My patch is collapsing 2 and 3 into a single API call to nova. The question I had was if we need that first call. For all non-ironic baremetal nodes, the compute service host and node hypervisor_hostname should be identical, so technically we could just search for the hypervisor details with the compute service hostname. Only in the case of ironic would we potentially get more than one hypervisor (node) for a single compute service (host). I don't think the nova CDM code really handles baremetal instances at all because it's not handling this kind of host:node 1:M cardinality elsewhere in the code, but you also can't do most things to optimize like cold or live migrating baremetal instances. I'm not exactly sure how Watcher deals with ironic but I know there is the separate BaremetalClusterDataModelCollector so I'm assuming watcher just optimizes for baremetal outside of the nova compute API? If this is true, then we can get rid of that first API all noted above and I don't need to write a nova spec to add a host filter parameter to the GET /os-hypervisors/detail API. [1] https://review.opendev.org/#/c/661121/2/watcher/common/nova_helper.py@65 -- Thanks, Matt
Hello, I haven't investigated thoroughly but I suspect that the bare metal nodes returned from the hypervisor calls are not handled by the nova CDM just like you expect. I think we should look into if bare metal nodes are stored in the compute_model as I think it would more sense to filter them out. Overall I think the way Watcher handles bare metal nodes must be analysed and improved for example the saving energy strategy uses the ironic client directly and there is currently not a single strategy that accesses the baremetal_model. Maybe I have time to test my suspicions in a test environment with both ironic and hypervisor nodes next week. On 5/23/19 11:40 PM, Matt Riedemann wrote:
While working on [1] I noticed that the NovaHelper.get_compute_node_by_hostname method is making 3 API calls to get the details about a hypervisor:
1. listing hypervisors and then filtering client-side by the compute service host - this fails if there is not exactly one hypervisor for the given compute service host name
2. search for the hypervisor given the hypervisor_hostname to get the id
3. get the hypervisor details using the id
My patch is collapsing 2 and 3 into a single API call to nova.
The question I had was if we need that first call. For all non-ironic baremetal nodes, the compute service host and node hypervisor_hostname should be identical, so technically we could just search for the hypervisor details with the compute service hostname.
Only in the case of ironic would we potentially get more than one hypervisor (node) for a single compute service (host).
I don't think the nova CDM code really handles baremetal instances at all because it's not handling this kind of host:node 1:M cardinality elsewhere in the code, but you also can't do most things to optimize like cold or live migrating baremetal instances.
I'm not exactly sure how Watcher deals with ironic but I know there is the separate BaremetalClusterDataModelCollector so I'm assuming watcher just optimizes for baremetal outside of the nova compute API?
If this is true, then we can get rid of that first API all noted above and I don't need to write a nova spec to add a host filter parameter to the GET /os-hypervisors/detail API.
[1] https://review.opendev.org/#/c/661121/2/watcher/common/nova_helper.py@65
On 5/24/2019 1:39 AM, info@dantalion.nl wrote:
I think we should look into if bare metal nodes are stored in the compute_model as I think it would more sense to filter them out.
The tricky thing with this would be there isn't a great way to identify a baremetal node from a kvm node, for example. There is a hypervisor_type column on the compute_nodes table in the nova cell DB, but it's not exposed in the API. Two obvious differences would be: 1. The hypervisor_hostname on an ironic node in the os-hypervisors API is a UUID rather than a normal hostname. That could be one way to try and identify an ironic node (hypervisor). 2. For servers, the associated flavor should have a CUSTOM resource class extra spec associated with it and the VCPU, DISK_GB, and MEMORY_MB resource classes should also be zero'ed out in the flavor per [1]. The server OS-EXT-SRV-ATTR:hypervisor_hostname field would also be a UUID like above (the UUID is the ironic node ID). [1] https://docs.openstack.org/ironic/latest/install/configure-nova-flavors.html -- Thanks, Matt
If the hypervisor_type is only available in the nova cell DB how does the OpenStackClient determine the hypervisor type? If I execute 'openstack hypervisor list' then the Hypervisor Type column is visible and correctly distinguishes between ironic and qemu hyervisors. On 5/24/19 9:27 PM, Matt Riedemann wrote:
On 5/24/2019 1:39 AM, info@dantalion.nl wrote:
I think we should look into if bare metal nodes are stored in the compute_model as I think it would more sense to filter them out.
The tricky thing with this would be there isn't a great way to identify a baremetal node from a kvm node, for example. There is a hypervisor_type column on the compute_nodes table in the nova cell DB, but it's not exposed in the API.
Two obvious differences would be:
1. The hypervisor_hostname on an ironic node in the os-hypervisors API is a UUID rather than a normal hostname. That could be one way to try and identify an ironic node (hypervisor).
2. For servers, the associated flavor should have a CUSTOM resource class extra spec associated with it and the VCPU, DISK_GB, and MEMORY_MB resource classes should also be zero'ed out in the flavor per [1]. The server OS-EXT-SRV-ATTR:hypervisor_hostname field would also be a UUID like above (the UUID is the ironic node ID).
[1] https://docs.openstack.org/ironic/latest/install/configure-nova-flavors.html
On 5/29/2019 4:10 AM, info@dantalion.nl wrote:
If the hypervisor_type is only available in the nova cell DB how does the OpenStackClient determine the hypervisor type?
If I execute 'openstack hypervisor list' then the Hypervisor Type column is visible and correctly distinguishes between ironic and qemu hyervisors.
I was mistaken, hypervisor_type is in the API response [1] so that's why you're seeing it in the OSC CLI. I must have missed it when looking at the API reference. [1] https://github.com/openstack/nova/blob/d4f58f5eb6e68d0348868efc45212b70feb0b... -- Thanks, Matt
refer to the comment by hidekazu, there is the case 1:M host:nodes when VMware vCenter driver is used. We can remove the first API call if we restrict Nova driver such as KVM only. Thanks licanwei
On 5/26/2019 8:52 PM, li.canwei2@zte.com.cn wrote:
refer to the comment by hidekazu, there is the case 1:M host:nodes when VMware vCenter driver is used.
As mentioned in the commit message, this hasn't been the case for the vCenter driver in nova since kilo: https://review.opendev.org/#/c/103916/ -- Thanks, Matt
Thank you Matt, then I think we can remove the first API call. 主 题 ：Re: [watcher] Question about baremetal node support in nova CDM On 5/26/2019 8:52 PM, li.canwei2@zte.com.cn wrote:
refer to the comment by hidekazu, there is the case 1:M host:nodes when VMware vCenter driver is used.
As mentioned in the commit message, this hasn't been the case for the vCenter driver in nova since kilo: https://review.opendev.org/#/c/103916/ -- Thanks, Matt
On 5/29/2019 4:32 AM, li.canwei2@zte.com.cn wrote:
then I think we can remove the first API call.
OK, cool. I'll do it in a separate follow up change so it doesn't make the existing patch more complicated and because then we can revert it if it causes a problem without reverting the whole other optimization. -- Thanks, Matt
On 5/23/2019 4:40 PM, Matt Riedemann wrote:
Only in the case of ironic would we potentially get more than one hypervisor (node) for a single compute service (host).
As I was working on the change, I realized that while this is true, it's not strictly what this API could return: GET /os-hypervisors/detail?hypervisor_hostname_pattern=$node_hostname The problem is that is a fuzzy match in the DB query [1] so if you have some computes like this: compute1 compute10 compute100 And we're searching on compute1 we could get back 3 hypervisors which would break on the watcher side, so we can't really use that. There was a similar problem in some nova CLIs [2] which essentially does the same kind of strict hostname checking that watcher already does. So unfortunately this means I can't replace that code in watcher just yet. But if the compute API adds a service host filter to the hypervisors API we could push the strict service hostname filtering into the server rather than the client (watcher). I can draft a nova spec for that. [1] https://github.com/openstack/nova/blob/d4f58f5eb6e68d0348868efc45212b70feb0b... [2] https://review.opendev.org/#/c/520187/ -- Thanks, Matt
participants (3)
-
info＠dantalion.nl
-
li.canwei2＠zte.com.cn
-
Matt Riedemann