CPU pinning blues

Albert Braden Albert.Braden at synopsys.com
Tue Nov 5 20:11:00 UTC 2019


I found the offending UUID in the nova_api and placement databases. Do I need to delete these entries from the DB or is there a safer way to get rid of the "phantom" VM?

MariaDB [(none)]> select * from nova_api.instance_mappings where instance_uuid = '4856d505-c220-4873-b881-836b5b75f7bb';
| created_at          | updated_at | id  | instance_uuid                        | cell_id | project_id                       | queued_for_delete |
| 2019-10-08 21:26:03 | NULL       | 589 | 4856d505-c220-4873-b881-836b5b75f7bb |    NULL | 474ae347d8ad426f8118e55eee47dcfd |                 0 |

MariaDB [(none)]> select * from nova_api.request_specs where instance_uuid = '4856d505-c220-4873-b881-836b5b75f7bb';
| created_at          | updated_at | id  | instance_uuid                        | spec                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| 2019-10-08 21:26:03 | NULL       | 589 | 4856d505-c220-4873-b881-836b5b75f7bb | {"nova_object.version": "1.11", "nova_object.changes": ["requested_destination", "instance_uuid", "retry", "num_instances", "pci_requests", "limits", "availability_zone", "force_nodes", "image", "instance_group", "force_hosts", "ignore_hosts", "numa_topology", "is_bfv", "user_id", "flavor", "project_id", "security_groups", "scheduler_hints"], "nova_object.name": "RequestSpec", "nova_object.data": {"requested_destination": null, "instance_uuid": "4856d505-c220-4873-b881-836b5b75f7bb", "retry": null, "num_instances": 1, "pci_requests": {"nova_object.version": "1.1", "nova_object.changes": ["requests"], "nova_object.name": "InstancePCIRequests", "nova_object.data": {"requests": []}, "nova_object.namespace": "nova"}, "limits": {"nova_object.version": "1.0", "nova_object.changes": ["vcpu", "memory_mb", "disk_gb", "numa_topology"], "nova_object.name": "SchedulerLimits", "nova_object.data": {"vcpu": null, "memory_mb": null, "disk_gb": null, "numa_topology": null}, "nova_object.namespace": "nova"}, "availability_zone": null, "force_nodes": null, "image": {"nova_object.version": "1.8", "nova_object.changes": ["status", "name", "container_format", "created_at", "disk_format", "updated_at", "id", "min_disk", "min_ram", "checksum", "owner", "properties", "size"], "nova_object.name": "ImageMeta", "nova_object.data": {"status": "active", "created_at": "2019-10-02T01:10:04Z", "name": "QSC-P-CentOS6.6-19P1-v4", "container_format": "bare", "min_ram": 0, "disk_format": "qcow2", "updated_at": "2019-10-02T01:10:44Z", "id": "200cb134-2716-4662-8183-33642078547f", "min_disk": 0, "checksum": "94d33caafd85b45519fca331ee7ea03e", "owner": "474ae347d8ad426f8118e55eee47dcfd", "properties": {"nova_object.version": "1.20", "nova_object.name": "ImageMetaProps", "nova_object.data": {}, "nova_object.namespace": "nova"}, "size": 4935843840}, "nova_object.namespace": "nova"}, "instance_group": null, "force_hosts": null, "ignore_hosts": null, "numa_topology": null, "is_bfv": false, "user_id": "2cb6757679d54a69803a5b6e317b3a93", "flavor": {"nova_object.version": "1.2", "nova_object.name": "Flavor", "nova_object.data": {"disabled": false, "root_gb": 35, "description": null, "flavorid": "e8b42da7-d352-441e-b494-77d6a6cd7366", "deleted": false, "created_at": "2019-09-23T21:19:50Z", "ephemeral_gb": 10, "updated_at": null, "memory_mb": 4096, "vcpus": 1, "extra_specs": {}, "swap": 3072, "rxtx_factor": 1.0, "is_public": true, "deleted_at": null, "vcpu_weight": 0, "id": 2, "name": "s1.1cx4g"}, "nova_object.namespace": "nova"}, "project_id": "474ae347d8ad426f8118e55eee47dcfd", "security_groups": {"nova_object.version": "1.1", "nova_object.changes": ["objects"], "nova_object.name": "SecurityGroupList", "nova_object.data": {"objects": [{"nova_object.version": "1.2", "nova_object.changes": ["name"], "nova_object.name": "SecurityGroup", "nova_object.data": {"name": "default"}, "nova_object.namespace": "nova"}]}, "nova_object.namespace": "nova"}, "scheduler_hints": {}}, "nova_object.namespace": "nova"} |
1 row in set (0.001 sec)

MariaDB [(none)]> SELECT * FROM placement.allocations WHERE consumer_id = '4856d505-c220-4873-b881-836b5b75f7bb';
| created_at          | updated_at | id   | resource_provider_id | consumer_id                          | resource_class_id | used |
| 2019-10-08 22:03:33 | NULL       | 3073 |                 1024 | 4856d505-c220-4873-b881-836b5b75f7bb |                 0 |    1 |
| 2019-10-08 22:03:33 | NULL       | 3074 |                 1024 | 4856d505-c220-4873-b881-836b5b75f7bb |                 1 | 4096 |
| 2019-10-08 22:03:33 | NULL       | 3075 |                 1024 | 4856d505-c220-4873-b881-836b5b75f7bb |                 2 |   48 |
3 rows in set (0.001 sec)

MariaDB [(none)]> SELECT * FROM placement.consumers WHERE uuid = '4856d505-c220-4873-b881-836b5b75f7bb';
| created_at          | updated_at          | id  | uuid                                 | project_id | user_id | generation |
| 2019-10-08 22:03:33 | 2019-10-08 22:03:33 | 734 | 4856d505-c220-4873-b881-836b5b75f7bb |          1 |       1 |          1 |
1 row in set (0.000 sec)


From: Albert Braden <Albert.Braden at synopsys.com<mailto:Albert.Braden at synopsys.com>>
Sent: Thursday, October 31, 2019 10:50 AM
To: openstack-discuss at lists.openstack.org<mailto:openstack-discuss at lists.openstack.org>
Subject: CPU pinning blues

I'm following this document to setup CPU pinning on Rocky:

https://www.redhat.com/en/blog/driving-fast-lane-cpu-pinning-and-numa-topology-awareness-openstack-compute

I followed all of the steps except for modifying non-pinned flavors and I have one aggregate containing a single NUMA-capable host:

root at us01odc-dev1-ctrl1:/var/log/nova# os aggregate list
+----+-------+-------------------+
| ID | Name  | Availability Zone |
+----+-------+-------------------+
|  4 | perf3 | None              |
+----+-------+-------------------+
root at us01odc-dev1-ctrl1:/var/log/nova# os aggregate show 4
+-------------------+----------------------------+
| Field             | Value                      |
+-------------------+----------------------------+
| availability_zone | None                       |
| created_at        | 2019-10-30T23:05:41.000000 |
| deleted           | False                      |
| deleted_at        | None                       |
| hosts             | [u'us01odc-dev1-hv003']    |
| id                | 4                          |
| name              | perf3                      |
| properties        | pinned='true'              |
| updated_at        | None                       |
+-------------------+----------------------------+

I have a flavor with the NUMA properties:

root at us01odc-dev1-ctrl1:/var/log/nova# os flavor show s1.perf3
+----------------------------+-------------------------------------------------------------------------+
| Field                      | Value                                                                   |
+----------------------------+-------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled   | False                                                                   |
| OS-FLV-EXT-DATA:ephemeral  | 0                                                                       |
| access_project_ids         | None                                                                    |
| disk                       | 35                                                                      |
| id                         | be3d21c4-7e91-42a2-b832-47f42fdd3907                                    |
| name                       | s1.perf3                                                                |
| os-flavor-access:is_public | True                                                                    |
| properties                 | aggregate_instance_extra_specs:pinned='true', hw:cpu_policy='dedicated' |
| ram                        | 30720                                                                   |
| rxtx_factor                | 1.0                                                                     |
| swap                       | 7168                                                                    |
| vcpus                      | 4                                                                       |
+----------------------------+-------------------------------------------------------------------------+

I create a VM with that flavor:

openstack server create --flavor s1.perf3 --image NOT-QSC-CentOS6.10-19P1-v4 --network it-network alberttest4

but it goes to error status, and I see this in the logs: ***

*** Post with logs got moderated so they are here:

https://paste.fedoraproject.org/paste/3bza6CJstXFPy8LatRJruA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openstack.org/pipermail/openstack-discuss/attachments/20191105/661bdd3b/attachment-0001.html>


More information about the openstack-discuss mailing list