On 18/12/2024 08:21, Junya Noguchi (Fujitsu) wrote:
Dear Mr. Stanley
Thank you for your response.
I understand that I can define custom resources using Placement and Provider Config[1][2]. I've also confirmed that Intel RMD (Resource Manager Daemon) is mentioned as a use case, but it seems there's no RMD integration for OpenStack[3].
I have a further question. When defining custom resources using Placement and Provider Config, does the service responsible for resource allocation and update of allocations to instances need to be implemented independently by the creator of the custom resource? Alternatively, does a mechanism exist within OpenStack components (e.g., Nova) to support this? For example, is there a function that calls a custom hook function when creating a domain in nova-compute?
I'm struggling to decide on the implementation approach. Should the resource allocation process be handled as a plugin (separable from the OpenStack codebase), or should I modify the libvirt driver to add the processing for allocating LLC and memory bandwidth?
nova does not allow any external process to modify the xml of an instance. so eitehr the cache allocation needs to be done entirly externally of nova without modifying the xml via an external agent via something like the kernel resoucectl virtual file system or nova need to be modifed to staticaly configure it vai the xml. we do not supprot any kind of hook point or plugin workflow to modify the domain xml. We also do not currently plan to exted the provider config feature to allow you register a dynamic handler for the custom resouce. ultimately im not convinced we should add CAT/RDT type functionality in the near term. The RDT technology is really not well designed. intel CAT (cache allocation technology) shoudl be called CCT (Cache Confinmenet Technology) but that does not have the same marketing appeal. instead of allcoatiog cache ways to processes and the hardware preventing other proicess form using that cache you are actually confining a process to a set of cache ways prevetnign it form using other cacheways. other processes are free to use the cache ways you configed the process too as well. there is no isolation at the hardware level so you can still get noisy neighbour effect form unconfied proceses. that mean you need to first reserve cache ways for exclusive use by openstack (preventing all host proces form usign them) and then config individual instance to the now free cache ways. the same applies to intels memomory bandwith allcoation feature. At least that was the case 5-6 years ago when i last worked on this. That makes this feature quite unsuited for a cloud envionment and very hared to integrated Into nova static resouce allocation architure. That why we proposed two versions originally, 1.) a simple static allcoation modle via libvirt. This ultimately did not really deliver enough value to warrant complexity. 2 external dynamic agent like RMD. ultimatly RMD was a slignly nicer approch even if it was very racy in that the RMD demaon could both mange the cache asignement of the background system processes and the vm but then you end up with nova/libvirt and RMD seperatlly trying to optimise the placemtn of the vm without knowwign about each other. im not saying that there is no way to supprot this in nova but it would be very difficult to do so. likely its more then one cycles works. before talking about an implementation i think it would be better to reflect on what because your trying to enable that migh inform a desing approch and the limitation of CAT/RDT may mean that it will never fullfile your usecase.
[1] https://docs.openstack.org/nova/latest/admin/managing-resource-providers.htm... [2] https://docs.openstack.org/placement/latest/ [3] https://networkbuilders.intel.com/docs/networkbuilders/resource-management-d... 3.1
Best regards, Junya Noguchi.