<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi,<br>
The Cyborg agent in a compute node collects information about
devices from the Cyborg drivers on that node. It then needs to push
that information to the Cyborg conductor in the controller, which
then needs to persist it in the Cyborg db and update Placement.
Further, the agent needs to collect and update this information
periodically (or possibly in response to notifications) to handle
hot add/delete of devices, reprogramming (for FPGAs), health failure
of devices, etc. <br>
<br>
In this morning's call, we discussed how to do this periodic update
[1]. In particular, we talked about how to compute the difference
between the previous device configuration in a compute node and the
current one, whether the agent do should do that diff or the
controller, etc. Since there are many fields per device, and they
are tree-structured, the complexity of doing the diff seemed large.<br>
<br>
On taking a closer look, however, the amount of computation needed
to do the update is not huge. Say, for discussion's sake, that the
controller has a snapshot of the entire device config for a specific
compute node, i.e. an array of device structures NewConfig[]. It
reads the current list of devices for that node from the db,
CurrentConfig[]. Then the controller's logic is like this:<br>
<ul>
<li>Determine the list of devices in NewConfig[] but not in
CurrentConfig[] (this is a set difference in Python [2]): they
are the newly added ones. For each newly added device, do a
single transaction to add all the fields to the db together.</li>
<li>Determine the list of devices in CurrentConfig[] but not in
NewConfig[]: they are the deleted devices.For each such device,
do a single transaction to delete that entry.<br>
</li>
<li>For each modified device, compute what has changed, and update
that alone. This is the per-field diff. <br>
</li>
</ul>
Say each field in the device structure is a string of 100
characters, and it takes 1 nanosecond to add, delete or modify a
character. So, each field takes 100 ns to update
(add/delete/modify). Say 20 fields per device: so 2 us to add,
delete or modify a device. Say 10 devices per compute node: so 20 us
per node. 500 nodes will take 10 milliseconds. So, if each node
sends a refresh every second, the controller will spend a very small
fraction of that time in updating the db, even including transaction
costs, set difference computation, etc. <br>
<br>
This back-of-the-envelope calculation shows that we need not try to
optimize too early: the agent should send the entire device config
over to the controller, and let it update the db per-device and
per-field.<br>
<br>
[1] <a class="moz-txt-link-freetext" href="https://etherpad.openstack.org/p/cyborg-rocky-development">https://etherpad.openstack.org/p/cyborg-rocky-development</a> <br>
[2] <a class="moz-txt-link-freetext" href="https://docs.python.org/2/library/sets.html">https://docs.python.org/2/library/sets.html</a><br>
<br>
Regards,<br>
Sundar<br>
</body>
</html>