<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:109590793;
mso-list-type:hybrid;
mso-list-template-ids:-1915453446 1732283286 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:20;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Calibri",sans-serif;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi guys,<o:p></o:p></p>
<p class="MsoNormal">I’m implementing the Watcher Scoring Module. As part of that, I need to expose the information about Scoring Engines through the API/Python CLI.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">The scoring engine list might be quite dynamic. Although the scoring engines will be pluggable through the stevedore plug-in model, a single plug-in might contain one or more scoring engines. In some scenarios this list will be static –
a plug-in developer will just expose few algorithms and that’s it. But in some other scenarios, the scoring engines might be implemented as external web services for example and there might be an on-going development process on data models, which will result
in multiple scoring engines in multiple versions, which might change quite frequently (e.g. few times a day).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Of course, the responsibility for handling all of that is entirely on the scoring engine plug-in developer. But it would be good to keep the scoring engine abstraction layer clean and simple, hiding all of these details.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">And here comes the problem:<o:p></o:p></p>
<p class="MsoNormal">Somehow the dynamic list of scoring engines has to be passed from Decision Engine (where the Scoring Engine abstraction layer will be sitting) to the Api / CLI. There are currently 2 options on the table how this could be done:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Option 1:<o:p></o:p></p>
<p class="MsoNormal">Allow Api to call Decision Engine directly through existing RPC Api (currently using messaging transport).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Option 2:<o:p></o:p></p>
<p class="MsoNormal">Let Decision Engine keep Scoring Engine information synced in the DB so that Watcher Api can simply query for this information as required.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Pros and cons of each option:<o:p></o:p></p>
<p class="MsoNormal">Option 1:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: Simpler implementation and no need for keeping DB in sync.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: No risk of data inconsistency. Nothing is being cached, data is always accurate. Decision Engine is a single source of truth.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: Scoring Engine Plug-in creates a simple stevedore plug-in, implements scoring engine classes, implements a factory class returning scoring engines and that’s all.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: Supports also more complicated scenarios with dynamic scoring engine list – encapsulated in the factory class.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: Dependency on Decision Engine – it needs to be up and running. Can be mitigated by caching the last response from Decision Engine – if DE RCP Api is not responding, the last known data could be returned.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: Not sure how reliable/performant RPC over messaging transport is. Need to test.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: Might have scalability issues (I believe there is only one Decision Engine instance, please confirm!). But this might be at least partially mitigated by caching on the Watcher Api level (e.g. if the last data was retrieved less
than X minutes ago, no need to query Decision Engine). In the context that this information is only used by Strategy developers to actually implement strategies using some Scoring Engines, it might be perfectly fine to cache data for longer periods of time
(1 hour or more).<o:p></o:p></p>
<p class="MsoNormal">Option 2:<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: Watcher Api decoupled from Decision Engine. Can work even if DE is not working or busy.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Good: In case of Watcher this option should scale better. Decision Engine typically has only one instance and is not subject to horizontal scalability (please confirm my understanding!).<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: More complicated implementation. For dynamic scenarios (adding scoring engines on the fly) requires some sort of notification mechanism, so that the DB will stay in sync. Can be done by exposing event handling in scoring engine
abstraction layer, but it’s unnecessary complication for simple cases with static data. But can be mitigated by using helper classes enforcing DB sync without actually exposing any events in the abstract classes (so if plug-in needs to sync DB, it calls some
helper method, all others just do nothing).<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: Potential issues with data consistency. If there is a problem or a bug in the sync code, it might be hard to recover from the problem without Watcher redeployment.<o:p></o:p></p>
<p class="MsoListParagraph" style="text-indent:-.25in;mso-list:l0 level1 lfo1"><![if !supportLists]><span style="mso-list:Ignore">-<span style="font:7.0pt "Times New Roman"">
</span></span><![endif]>Bad: Any change in the DB structure might require to change all the parties and even the existing plug-ins.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My preference is to go with option 1 because of the simpler implementation and no problems with data consistency. Nothing needs to be purged, synced, data is always accurate. If Decision Engine is not working, there is a bigger problem
anyway (but there is a mitigation by caching the DE last response).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I hope I managed to explain the concept and the problem.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I appreciate your opinion about that!<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Kind regards,<o:p></o:p></p>
<p class="MsoNormal">Tomasz<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p>---------------------------------------------------------------------<br>
<strong style="line-height: 11.25pt;"><span style="font-size: 9pt; color:
#595959;"><span style="font-family: 'Arial Narrow', sans-serif;">Intel
Technology Poland sp. z o.o.<br></span></span></strong><span style="color:
#595959; font-family: 'Arial Narrow', sans-serif; font-size: 9pt; line-height:
11.25pt;">ul. Słowackiego 173 | 80-298 Gdańsk | Sąd Rejonowy Gdańsk
Północ
| VII Wydział Gospodarczy Krajowego Rejestru Sądowego - KRS 101882 | NIP
957-07-52-316 | Kapitał zakładowy 200.000 PLN.</span></p><p>
<span style="font-size:8.0pt;font-family:"Arial
Narrow","sans-serif";
mso-fareast-font-family:"Times New
Roman";mso-bidi-font-family:Arial;
color:#595959;mso-ansi-language:EN-US;mso-fareast-language:EN-US;mso-bidi-language:
AR-SA">Ta wiadomość wraz z załącznikami jest przeznaczona dla określonego
adresata i może zawierać informacje poufne. W razie przypadkowego otrzymania
tej wiadomości, prosimy o powiadomienie nadawcy oraz trwałe jej usunięcie;
jakiekolwiek przeglądanie lub rozpowszechnianie jest zabronione.<br>
This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). If you are not the intended recipient,
please
contact the sender and delete all copies; any review or distribution by others
is strictly prohibited.</span></p><p class="MsoNormal"><o:p></o:p></p>
</body>
</html>