<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:PMingLiU;
panose-1:2 2 5 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@PMingLiU";
panose-1:2 2 5 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">Hi all.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We got several feedbacks about Sahara EDP’s future from some China customers.<o:p></o:p></p>
<p class="MsoNormal">Here are some ideas we would like to share with you and need your input if we can implement them in Sahara(Liberty).<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">1. Add a schedule feature to run the jobs on time:<o:p></o:p></p>
<p class="MsoNormal">This request comes from the customer, they usually run the job in a specific time every day. So it should be great if there is a scheduler to help arrange the regular job to run.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">2. A more complex workflow design in Sahara EDP:<o:p></o:p></p>
<p class="MsoNormal">Current EDP only provide one job that is running on one cluster.
<o:p></o:p></p>
<p class="MsoNormal">But in a real case, it should be more complex, they usually use multiple jobs to calculate the data and may use several different type clusters to process it.<o:p></o:p></p>
<p class="MsoNormal">For example: Raw Data -> Job A(Cluster A) -> Job B(Cluster B) -> Job C(Cluster A) -> Result<o:p></o:p></p>
<p class="MsoNormal">Actually in my opinion, this kind of job could be easy to implement by using Oozie as a workflow engine. But for current EDP, it doesn’t implement this kind of complex case.<o:p></o:p></p>
<p class="MsoNormal">Another concern is about Spark, for Spark it cannot use Oozie to do this. So we need to create an abstract layer to help to implement this kind of scenarios.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">However, any suggestion is welcome. <o:p></o:p></p>
<p class="MsoNormal">Thanks.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>