Open Stack

On 02/12/2015 05:15 PM, Trevor McKay wrote:
> Hi folks,
>
> Here is another way to do this.  Lu had mentioned Oozie shell actions
> previously.
> Sahara doesn't support them, but I played with it from the Oozie command
> line
> to verify that it solves our hbase problem, too.
>
> We can potentially create a blueprint to build a simple Shell action
> around a
> user-supplied script and supporting files.  The script and files would
> all be stored
> in Sahara as job binaries (Swift or internal db) and referenced the same
> way. The exec
> target must be on the path at runtime, or included in the working dir.
>
> To do this, I simply put workflow.xml, doit.sh, and the test jar into
> a directory in hdfs.  Then I ran it with the Oozie cli using the job.xml
> config file
> configured to point at the hdfs dir.  Nothing special here, just
> standard Oozie
> job execution.


very cool Trevor, i wonder if there is a greater pattern here we could 
identify. namely, the idea that a user could upload multiple job 
binaries and create some sort of nesting structure to the way they are 
interpreted. maybe there could be a standard substitution method for 
script like files. in this respect a user could create a standard 
wrapper script and allow different binary names to be substituted into 
the script. this may be too complicated but it occurred to me while 
reading your results.

regardless of the greater pattern, this is a good window into more ways 
for us to control the command arguments.

mike