<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 TRANSITIONAL//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; CHARSET=UTF-8">
<META NAME="GENERATOR" CONTENT="GtkHTML/4.8.5">
</HEAD>
<BODY>
Hi,<BR>
<BR>
Thanks for your patience. I have been consumed with spark-swift, but I can start to address these questions now :)<BR>
<BR>
On (1) (a) below, I will try to reproduce and look at how we can better support classpath in EDP. I'll let you know what I find.<BR>
We may need to add some configuration options for EDP or change how it works.<BR>
<BR>
On (1) (b) below, in the edp-move-examples.rst spec for Juno we described a directory structure that could be used<BR>
for separating hadoop1 vs hadoop2 specific directories. Maybe we can do something similar based on plugins<BR>
<BR>
For instance, if we have some hbase examples, we can make subdirectories for each plugin. Common parts can be<BR>
shared, plugin-specific files can be stored in the subdirectories.<BR>
<BR>
(and perhaps the "hadoop2" example already there should just be a subdirectory under "edp-java")<BR>
<BR>
Best,<BR>
<BR>
Trevor<BR>
<BR>
--<BR>
<BR>
Hi McKay<BR>
Thx for your support<BR>
I will talk details of these items as below:<BR>
<BR>
(1) EDP job in Java action<BR>
<BR>
The background is that we want write integration test case for newly added services like HBase, zookeeper just like the way the edp-examples does( sample code under sahara/etc/edp-examples/). So I thought I can wrote an example via edp job by Java action to test HBase Service, then I wrote the HBaseTest.java and packaged as a jar file, and run this jar manually with the command "java -cp `hbase classpath` HBaseTest.jar HBaseTest", it works well in the vm(provisioned by sahara with cdh plugin).<BR>
“/usr/lib/jvm/java-7-oracle-cloudera/bin/java -cp "HBaseTest.jar:`hbase classpath`" HBaseTest”<BR>
So I want run this job via horizon in sahara job execution page, but found no place to pass the `hbase classpath` parameter.(I have tried java_opt and configuration and args, all failed). When I pass the “-cp `hbase classpath`” to java_opts in horizon job execution page. Oozie raise this error as below<BR>
<BR>
“2015-01-15 16:43:26,074 WARN org.apache.oozie.action.hadoop.JavaActionExecutor: SERVER[hbase-master-copy-copy-001.novalocal] USER[hdfs] GROUP[-] TOKEN[] APP[job-wf] JOB[0000045-150105050354389-oozie-oozi-W] ACTION[0000045-150105050354389-oozie-oozi-W@job-node] LauncherMapper died, check Hadoop LOG for job [hbase-master-copy-copy-001.novalocal:8032:job_1420434100219_0054]<BR>
2015-01-15 16:43:26,172 INFO org.apache.oozie.command.wf.ActionEndXCommand: SERVER[hbase-master-copy-copy-001.novalocal] USER[hdfs] GROUP[-] TOKEN[] APP[job-wf] JOB[0000045-150105050354389-oozie-oozi-W] ACTION[<A HREF="mailto:0000045-150105050354389-oozie-oozi-W@job-node">0000045-150105050354389-oozie-oozi-W@job-node</A>] ERROR is considered as FAILED for SLA”<BR>
<BR>
So I stuck with this issue, I can’t write the integration test in sahara ( could not pass the classpath parameter), <BR>
I have check oozie official site . <A HREF="https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook">https://cwiki.apache.org/confluence/display/OOZIE/Java+Cookbook</A> found no help info.<BR>
<BR>
So about the EDP job in java, I have two problems right now:<BR>
a) How to pass classpath to java action as I mention before. So this also reminds me that we can allow user to modify or upload this own workflow.xml, then we can provide more options for user.<BR>
b) I concern that it’s hard to have a common edp-example for HBase for all plugin(cdh dhp), because the example code depends on third party jars(for example hbase-client.jar…) and different platform(CDH HDP) they may have different version hbase-client.jar, for example, cdh use hbase-client-0.98.6-cdh5.2.1.jar. <BR>
<BR>
attached is a zip file which contains HBaseTest.jar and the source code.
</BODY>
</HTML>