accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cornish, Duane C." <Duane.Corn...@jhuapl.edu>
Subject RE: pragmattically invoking map reduce job from a .war deployed in tomcat
Date Mon, 11 Feb 2013 15:14:22 GMT
Thanks, Bob.  I'll try this out as well.  When you say you deploy your job jars, are you just
referring to a jar of the entire project (including the map reduce jobs), or do you just jar
up the specific packages that contain the map reduce jobs, mappers, and reducers?  Also, am
I correct in assuming that you're not making runnable jars?

Thanks,
Duane





From: Bob.Thorman@l-3com.com [mailto:Bob.Thorman@l-3com.com]
Sent: Friday, February 08, 2013 2:26 PM
To: user@accumulo.apache.org
Subject: RE: pragmattically invoking map reduce job from a .war deployed in tomcat

I do this same thing, but during deployment of my war I also deploy my job jars and their
dependencies to $ACCUMULO_HOME\lib\ext to make sure they are available to the ensemble.

From: Cornish, Duane C. [mailto:Duane.Cornish@jhuapl.edu]
Sent: Friday, February 08, 2013 11:25
To: user@accumulo.apache.org<mailto:user@accumulo.apache.org>
Subject: pragmattically invoking map reduce job from a .war deployed in tomcat

I have an accumulo instance running on a small 4 node cluster.  The namenode is running tomcat.
 I am using a restful service to allow clients to access accumulo and kick off map reduce
jobs.

Previously, I had been creating runnable jars of my map reduce jobs.  The restful service
would make a command line call to run the jar.  i.e. "java -jar mapreducejob.java -arg1 -arg2
etc."

I am now trying to switch these jobs to be called programmatically.  I am aware that I need
to pass a comma separated list of jars into the ToolRunner.run call.  i.e.

arg[0] = "-libjar";
arg[1] = "accumulo-core-1.4.1,jar2" etc.
arg[2] = "myArg1";
arg[3] = "myArg2";
etc.

When I run my job, I get an error in in my map tasks in the jobtracker Admin webpage:


java.lang.RuntimeException: java.lang.ClassNotFoundException: <package.Name>.<functionNameOfMyMapper>

I don't know why the system can't find the mapper function.  It is in the same package as
my Job file and they should all be contained in the .war file that is deployed in tomcat.

Has anyone had success calling a map reduce job programmatically from within tomcat?

Thanks,
Duane

Mime
View raw message