hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eric_...@yahoo.com
Subject submitting jobs from a webapp
Date Mon, 31 Jan 2011 15:09:43 GMT
Hi,

Sorry if this is a trivial question but I have been battling with it for a 
while.  I have a question about submitting jobs from a web application 
potentially deployed on different machines than HBase.  I am using 
springframework and implemented a simple JobService class that acts as a spring 
service.  I can then implement all my jobs in separate classes and pass them to 
my jobService for execution.  When I run jobs through the hadoop command line, 
the job shows up in the job tracker.  However, when I submit jobs through my 
webapp, it does not.  Since I can step through the execution of my job in my 
eclipse environment, I don't think this job is running in hadoop/hbase's 
distributed environment.  Is that right?  

1. In short, how can I submit a job from my webapp and view its progress on the 
jobtracker?  

2. Additionally, I would also like to be able to get this information to display 
it in my own web application (so that users can track the status of their 
running jobs).  How can I query the Hadoop job tracker?

Thanks a lot for your help,
-Eric

here is snipets of my code:

class myJob extends IJob {
...
   public Job createSubmittableJob(Configuration conf, String[] args)      
Path inputDir = new Path(inputDir);
job = new Job(conf, jobName);
job.setJarByClass(Uploader.class);
FileInputFormat.setInputPaths(job, inputDir);
job.setInputFormatClass(TextInputFormat.class);
job.setMapperClass(TransactionLoader.class);
TableMapReduceUtil.initTableReducerJob(tableName, null, job);
job.setNumReduceTasks(0);
    }
...
}

class myJobService {
...
      
   public int submitJob(String name, IJob zjob, String[] args) {      
      
job = zjob.createSubmittableJob(dsm.getHBaseConfiguration(), def, otherArgs);
// "Submit the job to the cluster and wait for it to finish."
return job.waitForCompletion(true) ? 0 : 1;
   }
...
}
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message