hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Hawkins <tim.hawk...@bejant.com>
Subject New User Questions
Date Tue, 13 Jan 2009 00:19:16 GMT
I have just recently started working with hadoop and I have a few  
questions I would like to submit to the community.

1. Is there any remote-able interface to the jobtracker, so that I can  
create an application that would remotely be able to track jobs. Ie  
something like a rest or XMLRPC interface?.

2. Is it possible to attach additional metadata to a job for tracking  
purposes, we have a set of tasks that result in a large number of jobs  
per task, and I would like to tag jobs with an overall task id for  
tracking purposes.

A usage case would be a nutch crawl, our application uses nutch to  
scan domains on demand, each scan is a distinct task, and nutch spawns  
a set of mapred tasks for each stage of the crawl.

I would like to be able to determine completion status for a set of  
nutch crawls using an interface to job-tracker. 

View raw message