hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Phelps <...@opendns.com>
Subject Re: Getting (or setting) a job ID
Date Tue, 10 May 2011 23:47:46 GMT
We could make a change of that nature.  We currently use 
job.waitForCompletion() to submit jobs, if we switch to job.submit() is 
there an alternate blocking call to wait for the job to finish other 
than polling job.isComplete()?  I'm not currently seeing anything in the 
API.

- Adam

On 5/10/11 3:47 PM, Aaron Baff wrote:
> As part of the job submission, once it's submitted, grab the JobID from that object and
print it out on STDOUT or to a file and have your startup script(s) parse it out from there.
>
> --Aaron
>
> -----Original Message-----
> From: Adam Phelps [mailto:amp@opendns.com]
> Sent: Tuesday, May 10, 2011 3:45 PM
> To: mapreduce-user@hadoop.apache.org
> Subject: Getting (or setting) a job ID
>
> We have intermittently seen cases where a job will "freeze" for some as
> yet unknown reason, and thereby block other processes waiting for that
> job to complete.  I'm trying to modify our job-launching scripts to kill
> the job if it doesn't complete in a reasonable amount of time, however
> to do that I would need to know the job ID (ie along the lines of
> job_201104292017_9008).
>
> As far as I can tell there is no good way to map the name we use to
> create the job to the job ID from the command line (none of the "hadoop
> job" options do it).  Am I missing something in the CLI that would help
> with this?  Alternate ideas are also welcome.
>
> - Adam
>


Mime
View raw message