hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Armstrong <john.armstr...@ccri.com>
Subject Re: Jobs failing on submit
Date Fri, 26 Aug 2011 20:18:36 GMT
On Fri, 26 Aug 2011 12:20:47 -0700, Ramya Sunil <ramya@hortonworks.com>
wrote:
> Can you also post the configuration of the scheduler you are using? You
> might also want to check the jobtracker logs. It would help in further
> debugging.

Where would I find the scheduler configuration?  I haven't changed it, so
I assume I'm using the default.

This is what I see in the jobtracker logs when I submit the job:

2011-08-26 16:11:19,164 INFO org.apache.hadoop.mapred.JobTracker: Job
job_201108261610_0001 added successfully for user 'hdfs' to queue 'default'
2011-08-26 16:11:19,164 INFO org.apache.hadoop.mapred.JobTracker:
Initializing job_201108261610_0001
2011-08-26 16:11:19,164 INFO org.apache.hadoop.mapred.JobInProgress:
Initializing job_201108261610_0001
2011-08-26 16:11:19,165 INFO org.apache.hadoop.mapred.AuditLogger:
USER=hdfs	IP=127.0.0.1	OPERATION=SUBMIT_JOB	TARGET=job_201108261610_0001	RESULT=SUCCESS

Nothing shows up in the tasktracker logs when I submit the job.

> State "4" indicates that the job is still in the PREP state and not a
job
> failure. We have seen these kind of errors when either the cluster does
not
> have tasktrackers to run the tasks or when the queue to which the job is
> submitted does not have sufficient capacity.

So it's possible something has gone wrong with the job queue?  Is it
possible something's stuck in there?  How would I find it/clean it out?

> If you do not see this log message, that implies the cluster does not
have
> enough resources due to which JT is unable to schedule the tasks.

I do see this line in the TaskTracker logs; it might have something to do
with the problem, but I have no idea how to fix it.

2011-08-26 16:14:41,966 WARN org.apache.hadoop.mapred.TaskTracker:
TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is
disabled.

Thanks for the pointers.

Mime
View raw message