hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2960) A single TT disk failure can cause the job to fail
Date Sat, 05 Nov 2011 01:47:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144513#comment-13144513
] 

Eli Collins commented on MAPREDUCE-2960:
----------------------------------------

This affects the LTC as well. The log claims the job completed, however it was a wordcount
job and it never populated the output directory.

{noformat}
11/11/03 12:51:56 WARN mapred.JobClient: Error reading task outputhttp://eli-thi
nkpad:50316/tasklog?plaintext=true&attemptid=attempt_201111031152_0006_m_000001_
1&filter=stdout
11/11/03 12:51:56 WARN mapred.JobClient: Error reading task outputhttp://eli-thi
nkpad:50316/tasklog?plaintext=true&attemptid=attempt_201111031152_0006_m_000001_
1&filter=stderr
11/11/03 12:51:59 INFO mapred.JobClient: Task Id : attempt_201111031152_0006_m_0
00001_2, Status : FAILED
Error initializing attempt_201111031152_0006_m_000001_2:
java.io.IOException: Job initialization failed (255)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskC
ontroller.java:192)
        at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1231)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInforma
tion.java:1059)
        at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1
206)
        at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:112
1)
        at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2410)
        at java.lang.Thread.run(Thread.java:662)
Caused by: org.apache.hadoop.util.Shell$ExitCodeException: 
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
        at org.apache.hadoop.util.Shell.run(Shell.java:182)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
375)
        at org.apache.hadoop.mapred.LinuxTaskController.initializeJob(LinuxTaskC
ontroller.java:185)
        ... 8 more

11/11/03 12:51:59 WARN mapred.JobClient: Error reading task outputhttp://eli-thinkpad:50316/tasklog?plaintext=true&attemptid=attempt_201111031152_0006_m_000001_2&filter=stdout
11/11/03 12:51:59 WARN mapred.JobClient: Error reading task outputhttp://eli-thinkpad:50316/tasklog?plaintext=true&attemptid=attempt_201111031152_0006_m_000001_2&filter=stderr
11/11/03 12:52:02 INFO mapred.JobClient: Job complete: job_201111031152_0006
11/11/03 12:52:02 INFO mapred.JobClient: Counters: 4
11/11/03 12:52:02 INFO mapred.JobClient:   Job Counters 
11/11/03 12:52:02 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=0
11/11/03 12:52:02 INFO mapred.JobClient:     Total time spent by all reduces waiting after
reserving slots (ms)=0
11/11/03 12:52:02 INFO mapred.JobClient:     Total time spent by all maps waiting after reserving
slots (ms)=0
11/11/03 12:52:02 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=0
{noformat}
                
> A single TT disk failure can cause the job to fail
> --------------------------------------------------
>
>                 Key: MAPREDUCE-2960
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2960
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: tasktracker
>    Affects Versions: 0.20.204.0
>            Reporter: Eli Collins
>
> TaskInProgress#kill in the JT fails because TaskStatus#setFinishTimes fails because no
start time was set. There's no start time because TaskTracker#run (DefaultTaskController#initializeJob)
failed before it was set. The fix is to have TT#launchTask set the start time before it starts
the task runner, this way there's a valid start time even if TT#run fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message