hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leitao Guo (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5524) DiskChecker$DiskErrorException when 'reduce > reduce'
Date Wed, 18 Mar 2009 07:19:50 GMT
DiskChecker$DiskErrorException when 'reduce > reduce'
-----------------------------------------------------

                 Key: HADOOP-5524
                 URL: https://issues.apache.org/jira/browse/HADOOP-5524
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.19.1
         Environment: hadoop-0.19.1; CentOS 5
            Reporter: Leitao Guo


We have  9900 maptasks and 60 reducetasks in the job. When all the other 59 reducetasks have
finished, the last reducetask runs so slow and finally finished after throwing out a lot of
DiskErrorExceptions.

The following is the tasktracker log on which the reducetask is running. 

2009-03-18 14:39:52,025 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:39:57,028 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories

2009-03-18 14:40:00,695 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.977961% reduce > reduce

2009-03-18 14:40:02,032 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:07,036 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:12,040 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:17,045 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:22,050 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:27,054 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:32,058 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:37,062 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:42,066 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:47,136 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:52,140 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:40:57,143 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:41:02,147 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories

2009-03-18 14:41:06,760 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce

2009-03-18 14:41:07,152 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:41:09,762 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce
2009-03-18 14:41:12,158 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:41:17,162 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:41:22,168 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
2009-03-18 14:41:27,172 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message