hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (MAPREDUCE-145) DiskChecker$DiskErrorException when 'reduce > reduce'
Date Mon, 21 Jul 2014 22:04:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-145?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer resolved MAPREDUCE-145.
----------------------------------------

    Resolution: Cannot Reproduce

> DiskChecker$DiskErrorException when 'reduce > reduce'
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-145
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-145
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>         Environment: hadoop-0.19.1; CentOS 5
>            Reporter: Leitao Guo
>
> We have  9900 maptasks and 60 reducetasks in the job. When all the other 59 reducetasks
have finished, the last reducetask runs so slow and finally finished after throwing out a
lot of DiskErrorExceptions.
> The following is the tasktracker log on which the reducetask is running. 
> 2009-03-18 14:39:52,025 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:39:57,028 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:00,695 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.977961% reduce > reduce
> 2009-03-18 14:40:02,032 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:07,036 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:12,040 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:17,045 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:22,050 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:27,054 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:32,058 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:37,062 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:42,066 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:47,136 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:52,140 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:40:57,143 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:02,147 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:06,760 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce
> 2009-03-18 14:41:07,152 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:09,762 INFO org.apache.hadoop.mapred.TaskTracker: attempt_200903171026_0091_r_000030_0
0.9788374% reduce > reduce
> 2009-03-18 14:41:12,158 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:17,162 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:22,168 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories
> 2009-03-18 14:41:27,172 INFO org.apache.hadoop.mapred.TaskTracker: org.apache.hadoop.util.DiskChecker$DiskErrorException:
Could not find taskTracker/jobcache/job_200903171026_0091/attempt_200903171026_0091_r_000030_0/output/file.out
in any of the configured local directories



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message